Test-Based Accountability and International Comparisons: Lessons Ignored
Thus, the faith the U.S. has in education as a central institution for driving an internationally competitive economy is "[n]ot. . .link[ed] to reality." Likely, political and corporate leaders need the public to believe their claim in order to keep public schools focused on producing a compliant workforce, instead of allowing public education to fulfill its role in supporting human agency and democratic ideals.
â€˘ Even if the relationship between education quality and economic strength is not supported by the evidence, isnâ€™t the U.S. education system, as measured by PISA, lagging behind other nations, notably nations with much lower GDP? The basic charts offered by Tucker (2011) appear damning, but as Bracey (2008) warns, simple ranking of average test scores from single data points cannot offer a fair or accurate picture of much of anything of value concerning the quality of education in an entire countryâ€"particularly if we decontextualize that data from one important factor, poverty. Riddile (2010) presents a more nuanced analysis of PISA that compares apples to apples internationally by considering childhood poverty rates along with PISA data. The result shows that the U.S. sits at the top of ranking when poverty is considered:
Country/Poverty Rate/PISA Score
United States/ <10%/ 551
Finland/ 3.4%/ 536
Netherlands/ 9.0%/ 508
Belgium/ 6.7%/ 506
United States/10%-24.9%/ 527Canada/ 13.6%/ 524
New Zealand/ 16.3%/ 521
Japan/ 14.3%/ 520
Australia/ 11.6%/ 515
Poland/ 14.5%/ 500
Germany/10.9% / 497
Riddile also notes more problems with simplistic international comparisons by addressing Shanghai (Zhao, 2010):
"Shanghai, China topped the list with 556 but is not included in this analysis because Shanghai is a city not a country and because only 35% of Chinese students ever enter high school and because 'when you spend all your time preparing for tests, and when students are selected based on their test-taking abilities, you get outstanding test scores.'"
The two most repeated and compelling claims about education in the U.S., then, are factually inaccurate; thus, we have to be skeptical at best about Tucker (2011) pursuing an extending discussion of the U.S. adopting practices from other countries in order to reform educationâ€"even if we maintain the tenuous and distorting assumption that the primary purpose of education is to prepare students as future workers. But, Tucker's central goal for his report, to suggest how the U.S. can and should model education reform on successful international comparisons, does provide further evidence that we have misguided assumptions about the purpose of education, and thus are prone to continue pursuing flawed policies for reform.
The first focus offered by Tucker (2011) is addressing quality, primarily teacher quality. While this has been the central argument for the "no excuses" segment of the new reformers led by Secretary Duncan and Gates, the claim has some serious problems. First, discussions of teacher quality suffer the similar fate that international comparisons experience, oversimplification. Teacher quality is difficult to measure, just as student learning is, but assuming that the best students make the best teachers is at least debatable (Sears, Marshall, & Otis-Wilborn, 1994). Further, the argument that teacher quality must be increased to reform education rests on another flawed assumptionâ€"the impact of teacher quality on student outcomes. Sawchuk (2011) details the current understanding of teacher quality and student outcomes, concluding:
The in-school influence of teachers on student outcomes is considerably smallâ€"about 14% ( Hirsch, 2007) or 13%-17% (Hanushek, 2010). Thus, even if we accept elite students make elite teachers, and thus we need to recruit high-achieving students into the teaching profession, we are tinkering with a very small measurable influence on the exact data we are using to evaluate schools.What Tucker (2011) approaches but never fully addresses within the larger and misleading call to focus on teacher quality is how many countries he labels superior to the U.S. do treat, educate, and pay teachers. For one example, letâ€™s consider Finlandâ€"where teachers are required to complete a publicly-funded masters degree, where teachers are universally unionized, where teachers are not held accountable for standards or test scores ( Horn, 2010, November 22). While Tucker's echoing of Duncan and Gates the need for greater teacher quality is refuted by evidence, it seems likely that the U.S. could benefit from reconsidering how we treat, educate, and pay teachers, but the details of that consideration works against many of the commitments of corporate reformers. [emphasis added]
Equity, the second focus presented by Tucker (2011), appears justified by, ironically, the flaws inherent in his initial premise based on raw comparisons of PISA data. While the argument for equity focuses on in-school equity without considering the need to address social inequityâ€"the primary reform needed in the U.S.â€"Tucker raises important questions about stratified course offerings (tracking), teacher assignments (students coming from poverty tend to have least experienced and un- or under-qualified teachers), and the potential for schools to address high-poverty students more effectively than we currently do in the U.S.
While the equity section from Tucker (2011) has the most potential for being valuable in the education reform debate, we should note, again, that out-of-school equity is essentially ignored and that this section is dwarfed by the other two sections, the teacher quality section being seven times in length and the final section being just a bit more than double the equity section. Yes, equity matters, but Tucker's discussion helps highlight that most new reformers persist in undervaluing the impact of social inequity as well as the greatest flaw in public educationâ€"that school perpetuates the inequity children experience in their lives outside formal education.
The final section, productivity, reveals the corporate commitments driving the report. Tucker (2011) endorses the business model for reforming and running schools, accountability, and merit-based incentives for teachers and schools to perform. Again, even if we remain within the assumptions about international comparisons, test scores, and the relationship between education and economies, Tucker's third focus is not based on evidence. Hout and Elliott (2011), using test scores and international comparisons, conclude that the current and prolonged accountability era driven by standards, testing, and high accountability "have not increased student achievement enough to bring the United States close to the levels of the highest achieving countries" (p. S-3). What we should insert here is that nearly three decades of focusing on high accountability in order to raise U.S. test scores to compete internationally has produced two conclusions the new reformers will not acknowledge: (1) Test scores remain most strongly connected to out-of-school factors, and (2) accountability paradigms do not work.
Accountability has never worked, but neither has merit-based incentive programs. Kohn (2003) has discredited merit pay at both the corporate and education levels, concluding about merit pay for teachers:
Amen. [Ohanian comment]
As well, the merit pay argument fails for the same reason international comparisons based on test scores failâ€"test scores and linking those scores to teacher quality are unstable, misleading, and corrosive to the teaching/learning dynamic ( Baker, et al. 2010).
The historical and current education reform movement, then, is doomed to fail (yet again) as long as leaders persist in placing tests at the center of determining education quality, international comparisons, and recruiting, preparing, and paying teachers. The reform movement is also futile as long as schools remain primarily to train a compliant workforce, as long as schools are managed like businesses, and as long as accountability drives that reform regardless of decades of evidence that standards, testing, and accountability do not work.
Finally, and most importantly, education reform will always fail students and our society if we fail to learn the lessons taught by international comparisons and testingâ€"the weight and impact of a child's life is the central issue of equity our culture must address as we also commit fully to equitable schools. To maintain tunnel vision on schools and simplistic use of data serve only the privileged at the expense of everyone else.
Baker, E. L., Barton, P. E., Darling-Hammond, L., Haertel, E., Ladd, H. F., Linn, R. L., Ravitch, D., Rothstein, R., Shavelson, R. J., & Shepard, L. A. (2010, August 20). Problems with the use of student test scores to evaluate teachers. EPI Briefing Paper #278. Washington DC: Economic Policy Institute. Retrieved 12 June 2011 from http://www.epi.org/publications/entry/bp278
Bracey, G. W. (2008, December 9). International Comparisons: More Fizzle than Fizz. Huffington Post. Retrieved 10 January 2011 from http://www.huffingtonpost.com/gerald-bracey/international-comparisons_b_149690.html
Bracey, G. W. (2004). Setting the record straight: Responses to misconceptions about public education in the U. S. Portsmouth, NH: Heinemann.
Hanushek, E. (2010, December). The economic value of higher teacher quality. Working paper 56. Washington, DC: Calder, The Urban Institute.
Hirsch, D. (2007, September). Experiences of poverty and educational disadvantage. Joseph Rowntree Foundation. York, North Yorkshire, UK. Retrieved 27 December 2007 from http://www.jrf.org.uk/knowledge/findings/socialpolicy/2123.asp
Horn, J. (2010, November 22). U.S. education reformersâ€™ cartoon version of Finlandâ€™s teacher education system. Schools Matter [blog]. Retrieved 8 March 2011 from http://www.schoolsmatter.info/2010/11/u-s-education-reformers-cartoon-version.html
Hout, M., & Elliott, S. W. (2011). Incentives and test-based accountability in education. Washington DC: The National Academies Press. Retrieved 23 June 2011 from http://www.nap.edu/catalog.php?record_id=12521
Kohn, A. (2003, September 17). The folly of merit pay. Education Week. Retrieved 7 July 2011 from http://www.alfiekohn.org/teaching/edweek/meritpay.htm
Riddile, M. (2010, December 15). PISA: Itâ€™s poverty not stupid. The Principal Difference [blog]. Retrieved 11 January 2011 from http://nasspblogs.org/principaldifference/2010/12/pisa_its_poverty_not_stupid_1.html
Sawchuk, S. (2011). EWA research brief: What studies say about teacher effectiveness. Washington DC: Education Writers Association. Retrieved 7 July 2011 from http://www.ewa.org/site/PageServer?pagename=research_teacher_effectiveness
Sears, J. T., Marshall, J. D., & Otis-Wilborn, A. (1994). When best doesnâ€™t equal good: Education reform and teacher recruitment, a longitudinal study. New York: Teachers College Press.
Tucker, M. S. (2011, May 24). Standing on the shoulders of giants: An American agenda for education reform. Washington DC: National Center on Education and the Economy. Retrieved 7 July 2011 from http://www.ncee.org/wp-content/uploads/2011/05/Standing-on-the-Shoulders-of-Giants-An-American-Agenda-for-Education-Reform.pdf
Zhao, Y. (2010, December 10). A true wake-up call for Arne Duncan: The real reason behind Chinese students top PISA performance. Yong Zhao [blog]. Retrieved 11 January 2011 from http://zhaolearning.com/2010/12/10/a-true-wake-up-call-for-arne-duncan-the-real-reason-behind-chinese-students-top-pisa-performance/
FAIR USE NOTICE
This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of education issues vital to a democracy. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information click here. If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner.