in the collection
Organizational Improvement and Accountability Lessons for Education from Other Sectors
Ohanian Comment: Below is a summary of the Rand study. The url for the full document is given below. The idea that educators have something to learn from the health care system in this country is rather mind-boggling. And if that doesn't boggle you enough, the study also explains what teachers can learn from lawyers and from lean production methodology.
According to this report,"the most thoroughly scientifically validated consensus development method in health care (although not commonly used) is one called the modified Delphi method, pioneered by the RAND Corporation in the 1970s." Conservatives have been warning against the Delphi technique for decades.
The William and Flora Hewlett Foundation
Performance-Based Accountability in Education
In December 2001, the U.S. Congress approved a reauthorization of the Elementary and Secondary Education Act (ESEA) and renamed it the “No Child Left Behind Act” (P.L. 107-110, H.R. 1). The cornerstone of the No Child Left Behind Act (NCLB) is a performancebased accountability system built around student test results. This
increased emphasis on accountability represents an important change from past federal educational initiatives, which focused primarily on the provision of services. Supporters of NCLB argued that previous
educational reforms were unsuccessful in large measure because they ignored student outcomes. Borrowing from successful private-sector
management practices, they made the case that student achievement would only improve when educators were judged in terms of student
performance and consequences were attached to the results.
Three basic elements make up the performance-based accountability system required by NCLB: goals; assessments for measuring the attainment of goals and judging success; and consequences (rewards or sanctions). The goals are embodied in a set of content or performance standards that schools and teachers use to guide curriculum and instruction. Tests are developed to measure student learning and determine whether students have mastered the standards.
Improved performance on the tests leads to rewards that are intended to reinforce effective behavior; poor performance on the
tests leads to sanctions and improvement efforts that are intended to modify ineffective behavior. Some of the incentives operate through
parents. If a child’s school is deemed to be in need of improvement, parents can request a transfer to another school and/or supplemental
educational services from private providers.
As clear as these procedures may seem, the key principles underlying NCLB accountability are largely untested in education. The mechanisms through which the system is intended to work to improve student achievement and eliminate failing schools are not well understood. In this environment, decisionmakers at the state, district, and school levels are looking for guidance to help them make their systems as effective as possible. One place to look for possible insights into effective accountability mechanisms is outside the educational
sector. The purpose of this project is to examine accountability in other fields to find lessons that might be relevant for educators.
Accountability in Other Sectors
We cast our net widely before selecting specific instances of accountability to study. We solicited recommendations from educational researchers as well as research colleagues who study organizations in other fields. We also reviewed the debate within education surrounding
the passage of NCLB for references to accountability in other domains. The final set of cases reflects our desire to present examples
that are relevant, interesting, and diverse. Our sample includes cases from both the manufacturing and service sectors. In each case,
we tried to understand the processes through which providers are held accountable, how well these processes have worked, and whether
they might be applicable to education.
We examined five accountability models:
• Two accountability models drawn from the manufacturing sector (although now spreading to service industries): the Malcolm Baldrige National Quality Award Program and the Toyota Production System (TPS). Strictly speaking, these are models of organizational improvement set within the larger context of market accountability rather than full-fledged accountability systems. Both, however, offer ways to improve organizational efficiency.
• A performance incentive model used in the evaluation of job training programs for the poor established by the Job Training Partnership Act (JTPA) of 1982 (now replaced by the Workforce Investment Act (WIA) of 1998).
• Accountability in the legal sector. The legal accountability model is largely based on notions of “professional accountability,”
which entail controlling entry into the profession, mandatory capacity-building, self-policing, and protecting client concerns.
• Accountability in health care. We explored three aspects of health care accountability that seemed particularly relevant for education: clinical practice guidelines, use of statistical risk adjustment methods, and the public reporting of health performance
These models differ in terms of their comprehensiveness, effectiveness, and applicability to education. In this monograph, we describe each model, summarize the relevant research on effectiveness, and draw specific lessons for educators.
Implications for Education
We recognize that the education sector has unique characteristics that set it apart from the other sectors we examined. Yet we believe the analyses of these different accountability models offer useful insights on ways to enhance system-wide accountability in education, including how to improve the operation of schools and districts to achieve higher performance. Specific lessons learned for education include the following:
Broaden performance measures. Educators should be careful when setting performance objectives because the objectives will drive behavior
—for better or for worse. Broadening “what counts” in the system is one way to diffuse the pressure to focus too narrowly and to
deemphasize other important priorities.
Make sure performance goals are fair to all students and schools.
The accountability system should establish reasonable improvement targets for all schools and should not reward or penalize schools or
districts for factors beyond their control. The goal of fair comparisons also needs to be balanced against the goal of closing the gap between successful and unsuccessful students. Nevertheless, the experiences of JTPA/WIA and health point out the advantages of performance targets that are sensitive to initial inputs.
Develop standards of practice in promising areas and encourage professional accountability. Movements to create more-explicit standards of practice would foster professional accountability and provide
guidance to help schools and districts improve their performance. We encourage educators to select promising areas in which more-detailed
practice guidelines might be developed. Such guidelines can form the basis for more-detailed standards for the teaching profession so teachers can be more aggressive about monitoring their own professional competence. These steps would help broaden and deepen accountability in education. Develop an integrated, comprehensive strategy to help schools and districts improve their performance. This research points to four key
elements of an improvement strategy:
• Undertaking a focused institutional self-assessment (including asking the right questions and assembling the right kinds of information)
• Understanding the school system as a linked process
• Developing and applying an expanded knowledge base about effective practice in varying situations
• Empowering participants in the process (notably teachers) to contribute to improvement efforts. Developing and adopting such a strategy in education will require time, effort, and a willingness to adapt principles from outside the educational sector. Pilot efforts to adapt and test these components in diverse schools settings and focused efforts to create educational applications would be a good starting place to try to take advantage of the successful experiences of other sectors.
This investigation of accountability in other sectors sharpens our thinking about accountability in education. It suggests ways in which educators can develop better strategies for improving the performance
of schools and districts and policymakers can redefine educational accountability to make it more effective. It is worth pointing out that,
although education has much in common with business, law, and health care, it faces unique challenges that other sectors do not face.
Nevertheless, educators have much to learn from these other fields. In the end, they will have to develop an accountability model that addresses their unique situation. However, there is much they can draw on from accountability efforts outside of education.
Brian Stecher and Sheila Nataraj Kirby, editors
INDEX OF THE EGGPLANT