in the collection
Atlanta J-C Asks What Exam Makers' History of Mistakes Will Mean with Enormity of Testing Under NCLB 2001: Harcourt Educational Measurement fails to provide results of the Stanford 9 test in time to be used to assess students before the end of the school year.
2002: Harcourt made errors while grading exams, affecting the proper placement of students throughout the state.
March 27, 2003: Questions used on practice exams for the Criterion-Referenced Competency Test appear on the actual tests given to students. State officials say one of two testing firms, Measured Progress or Riverside Publishing, is at fault.
Some high school seniors in Minnesota couldn't graduate with their classmates.
More than 8,000 students in New York City were improperly sent to summer school.
Closer to home, nearly 600,000 Georgia students won't take the state's most important standardized test this year.
Don't blame the students, the parents, the teachers or even the government. The fault lies with the private companies that created or scored the different standardized tests. Over the past few years, the results of several other standardized tests across the nation also have been tainted by mistakes made by private testing companies.
And as these tests come to mean more under President Bush's No Child Left Behind act, some educators and advocates are getting unnerved by the errors.
It's an industry "over which there is less regulation . . . than there is over what goes in your pet's food," said Robert Schaeffer of the National Center for Fair and Open Testing, which opposes using standardized exams as the sole indicator of success. "The problem is there is no Underwriters Lab or Consumer Reports or Food and Drug Administration that certifies that these tests are safe and effective. There's no particular license or, oddly, no test someone has to pass."
Under the federal No Child Left Behind act, all school systems must have a way to assess the progress of students in grades three through eight by 2005-06. That's nearly 22 million students nationwide.
For most states, that will be done through high-stakes testing, a battery of standardized exams that are supposed to be aligned to the state's curriculum. The tests will be used to judge each school's progress. Poor-performing schools could receive a range of sanctions, while high-performing schools could be rewarded.
No Child Left Behind has created a boom in the number of tests needed and has put further demands on the testing industry, which is dominated by a handful of big companies and some smaller competitors.
"We're kind of in a position of competing for the contractors, which is backwards," said David Harmon, director of testing and assessment for the Georgia Department of Education.
Georgia has had its share of problems with standardized testing and the companies that create the exams.
For two years in a row, Harcourt Educational Measurement made mistakes with the Stanford 9 --- a standardized test used to compare third-, fifth- and eighth-graders around the country and assess student achievement.
In 2001, Harcourt was more than a month late delivering the results for Georgia test-takers. Then, in 2002, the company made errors while grading the exams taken by Georgia students. The mistakes affected the placement of students throughout the state, including at least 200 children who couldn't participate in the admission lottery to attend the coveted Kittredge Magnet School in DeKalb County.
Most recently, on March 27, state Superintendent of Schools Kathy Cox announced that some questions used on practice exams for the Criterion-Referenced Competency Test, the state's curriculum test, had appeared on the actual tests that were to be given to students this month. The tests were scotched in all grades except four, six and eight, and the state is still sorting out how the error occurred.
Georgia education officials say one of two testing companies --- Measured Progress or Riverside Publishing --- is responsible. Officials from Measured Progress said they are helping the state investigate and expect some type of answer next week. Officials from Riverside Publishing, a subsidiary of publishing giant Houghton Mifflin, would not comment.
Meanwhile, teachers who had been preparing students for these tests all year won't have the chance to give the exams.
Susan Padgett-Harrison, director for the office of assessment in Cherokee County, thought students and teachers would be glad the tests were canceled. "Instead, my e-mail box was flooded with communications from teachers and parents and principals saying, 'We wanted to take the test. We wanted to show how well we were doing.' "
Last year, Georgia schools gave the CRCT to grades one through eight --- more grades than most states are testing. But because of this year's problems, educators won't be able to track the progress of most of these students.
And next year, the stakes will be higher.
Under state law, third-graders who don't score high enough on certain portions of the curriculum test next year won't get promoted. In coming years, students in fifth and eighth grades will face similar standards.
The state is also set to begin assigning schools a grade of A, B, C, D or F next year. And federal law mandates that states report which schools are improving and which are not. Both require that the curriculum test be the main barometer of success.
Good grades could lead to more money and flexibility for school systems, while poor marks could lead to mandated improvements or a state takeover. Georgia still can meet the federal requirements this year by giving the test to just grades four, six and eight, officials said.
"That's not the optimum situation we wanted," said Nick Smith, a spokesman for the state Department of Education. "In fact, Georgia was ahead of the game. Now, we're maintaining the pace."
Georgia isn't the only state having problems. Over the past several years, snafus have originated with the standardized testing industry in Florida, New Mexico, California, Nevada and other states.
In Minnesota, NCS Pearson, a large testing company, wrongly scored thousands of Minnesota Basic Skills Tests in 2000. Some students were forced to go to summer school before the errors were discovered. Others weren't allowed to graduate on time. A class-action lawsuit was filed, and last year NCS Pearson agreed to a multimillion-dollar settlement with students.
In 1999, about 8,600 New York City students were wrongly sent to summer school after an error by testing company CTB/ McGraw-Hill.
The companies see these as isolated incidents given the number of tests that are administered.
"Our company processes 10 million student tests per year without a hitch," said Rick Blake, a spokesman for Harcourt. "There have been errors, but they've been relatively infrequent."
Too much business?
The issue might be as simple as supply and demand.
With "accountability" the buzzword in education, the demand for testing has skyrocketed during the last several years. Unlike Georgia, some states were not giving curriculum tests and have had to scramble to develop programs.
"Because of accountability and the movement across the country with No Child Left Behind, we have relied on two or three very large companies for testing needs," said Ellen Cohan, associate superintendent for teaching and learning with the Forsyth County school system. "It must be overwhelming for them."
Only a handful of large companies administer, score and report test results, including Riverside, Harcourt, NCS Pearson and CTB/McGraw-Hill. There are some smaller operations as well.
Stuart Kahl, president of Measured Progress, said the testing industry has started to expand to meet the demand.
"There were many companies that were niche players," Kahl said. "A lot of those players who were already in the industry have already developed full capacity."
For instance, Kahl pointed to the Educational Testing Service. The Princeton, N.J.-based company once specialized in higher education entrance exams such as the SAT but has now entered the k-12 testing business.
Kahl said the biggest challenge facing the industry is time. He said the companies need to create individualized tests based on each state's curriculum. The companies used to have years to develop tests, he said, but that's changed.
"Instead of taking three or four years to develop an assessment system, you've got three or four months," Kahl said.
Harmon, Georgia's testing director, said it's important for the state to have more choices. For instance, he said the state Department of Education is looking into whether Georgia universities could get involved in administering and scoring the tests.
"It will be a challenge," Harmon said, "but still, we have to have more alternatives."
Schaeffer, of the National Center for Fair and Open Testing, said states need to build strict performance standards into contracts with testing companies and follow through with sanctions if mistakes occur.
He also suggested that the industry needs to be regulated in some manner.
"There needs to be a system of public oversight and control over this industry, which plays an exceptionally important role," Schaeffer said. "You need a national agency."
The federal Department of Education has no plans to get into the business of monitoring the industry. Dan Langan, a spokesman for the U.S. Education Department, said it's up to the states to make sure tests are administered and scored accurately.
Maureen DiMarco, senior vice president for Houghton Mifflin, said she is confident the testing industry is ready to handle all the work. But she realizes a lot is riding on the tests.
"It's like any industry that has an incredibly high safety record, but there's been an accident," she said. "Any accident is too many."
PROBLEM RESULTS: Georgia has had problems with standardized testing companies in recent years:
Exam's pressure not just on kids
April 14, 2003
INDEX OF THE EGGPLANT