|
|
9486 in the collection
How the New York Math Exam Went So Wrong
When thousands of high school students failed this month's Math A Regents exam, complaints from Buffalo to Oyster Bay poured in to the state Education Department.
In days, the uproar grew so strong that the education commissioner acquiesced. He not only threw out the test results, he canceled the August exam and called for a panel to investigate what went wrong.
But educators say flawed Regents exams are nothing new.
"The good news about this exam is that it's so bad that at last someone has to pay attention to what's going on at State Ed," said Betty Krist, who directs the University at Buffalo's math program for gifted high school students. "Everything that's bad about this exam has happened before with other exams."
This year, things seemed to come to a head, with an unusually large number of complaints and problems with the Math A exam. The impact was more painful for more people, too, because this is the first year every student in the state had to pass this math exam to graduate.
Officials from the state Education Department say it's too early to tell what was wrong with the exam, if anything. It's possible the huge outcry from teachers this year was related to the sheer number of students taking it, said Roseanne DeFabio, assistant education commissioner for curriculum, instruction and assessment.
"When a test is new, we get a lot more response," she said, "or when a test is difficult for their students."
Educators say two main factors contributed to the high failure rate for the June exam.
For one thing, those who took the exam in June generally were not the best students. More accelerated students took the exam in January, after studying the course for a year and a half. Students who needed more help took two years, or even three, to complete the course, and took the exam in June.
Beyond that, many teachers speculated that the process has been hurt by the recent downsizing of the state Education Department - and specifically the math bureau.
State education officials said they developed the Math A exam the same way they create all Regents exams, employing a lengthy process that involved many checks and balances.
"I can't tell you anything was really done differently in terms of the process," said DeFabio.
Riddled with problems
The state spent more than 18 months and thousands of dollars to develop the test. Dozens of teachers across New York contributed questions. Hundreds of students pilot-tested the exam. Statisticians spent weeks analyzing the results to ensure a proper distribution of questions.
Despite all that work, the test was riddled with problems, teachers say. Local math experts say a number of things contributed to the weakness:
• Two major segments of the course, trigonometry and equation systems, were not covered in the exam, while more emphasis was given to material from other parts of the course.
• At least one of the multiple-choice questions had more than one possible correct answer.
• The wording of many problems made it difficult to figure out what the question was about.
• In an effort to incorporate math problems into everyday settings, authors of the test crafted some situations that were so outrageous as to be "foolish," one math expert said.
• The rubrics, or grading guidelines, the state provided to teachers who scored the test are so stringent that there's virtually no leeway for teachers to reward students for thinking creatively.
• Because graphing calculators were allowed but not required, some students had a time advantage during the three-hour test.
"It's a no-brainer to say it was a bad test, to say it was a badly constructed test," said Tom M. Giambrone, a professor of mathematics at Buffalo State College. "They have to be more careful about the way these tests are constructed."
Previous complaints
This isn't the first time a Regents exam has drawn complaints.
Last year, educators complained that the physics exam was graded too harshly and asked Commissioner Richard P. Mills to reconsider the rating scale. He refused.
The New York State Council of School Superintendents then sent letters to colleges, asking them to disregard the low scores on the physics exam and calling the grades "suspect."
Then, in February, the Association of Mathematics Teachers of New York State sent a letter to the state detailing four "egregious" errors on the Math A exam students took in January, and five mistakes on the Math B exam.
The group got a response from the state thanking it for the input and noting that the department altered one question on the Math B test, allowing teachers to award credit for a second correct answer.
Beyond that, though, the state did not respond to the group's specific complaints.
"My concern is we have seen with this new Math A assessment, over the past (four years), minor problems with the test: incorrect mathematics, more than one correct answer, problems with the wording," said Bob Hazen, president of the Association of Mathematics Teachers of New York State. "I'm not sure how or if we can get rid of those minor problems that show up."
He believes that the test is evidence of a state Education Department that has too few people.
The state has issued corrections to one multiple-choice question on the Math A exam that had at least two acceptable answers, and a correction to a Korean translation of another exam, according to DeFabio.
All the problems that have cropped up in smaller ways on other exams seemed to descend en masse on the Math A exam this month.
Last year, teachers found fewer geometry questions on the Math A exam than they expected. This year, they found no questions that test students' knowledge of trigonometry or equation systems.
"Our teachers probably spent 25 to 30 percent of the year on those two things, and no questions were on the final," said Gary R. Cooper, superintendent of the Frontier schools. "The material that was being tested in many cases left out a lot that was being taught."
Multiple answers
Even more troubling to some people, though, was what was on the exam.
One of the multiple-choice questions, No. 14, turned out to have three correct answers instead of one. The problem, designed to test students' ability to perform operations in the right order, failed to account for the fact that the correct solution could be attained in various ways.
"If I was doing it with a calculator, I would do it one way," said Hazen. "If I was doing it in my head, I would do it another way. If I were doing it on paper, I would do it another way."
The state eventually allowed two possible answers to that question. The third correct answer, though, still does not earn credit for a student. (The full text of the exam and the answer key can be found online at www.nysedregents org/testing/mathre/regentmatha.
html.)
Difficult to understand
The people who wrote the test questions - teachers and college professors trained and commissioned by the state - went to great lengths to incorporate math problems into everyday situations. But sometimes, educators said, the explanation for a single item got so wordy it was hard to tell what they were after.
One question, for instance, paints a scenario in which students must determine which of two arrangements of letters and numbers would allow a state to issue more license plates. The wordiness muddled the meaning of the question, some said.
"It's really a pretty ordinary question - if you live long enough to get through all this reading," said UB's Krist.
In a similar vein, the test sometimes went overboard to try to weave math into ordinary situations, leading to "foolish" questions, Krist said.
One of the most infamous questions from the exam, for example, asked students to imagine a straw resting diagonally in a rectangular box. The point of the problem is to get the students to employ the Pythagorean theorem to figure out the length of the straw - but many teachers wonder where the relevance is to real life, why anyone would ever want to put a straw through a box.
"Some of these contexts are just so contrived, it's hard to make sense of it," Krist said. "When you read through this, the first thing that hits you is, "This is the goofiest thing I can imagine.'
"Then you have to go back through it. It's that kind of thing that is undermining math. Math isn't about answering stupid questions."
Added Jack Blanch, chairman of the math department at Kenmore East High School: "I think we're all curious as to how a couple of these questions got through the field test."
DeFabio, the assistant education commissioner, said the word problems were designed to "show a real-life application. There will be differences of opinion of what's good mathematical reasoning."
Unfair advantage?
Besides controversies over the questions are debates over how students can take the test: They may use a graphing calculator, but are not required to use one. At the minimum, a scientific calculator is required.
But those $100 graphing calculators give students who own one - or whose school supplies them - a distinct advantage with any questions that require visualizing a graph, teachers said.
"The equity issues are huge. It's just as if you were going to work - driving a car versus walking versus riding a bicycle," Krist said. "How much time you spend would be an issue, and how much effort you put in would be an issue. It's equity in the 21st century."
State officials said all the questions can be solved using a scientific calculator.
"The test is designed so it's not supposed to give an advantage to a student with a graphing calculator," DeFabio said.
DeFabio said the state followed all the standard procedures in creating this month's Math A exam.
Teachers from across the state submitted questions of different types - multiple choice, short answer and extended answer - covering various aspects of the course material. Those questions are screened for sensitivity, accuracy and readability. A committee of teachers and mathematicians reviews the questions, revising some and discarding others.
Ultimately, questions are organized into field tests, which are sent to a cross section of schools across the state about a month before students take the actual Regents exam. Teachers give the tests to the classes of their choosing, then submit the results to the state.
An outside company is hired to analyze the results and to create the scales that will be used to grade the exam.
The questions are screened again; this time they're either used or thrown out. The questions that remain are used on the actual Regents exam.
"It's a very scientific and complex process," DeFabio said.
In all, the process takes at least 18 months and costs tens of thousands of dollars, although the state could not determine exactly how much it spends each year on exam development.
While the state reviews the exam, debate rages on among teachers, students and parents.
Some educators concede there might have been some problems on this particular exam, but overall, it's indicative of the more challenging standards and of a state that's beginning to make math mandatory for students of all abilities and interests.
"You can blame it on the test all you want," said Richard Timbs, superintendent of the Erie 2 Board of Cooperative Educational Services. "But even if the rubrics were broader, the exam was shorter, the point is there are still a number of students who can't do mathematics very well."
— Mary Pasiack
How the Math A exam went so wrong
Buffalo News
June 30, 2003
http://www.buffalonews.com/editorial/20030629/1038463.asp
INDEX OF OUTRAGES
Pages: 380
[1] 2 3 4 5 6 Next >> Last >>
|
