|
|
9486 in the collection
Suspicious test scores widespread in Georgia: Findings follow more than a year of AJC reporting about CRCT results
Ohanian Comment: Statewide, more than half of elementary and middle schools had at least one classroom where erasure marks were so unusual that cheating may have occurred...
I know little about Governor Perdue, but when any politico starts talking about the need for things being impeccable beyond reproach, I want to go count the silver spoons that belonged to my great-grandmother. Since Governor Perdue stood on the capitol steps during the drought of 2007 and prayed for rain, maybe now he should get out on those steps and pray for truth in testing.
Of Interest: Note how the Center for an Educated Georgia is identified simply as a nonprofit. Indeed. Now read a bit of the director's bio from the organization website:
Ben Scafidi . . . a Fellow with the Friedman Foundation for Educational Choice, chair of Georgia's Charter Commission, and is a member of Georgia's Charter Advisory Committee. He was the Principal Investigator on "The Taxpayer Costs of Divorce and Unwed Childbearing" study released by Georgia Family Council and other organizations.
I agree with former teacher Donna Williams. When you do nothing about unremitting poverty and at the same time tell teachers their pay and their very jobs depend on test scores, then what does any sane person think will happen? The solution isn't to add resources to catch cheaters; the solution is to ask what kinds of testing would actually help students--and, more importantly, to ask what children really need.
I would suggest that the children in the state of Georgia would be much better off if the state conducted an extensive analysis of lead poisoning instead of putting so much faith in standardized testing. Michael Martin presents solid evidence of the harm that comes from ignoring this threat.
In her blog, Maureen Downey asks, "How can parents know if their kids are learning if test results are not valid?" I'd ask How can parents know if their kids are learning if test results are valid? Has she looked at the test? I am unwilling to cede that kind of confidence to CTB/McGraw-Hill.
Downey also calls this changing answers on tests "educational malpractice of the worst kind." Educational malpractice? Yes. Of the worst kind? Not even close.
Am I condoning cheating? Of course not. "Understanding" and even "sympathizing" is not the same as "condoning. But what I am saying is:
1) CTB/McGraw-Hill tests are never going to tell you what you need to know about children's abilities.
2) The time spent prepping for CTB/McGraw-Hill tests would be much better spent doing other things.
3)The money spent on CTB/McGraw-Hill tests would be much better spent elsewhere.
By Heather Vogell and John Perry
One in five Georgia public schools faces accusations of tampering with student answers on last spring's state standardized tests, officials said Wednesday, throwing the state's main academic measure into turmoil.
The Atlanta district is home to 58 of the 191 schools statewide that are likely to undergo investigations into potential cheating. Another 178 schools will probably see new test security mandates, such as stepped-up monitoring during testing.
The findings singled out 69 percent of Atlanta elementary and middle schools -- far more than any other district -- as needing formal probes into possible tampering.
Wednesday, Atlanta Superintendent Beverly Hall said the new findings do not prove scores were falsified. The district will investigate, she said.
"I'm concerned with the number of schools and convinced we should do a detailed analysis to get to the bottom of this," said Hall.
After a more limited state testing probe that named an Atlanta school last year, Gov. Sonny Perdue said Hall ignored strong evidence of cheating.
The state Board of Education heard details Wednesday of the findings, which follow more than a year of stories by The Atlanta Journal-Constitution about suspicious scores.
Perdue said the state's education system must take steps to salvage the credibility of the state Criterion-Referenced Competency Test, the primary measure of achievement in grades one through eight.
"This has got to be impeccable beyond reproach," Perdue said of the test. "The sad thing is that it really is the students that are being cheated."
Statewide, more than half of elementary and middle schools had at least one classroom where erasure marks were so unusual that cheating may have occurred, the analysis shows. State and local officials will take follow-up steps in the 20 percent of schools with the most classrooms in question.
The extent of the suspicious answer changes is stunning, said Gregory Cizek, a University of North Carolina testing expert. He has studied cheating for more than a decade, but said he doesn't know of another state that has detected so many potential problems.
"I'm sort of speechless,' he said. “You hear about it widespread within a school sometimes, but nothing like this.”
He praised state officials for taking action. "Kudos to Georgia for getting serious about this."
Wrong-to-right changes
The findings came from a statewide analysis of erasure marks on student answer sheets that the state's testing contractor conducted for free, state officials said.
The company looked for high numbers of wrong answers that were erased and changed to right answers on reading, English and math tests. The analysis highlighted classrooms where the number of changed answers per student diverged greatly from the state average.
At Atlanta's Gideons Elementary, for instance, an average of 27 of 70 answers on each fourth-grader's math test were changed from wrong to right in one classroom. At the city's now-closed Blalock Elementary, an average of 26 of 70 answers on the fifth-grade math test were erased and corrected.
Some teachers give tests in more than one subject. Each subject test was counted separately.
Schools where suspicious erasures showed up in 5 percent or fewer classrooms were categorized as "clear" by the Governor's Office of Student Achievement, which led the probe. Schools with 6 percent to 10 percent of classrooms in question were labeled of "minimal concern." The achievement office is recommending monitoring or the rotating of teachers during testing for such schools.
The next two categories of schools are likely to see the strongest actions, including local investigations into answer-changing with results submitted to the state.
Schools of "moderate concern" had 11 percent to 24 percent of classrooms exhibiting suspicious erasures. In schools of "severe concern," 25 percent or more of classrooms showed highly unlikely percentages of wrong-to-right changes.
The achievement office said the chances of a classroom being flagged erroneously were less than one in 1,000. But state Superintendent Kathy Cox cautioned the public to wait for results of additional investigations before forming an opinion.
Ben Scafidi, director of the nonprofit Center for an Educated Georgia in Norcross, said the findings are likely to upset parents.
"I'm a parent, I'm mortified," said Scafidi, who has two children in public schools. "I think parents are very concerned about how their children do on the CRCT, and now they don't know how their children are doing."
The top five most suspicious schools statewide were in Atlanta. Gideons Elementary, Peyton Forest Elementary, F.L. Stanton Elementary and Usher/Collier Heights Elementary all had more than 78 percent of classes under suspicion.
In the district's Parks Middle School, nearly 90 percent of classrooms had highly unusual erasures.
Calls to all five schools were not returned Wednesday.
Perdue and Atlanta's Hall had a public spat last summer when she said the state's investigation into retest shenanigans didn't prove cheating. Perdue called her response outrageous.
On Wednesday, Hall said the district can't risk tarnishing the reputations of innocent school staff by jumping to conclusions.
"We are going to look very deep at all the classes involved and all the data," she said. "Where there are other corroborating factors that seem to indicate something else took place we are going to pursue it very, very directly."
If cheating is found, she added, the district will take immediate action.
A dozen Atlanta schools came under scrutiny after an October AJC story reported extraordinary gains or drops in scores between last year and the year before. Following a 2008 AJC story questioning some schools' gains on CRCT retests, the state investigated and sanctioned educators in four schools in four districts —--including one Atlanta school -- last year.
Perdue said districts will not be permitted to ignore the state's latest findings. "We will not allow this to be whitewashed," he said.
Cizek, however, said allowing districts to investigate themselves is akin to airport security allowing passengers to pat themselves down. He said an independent agency should conduct the inquiries.
Perdue said that, before the public release, he phoned Hall and the superintendent of Dougherty County in Albany, which had the second highest number of schools -- 14 -- in the moderate or severe categories.
Cheating to survive?
Dougherty schools spokesman R.D. Harter said the district is seeking more information and hopes to bring in outside experts to investigate the allegations.
Students who pass the CRCT only because of falsified scores miss extra tutoring available to those who fail. Perdue said systems that were flagged as suspicious will be expected to offer such services to children who should have received them.
The validity of the CRCT is crucial for one of the governor’s major initiatives this year: a plan to tie teachers' pay to student performance, partly based on test scores. Such an approach is also being pushed by the U.S. Department of Education.
Perdue said he informed U.S. Secretary of Education Arne Duncan of the findings of the cheating investigation. He said Duncan was "excited" that the state was taking an aggressive approach to test score validity.
But the focus on test scores has its detractors. Former teacher Donna Williams of Macon wrote in an e-mail that while cheating isn't justifiable, the No Child Left Behind Act may have created an environment where schools think they must cheat to survive.
"We are doing students and, indeed, ourselves a great disservice with the unrelenting use of scores," she wrote.
In addition to whatever measures the state board approves regarding individual schools, the state will be taking steps to increase the security of all tests, Perdue said. He declined to give specifics.
Student achievement office Executive Director Kathleen Mathers said the state hopes to do a similar analysis next year. It might also look at other tests besides the CRCT.
"Given the resources, we'd like to do this on a year-to-year basis,' she said.
-- Staff writer Kristina Torres contributed to this article.
CRCT tampering: Crisis in confidence, character and conscience for Atlanta Public Schools
by Maureen Downey, blog
In a jaw-dropping meeting with Gov. Sonny Perdue and his staff Tuesday, the AJC learned that a review of all 2009 CRCT answer sheets identified 74 "severe" elementary and middle schools schools in which there appeared to be widespread answer-sheet tampering. The worst offender was Atlanta Public Schools in which cheating appears to have occurred in 37 of its 55 elementary schools.
Here is the link to the AJC searchable database of all schools. Here is a link to the list of severe-only schools. Here is a link to the detailed news story. [posted above] And this takes you to a map.
Responding to earlier evidence of cheating, including an analysis by the AJC, the state had every 2009 answer sheet reviewed to measure how often kids changed wrong answers to right by virtue of erasures on the sheets. Because every test sheet was checked, the state was able to develop a reliable index of how often test answers were changed from wrong to right and flag schools that had inordinate occurrences of answer changes, right down to the classroom level.
It then flagged schools that had higher-than-average numbers of wrong-to-right answers, and found troubling patterns, most of which occurred in Atlanta schools and in Dougherty County schools. To understand, look at third grade math scores. Reviewing the answer sheets of 125,000 third graders, the state found that the average student changed 1.87 answers from wrong to right.
If there was a third-grade classroom in which the students on average changed 4.8 answers from wrong to right, a flag went up, said Kathleen Mathers, executive director of the Governor’s Office of Student Achievement. "That change was so much bigger than what we saw in classrooms across the state."
APS Superintendent Beverly Hall must address a culture of cheating in her schools that threatens all her reform efforts.
APS Superintendent Beverly Hall must address a culture of cheating in her schools that threatens all her reform efforts.
Then, the state examined how each individual class performed on the test and how many answers went from incorrect to correct. They compared each classroom to the state average. To be flagged, the changes from wrong to right answers had to be well above state average, so much so that it could not be a matter of chance. Then, the state looked at the schools as a whole and found widespread instances of improbable answer changes.
On its list of the 74 schools with the highest number of classrooms with questionable --unbelievable, in fact -- erasures from wrong to right, APS has 43 schools and Dougherty has eight schools.
Out of the 74 flagged schools, DeKalb had six schools including the much acclaimed DeKalb Path Academy Charter School. Fulton has three schools. Clayton has two schools, including Lewis Academy of Excellence, a charter school that appeared before the state board of education this morning to plead for a reconsideration of state charter status, citing its academic achievement and its performance on state tests.
No Gwinnett, Cobb, Fayette, Cherokee, Rockdale, Decatur, Forsyth or Henry schools are on the list of 74.
With evidence suggesting that tampering of test sheets took place in 67 percent of its 55 elementary schools, Atlanta Public Schools is now facing a crisis in culture, confidence, conscience and character.
This data show that a culture of cheating exists in Atlanta schools, a culture that may have taken root before reform-minded Superintendent Beverly Hall arrived a decade ago or may be a result of her relentless pressure on her schools to improve and do it quickly.
Either way, this culture cannot be tolerated and must be banished, even if it means a wholesale firing of staff. (I suspect that some will call for Hall's firing. If the cheating traces back to her, she will have to resign. There may well be a case to be made that she should have known that this was going on in her district.)
This is not a few bad apples. This is rot to the core of APS and it cannot be addressed with training or memos. The shakeup at APS should bounce desks off the floor and rattle pictures off the walls.
Otherwise, how can parents know if their kids are learning if test results are not valid?
On the issue of conscience, how could schools -- including some in which eight out of 10 classrooms had compelling evidence of cheating — promote students to the next grade who were not able to do the work, yet had soaring CRCT scores?
This is educational malpractice of the worst kind.
It not only hurts the children, but it victimizes the next teacher in the chain who can’t understand why her student who scored proficient in reading the year before now can’t sound out a sentence. Under the federal No Child Left Behind law, those students may have qualified for tutoring based on their undoctored scores, says Mathers, of the Governor’s Office of Student Achievement. "Students were deprived of those opportunities."
Superintendent Hall should not waste a minute arguing with the state"s findings. As a clearly concerned Gov. Perdue told us yesterday in an hour-long meeting, "The facts are what the facts are. Trust me, we will not allow this to be whitewashed. I can't think of any superintendent who has the students' best interests at heart who won't want to find out what happened and where."
Gov. Perdue met with the AJC Tuesday to share a stunning study of how often test answers were changed on last year's CRCTs.
Perdue said he is counting on systems to investigate and right any wrongs, including getting true assessments of their students and providing academic help to kids who aren't really proficient.
"There is strong evidence here," he said. "It is not my job to impugn or indict any one person or school. I may change my mind as I go down the line."
When you look at the lists -- and our AJC technical staffers have been working all night to get this data from the state in easy-to-use form for you -- you will see that the schools with the greatest instances of tampering are poor and minority. These are the schools that have the farthest to go to get their kids to proficiency on state tests.
You do not see the suburban powerhouse counties on the list. I also have to note that some districts with high poverty enrollments are not on the list. (Along with the 74 truly problematic schools, the state assembled a list of 117 schools with moderately troubling test irregularities.) So, between the severe- and moderate-concern schools, there are 191 or 10 percent of the state's elementary and middle schools that have test results that merit monitoring.
But the searing findings raise so many questions. Where did the cheating take place -- in the classroom or after the answer sheets were turned into the school offices?
In some instances, as many as 48 answers on a test sheet were changed from wrong to right. Why would a teacher or principal go that far to ensure a single student passed? How desperate were they?
Are the honest teachers turning a blind eye or are their complaints ignored?
How does APS rebuild after this? Why did so many Atlanta schools resort to cheating when national testing showed that APS was, in fact, raising achievement? Or, are those test results now in doubt, too?
How can teachers at the 43 APS schools on the state's most extreme list go to work tomorrow knowing that all their good work -- and there is clearly some good work amid this wreckage -- is now in question?
We need to talk about whether we are asking too much of students or too little of schools?
Let's start.
(I just received the official statement from GOSA on this and am tacking it on here as it gives more information on the process:
The Governor's Office of Student Achievement (GOSA) today released the results of a spring 2009 Criterion Referenced Competency Test (CRCT) erasure analysis. GOSA partnered with CTB-McGraw Hill (CTB), the state's testing vendor in charge of developing and scoring CRCT exams, to conduct a comprehensive examination of all statewide CRCT answer documents for grades 1 through 8. The analysis focused on the number of wrong answers that had been changed to right answers on individual student answer sheets in Reading, English-Language Arts, and Mathematics.
"The analysis looked on average at 125,000 test takers in every subject and grade level at which the CRCT was administered and provided a clear picture of typical student test behavior against which all schools could be compared," said GOSA Executive Director, Kathleen Mathers. "Our recommendations are intended to eliminate future problems and help students who have been adversely affected by test tampering."
In the analysis, CTB psychometricians scanned answer documents to identify total erasures per classroom, flagging those classrooms in which the number of wrong-to-right changes proved to be three standard deviations (SDs) or more above the state average. Less than 0.15% of test takers would be expected to fall in that range naturally.
Based on the analysis, schools were placed in varying categories according to their percentage of flagged classrooms. 80% of Georgia's elementary and middle schools fell into the "Clear" category, meaning less than 6% of the classes within a given school were flagged; 10% fell into the "Minimal Concern" category with 6%-10% of classes flagged; 6% were determined to be in the “Moderate Concern” category with 11%-24% of classes flagged; and only 4% were termed "Severe Concern" as defined by a school having 25% or more of its classes flagged for wrong-to-right changes.
Recommendations on which the State Board of Education will vote range from requiring local Superintendents to conduct internal investigations to determine the causes of testing irregularities to schools rotating teachers during the 2010 CRCT test administration so that they administer the test to students they have not taught. In addition, state monitors will be placed in all schools in the severe concern category during this spring's test.
"Important decisions will be made from this data that are critical to the future of Georgia’s children," said GOSA Deputy Director, Dr. Eric Wearne. "Overall, Georgia's schools are performing well and continue to excel in student achievement."
The CRCT is a standardized assessment given to students in grades 1-8 in Georgia. The test is designed to measure how well students at each grade level have learned the state’s curriculum. CRCT results are used to determine whether schools have made Adequate Yearly Progress (AYP) as required by No Child Left Behind (NCLB).
GOSA plans future analyses of standardized test scores, possibly including End of Course Tests (EOCT) and Georgia High School Graduation Tests (GHSGT) and will also examine graduation and dropout rates and other factors that determine student achievement. Please visit www.gaosa.org to see the full 2009 CRCT erasure analysis report.
APS Dr. Hall: “Why would I want to cover up cheating?”
5:40 pm February 11, 2010, by Maureen Downey
In an hour-long conference call with the AJC today, Atlanta Superintendent Beverly Hall said she is anxious to authorize an independent review by outside experts of questionable CRCT test sheets because the reputations of "58 principals, 444 teachers and hundreds of students are at stake."
Dr. Beverly Hall promises a vigorous investigation of CRCT answer sheets and asks the public to reserve judgment.
Dr. Beverly Hall promises a vigorous investigation of CRCT answer sheets and asks the public to reserve judgment.
"Clearly, we have already been tried in the court of public opinion," said Hall. "But I really look at the data. You are looking at the summary data. On first blush, it indicates a major problem. But I know enough about data analysis to know that is not certain. We are going to have to look class by class.
"If any teacher, any principal, any administrator cheated, no one will have to ask me to take care of it. I will take care of it. That person will not and should not work for APS or any other system," she said.
Hall plans to meet with her principals when she returns from a conference in Arizona and a weekend meeting with the Gates Foundation and remind them to continue leading their schools and focusing on achievement. "This is America. You are still innocent until proven guilty, I think," she said
As the AJC reported today, one in five Georgia public schools faces accusations of tampering with student answers on last spring's state standardized tests, officials said Wednesday, throwing the state’s main academic measure into turmoil. The state’s report singled out 69 percent of APS elementary and middle schools -- far more than any other district -- as needing formal probes into possible tampering.
While aware of the strong factual case against APS based on a state analysis of every 2009 CRCT test, Hall said there could be explanations why her schools would have far more answer sheet erasures from wrong to right.
"I am very concerned with the number of schools," she said. "We must do a very detailed analysis, and I am convinced when we do, we will get to the bottom of this. I believe it is incumbent to get an independent review of every classroom involved, every teacher involved, every principal involved so we determine just what happened and why we had what appeared to be exceptional number of erasures."
"I also have to say for the record that hard as it is for some people to believe, when children are held to high standards, when teachers use appropriate strategies, when there are high expectations, a lot of support, intervention strategies, children do improve and some of them improve dramatically," Hall said.
As possible factors for the suspicious erasure patterns, Hall cited her system's small class and school size, saying that averages could be skewed because of the smaller numbers. She also said it was possible that the instruction given to APS students on test-taking -- answer questions that they are uncertain about by shading lightly and come back and review and change -- could be factors.
(The problem for me is that other systems have small classes and schools, including Richmond and Clayton. Those systems have suspicious erasures as well, but nowhere near Atlanta’s total. And as for their test-taking approaches, Atlanta kids probably aren't too different in how they take the tests than their counterparts across the state.)
Hall intends to examine access cards records and building occupancy reports to see if school personnel were in buildings late at night during the CRCT test week. She notes that two of her schools on the list had state Department of Education monitors during the spring 2009 CRCTs.
As well, APS central office staff also was on hand, although Hall's deputy acknowledged that the test sheets remained in the school for several days before being delivered to a central site. And Hall said those monitors were not fixed in specific classes but policing the entire building. Still, she thinks many precautions were in place.
"That does not mean someone cannot circumvent it," she said. "That is the question mark out there."
Her system’s many corporate and foundational supporters -- Hall is a national superintendent of the year and has legions of fans -- are not panicking because they have been in the schools and seen the hard work under way, she said. In fact, most of the calls thus far have been supportive of the system and its track record. She cited the system's NAEP scores that have shown steady and consistent growth since 2003 in every subject and every grade tested.
And despite the contention of many teachers on this blog, Hall said the system welcomes and encourages whistle blowers, citing a hot line where teachers could lodge complaints. She said all complaints were investigated.
"Kids, regardless of socioeconomic background, if they are taught well, they will learn," she said. "Why would I want to cover up people who were cheating because it takes away from that. People in Georgia now they are saying, 'We always knew they were cheating because those kids could never achieve at that level.' It is in my best interest to ferret out and deal aggressively with anyone who thinks we need to cheat to prove that our children can do well."
Just now, while I was transcribing the tape from the interview with Dr. Hall and writing this entry, a former APS central office employee called me to tell me that whistle blowers who brought complaints were driven out of the system. The caller pledged to send me names of dismissed teachers. I only cite this here to point out that this is a complex issue with many moving parts and many polar positions.
I think Dr. Hall has an uphill battle in showing that the erasures are legitimate, given how out of line APS is from the rest of the state.
Stay tuned. This is going to be a frantic three months as the state wants systems to wrap up their investigations by the end of the school year. That is tougher for APS since it is bringing in an outside team and has so many more schools.
— Heather Vogell, John Perry, & Maureen Downey blog
Atlanta Journal-Constitution
2010-02-11
http://www.ajc.com/news/suspicious-test-scores-widespread-296490.html
INDEX OF OUTRAGES
Pages: 380
[1] 2 3 4 5 6 Next >> Last >>
|
