The Grim Testing News You Haven't Seen in the Standardisto Boston Globe
An analysis by Boston-based researchers show the state MCAS pass rate reported by the Department of Education may be exaggerated by as much as 20 percent.
On March 3, the state Department of Education, or DOE, announced that 90 percent of the class of 2003 had passed the Massachusetts Comprehensive Assessment System, or MCAS, the test required for high school graduation. Problem is, the pass rate was calculated using only the current crop of seniors. When all students who have been part of the Massachusetts class of 2003 are included in the count, the MCAS pass rate is 70 percent, or 20 percent lower than DOE claims.
According to researchers, this method of calculation does not account for attrition, that is, students who have dropped out, been retained in a grade, transferred out of state or into private school, or simply disappeared from the rolls. Some students who repeat grade 9 may be included in the pass rates reported for the grades following, but many will not.
Because the MCAS pass rate reported for the class of 2003 does not consider all the students who were enrolled in the class prior to the 12th grade, the DOE has discounted 16,991 students - 22 percent of the original class - from its calculation of pass rates.
"The DOE's statewide pass rates are dramatically inflated," said Anne Wheelock, researcher with the Progress Through the Educational Pipeline Project at Boston College. "The problem with the DOE report, in general, is that it gives only the proportion of current seniors who have passed. This method of calculating pass rates has inflated MCAS pass rates for the class of 2003, statewide, by race. It also inflates pass rates in many districts."
According to the report, in Cambridge this translates to a pass rate of 73 percent instead of 76 percent but in other high-need districts the difference is much higher. In Somerville, for example, the pass rate drops from 94 percent to 64 percent. In Boston it slips from a DOE figure of 70 percent to a "on-time pass rate" of 48 percent.
The report also indicates that not a single one of 19 high-need districts show real pass rates over 80 percent, and nine show real pass rates of 50 percent or less. Other DOE rates also mask wide variation in district pass rates as in the case of charter schools and wealthy districts.
When rates are recalculated to account for all members of the class of 2003, results show the variability that exists, even among districts with the same DOE rate. For example, among 15 districts showing a DOE pass rate of 93 percent, pass rates that account for all students range from 88 percent to 55 percent.
Researchers suggest that dividing the number of students passing MCAS by the number of students enrolled in the class in 9th grade will yield a more accurate pass rate - what they call the "on-time pass rate" - for each class. Specifically, the "on-time pass rate" for the class of 2003 would be based on those students who enrolled in ninth grade in October 1999, and passed MCAS in time to graduate with their class in spring of 2003.
DOE spokespersons say that the report is baseless and does not count.
"We disagree with the whole premise of the report because it makes assumptions that if those students stayed in class, they would fail," said Heidi Perlman, media director, DOE. "It's a national phenomenon that every student who starts in grade 9 does not necessarily graduate with that class."
In fact, the 1,300 students from the class of 2003 were moved ahead and they graduated with the class of 2002, she added. The dropout rate in 2003 has also gone down, she said.
As the debate continues as to which members of the senior class should be counted, or whether students who left school but are in GED programs should be included, the larger problem is that assessing pass rates based on 12th grade enrollments is misleading from the start, Wheelock said.
Walt Haney and George Madaus from the National Board on Educational Testing and Public Policy, Boston College who worked on this report believe that district pass rates are not the only trouble; results by race are also similarly distorted.
For instance, the DOE reports that 3,748 out of 4,984 African American seniors have passed MCAS, resulting in a pass rate of 75 percent. But 7,003 African American students were enrolled in Massachusetts' original class of 2003 in 1999 and the pass rate based on grade nine enrollment is then 54 percent.
Similarly, the pass rate for Latino students drops from 70 to 40 percent and for Asians from 90 to 79 percent.
"The failure to account for all students who were originally with the class of 2003 results in MCAS pass rates that are distorted statewide, by race, and for each district," said Haney.
Researchers believe that MCAS pass rates, both for the class of 2003 and subsequent classes, should be restated based on ninth grade, or even eighth grade, but not 12th grade, enrollment. Regardless of the reasons for student attrition, all original members of the class of 2003 should be included in the count.
"Calculating MCAS pass rates in a way that includes all students, rather than 12th graders only, would be more consistent with other indicators reported for each class, including the graduation rate required by federal No Child Left Behind legislation and reported for each class," said Madaus. "In addition, pass rates calculated to include all students would be more useful to policy-makers."
According to Perlman their study "makes a meaningless point...We are not changing anything."
— Auditi Guha
Researcher says MCAS pass rate at 70, not 90 percent
Cambridge Chronicle
March 12, 2003
http://www.townonline.com/cambridge/news/local_regional/cam_covccmcas03122003.htm
INDEX OF OUTRAGES
