Orwell Award Announcement SusanOhanian.Org Home


The Eggplant

 

in the collection  

Defining "Adequate Yearly Progress"

No doubt about it, No Child Left Behind brought tremendous changes to America's schools in the 2002-03 school year.

States scrambled to add standardized tests. Schools beefed up programs aimed at struggling students. And, supporters say, the landmark legislation made schools focus more than ever on helping minority, limited-English, and special-needs students.

But one thing No Child Left Behind didn't do was provide a true national system for grading schools. All schools in all states must meet test score goals called Adequate Yearly Progress, or AYP. But exactly what that means varies widely from state to state. Each state has its own expectations and its own means of testing, making it tough to get an accurate picture of how the nation as a whole is performing.

Take the following examples from the 2002-03 school year:

53 percent of North Carolina schools missed making AYP last year.

30 percent of Oregon schools didn't make AYP.

13 percent of Kansas schools missed it.

And in Florida -- a state already known for its tough accountability program -- a whopping 87 percent of schools missed AYP.

Of course, Title I schools that don't meet AYP for two or more consecutive years face an increasing array of sanctions, ranging from having to offer student transfers to bringing in a completely new staff. So it isn't surprising that many educators look at the AYP disparity between states and wonder if it is fair. After all, why should nearly nine in 10 Kansas schools meet the federal guidelines in year one while nearly nine in 10 Florida schools fall short?

WHY THE DIFFERENCES?

But Ross Wiener of the Education Trust, a Washington-based school reform group that supports No Child Left Behind, says observers should look past the disparities.

"The system wasn't set up to allow for interstate comparisons," he says

Wiener says there are a number of legitimate reasons for the wide variation of AYP rates from state to state. For example, low-performing students in some states are clustered in a relatively small number of schools or school districts. These states are likely to have fewer schools missing AYP than states where low-performing students are more widely distributed.

States with large achievement gaps between white and minority students are more likely to have large numbers of schools failing to meet AYP, even if their overall academic performance is high. Also, some states test only a few grades at this point, which means they have fewer potential stumbling blocks. No Child Left Behind requires that grades three through eight, as well as one grade in high school, be tested by 2005-06. Once states add more tests, fewer schools likely will meet AYP.

On the surface, it appears that Minnesota, with 93 percent of its schools meeting AYP in 2003, should be celebrating. But that doesn't mean the state's schools have done all they should. Nor does it mean that Minnesota has lowered its standards to meet the NCLB standards.

A key reason that the number of schools that made AYP was so high is that the state's testing program is still in its infancy. The first batch of AYP results didn't include test scores for middle and high schools.

Before NCLB, Minnesota tested reading and math only in grades three and five -- and the sole consequence was having the results printed in the local newspaper. But next year, all schools, including middle and high schools, will be judged on their standardized test scores.

"Next year, we'll have many more schools not making AYP," says Bill Walsh, spokesman for the Minnesota Department of Education.

SETTING BENCHMARKS

No Child Left Behind provides a formula for states to determine their initial AYP benchmarks -- the minimum test scores each subgroup must make to reach AYP. The starting point, based on 2001-02 test scores, must be either the lowest performing subgroup or the passing rate for the school at the state's 20th percentile. States must choose the higher of these two marks.

"Setting the initial benchmarks was pretty easy," says Alexa Pochowski, assistant state commissioner in the Kansas State Department of Education.

But although all states determine their initial benchmarks roughly the same way, the actual scores are all over the map. In Oregon, baseline scores were 40 percent passing in reading and 39 percent in math. Pennsylvania had comparable first-year benchmarks: 45 percent passing in reading, 35 percent in math. The minimum passing rates in Delaware were 57 percent in reading and 33 percent in math.

But in California, the elementary and middle school benchmarks have been set to 13.6 percent in reading and 16 percent in math, according to the California Department of Education. The high school passing rates are even lower: 11.2 percent in reading and 9.6 percent in math.

Wiener says these comparisons are misleading, noting that the academic rigor of a state's testing program should not affect the numbers of schools meeting AYP. Each state sets its initial AYP benchmarks using test data from previous years, and current students' scores are compared with previous scores on the same tests. Whether a state's AYP starting points are high or low depends on student performance in previous years.

Wiener says he isn't concerned about states with low starting points. He says some districts -- particularly those with strong existing accountability programs -- simply might be farther along at this stage.

In fact, he believes, there is reason to be somewhat concerned about states with high AYP starting points. States where test scores already are high may not have made their curriculum standards rigorous enough, he says.

While No Child Left Behind provides a blueprint for setting initial benchmarks, each state must decide how to increase those scores over the 12-year duration of the act. Pochowski says Kansas will increase the benchmark scores -- the minimum passing rate for a subgroup -- every year but two over the life of No Child Left Behind.

But other states, including Delaware, increase their benchmark scores only every other year. And some states, perhaps anticipating legislative changes in No Child Left Behind, have kept their minimum passing rates relatively low now but promise to increase them dramatically in later years.

DEFINING A SUBGROUP

What makes No Child Left Behind different from most existing state accountability programs is its insistence that all student subgroups -- major racial groups, low-income students, disabled students, and students with limited English skills -- meet test score goals. Traditionally, states have graded schools on the combined performance of the student body.

But groups need to be of a certain size to be statistically valid. Subgroups that are too small don't count when factoring AYP.

NCLB doesn't specify how large those subgroups should be, however; that decision is left up to individual states. The subgroup definition is perhaps the biggest decision a state makes in determining its AYP fate.

The smaller the definition of a subgroup, the more likely it is that a school will have to count that subgroup in its AYP report. To look at it another way, states can remove stumbling blocks -- and potentially have fewer schools that miss AYP -- if they make their subgroup size large.

Robin Taylor, associate secretary for assessment and accountability in the Delaware Department of Education, says education officials in her state debated this subject extensively before settling on a subgroup size of 40. North Carolina also set its subgroup size at 40 students.

"We had some who want to make it very high," Taylor says. "But there was general consensus that 40 was an acceptable and accurate number."

For Taylor, determining subgroup size is a balancing act. On one hand, a group needs to be of a certain size to be accurate. It wouldn't be fair, for example, to place a school's AYP fate in the hands of a three-student subgroup.

But on the other hand, creating subgroups that are too big circumvents No Child Left Behind's core intent, which is to hold schools accountable for the performance of all groups of students.

Minnesota went the other route and defined subgroups as 20 or more students. But the state received permission from the U.S. Department of Education to make the special education subgroup 40 students, not 20.

"That took a lot of schools off the preliminary list," Walsh says.

PARTICIPATION RATES

No Child Left Behind demands that each school test at least 95 percent of its students. This has been a problem in some states and a nonissue in others.

In Kansas, Pochowski says, most schools tested 97 to 99 percent of students, and the participation rate requirement wasn't an issue. It also wasn't an issue in Delaware. Taylor says nine schools didn't meet the state's participation rate target, but these schools also missed AYP on test scores.

Taylor says the state always has put an emphasis on participation, which is why 98 to 99 percent of students get tested now. State policy mandates that students who miss the tests receive a score of zero, which gives school officials plenty of incentive to get everyone in class on test day.

But in states such as Minnesota, where high-stakes testing is relatively new, the 95 percent rule has been a sticking point. Walsh says some parents wouldn't let their children take the tests in protest, even though that counts against their school, and there has been plenty of grumbling from teachers and the public.

Walsh says he expects the testing controversy to gradually die down as students, parents, and teachers get used to the new reality of No Child Left Behind.

"That's a learning curve problem," he says.

DRAWING CONCLUSIONS

Wiener says educators should be neither surprised nor dismayed that states choose different approaches to tackling AYP. "States are primarily responsible for running public education," he says. "It's not going to be the same in every state."

NCLB offers one positive method for making state-to-state comparisons, though. All states must administer the National Assessment of Educational Progress reading and math tests, which at least will give one means of comparison.

A few patterns have emerged, although none is particularly unexpected. The limited-English and disabled student subgroups were the ones most likely to fall short of AYP targets. In fact, some critics say that, by definition, these groups never will perform at the level of other students. Students with limited English skills, for example, "graduate" out of that subgroup once they attain the English language mastery needed to pass the test.

Many states also had more problems in high schools and middle schools than in elementary schools. In Delaware, for example, a majority of middle and high schools didn't meet AYP. Taylor says this could be because some middle and high school students aren't taking enough challenging courses.

"In elementary school, everybody is working toward reading and math," she says. Also, she says many schools don't offer enough reading comprehension in the higher grades.

Many states will have to expand their testing programs as a result of No Child Left Behind. Kansas, for example, currently tests one grade each in elementary, middle, and high school. But Pochowski says the state is adding tests and ultimately will test grades three through eight, plus high school, as the act requires.

"That's what we've been working on," she says.

The expanded Kansas tests will be offered via computer, Pochowski says. This way, teachers will get their results within 24 hours and will be able to break down the data any way they choose. Pochowski says this will allow teachers to give students immediate help in exactly the areas they need.

But not everything has gone smoothly in Kansas, Pochowski says. The state has yet to find an acceptable means of testing moderately disabled students. These students are too advanced for the alternate assessment given to severely disabled students, but they also aren't ready for the standard test. Kansas has created a modified exam for these students, but so far the U.S. Department of Education hasn't signed off on it.

In Delaware, AYP evaluations include science and social studies tests as well as reading and math. "Our stakeholders felt it was important to include science and social studies," Taylor says.

Other states use average attendance and require high schools to include dropout rates in their AYP calculations.

Wiener says it may take several years of testing before any definite conclusions beyond the obvious can be drawn. But he says the national struggle to meet AYP already has had three benefits:

AYP is identifying achievement gaps between groups of students within schools. Many of these gaps had been hidden in state assessments that simply looked at a school's overall performance.

Many low-performing schools are showing improvement.

AYP results show that some schools are effectively teaching all groups of students, even traditionally underachieving groups.

Wiener also says he expects the vast differences in AYP success between states to gradually even out, at least to a degree. In the meantime, states, districts, and schools will continue to try to solve the AYP puzzle.

"This is a learning time for many schools," says Brian Christopher, deputy press secretary of the Pennsylvania Department of Education.


--------------------------------------------------------------------------------

Bruce Buchanan covers K-12 education for the Greensboro (N.C.) News & Record.

— Bruce Buchanan
Education Vital Signs/American School Board Journal
http://www.asbj.com/evs/04/accountability.html


INDEX OF THE EGGPLANT


FAIR USE NOTICE
This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of education issues vital to a democracy. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information click here. If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner.