by Monty Neill‚ Aug. 04‚ 2005
The National Assessment of Education Progress (NAEP) long-term trend report comes out every few years, tracking scores in reading and math on NAEP exams administered to large enough national samples to provide strong estimates overall and by race and gender of students at ages 9, 13 and 17.
The recent release produced some rare positive press for education -- scores were up in reading and math for 9 year-olds and in math for 13-year-olds. At the same time, the achievement gap had narrowed at least as compared with 1999, the previous assessment. The Bush administration quickly took credit for the gains, attributing them to the federal No Child Left Behind law (NCLB), as did assorted media, and called for expanded high school testing.
A careful reading of the report, however, indicates that the "spin" by advocates of high-stakes testing is not justified by the data.
The report is modest in length, not very text heavy, with lots of good tables and charts. http://nces.ed.gov/nationsreportcard/pubs/2005/2005464.asp The tables commonly show the score points at every NAEP administration since the long-term trend study began in the 1970s. The long-term trend report does not produce state-level scores. Those are on a separate assessment and will be released this fall.
Some observations:
Reading overall scores
The basic claim that scores for 9-year-olds increased is true. Those scores, which had ranged from 209 to 212 from 1984 thru 1999, have bumped up to 219 in 2004. The gains for lower-scoring students were larger than for higher-scoring students. The reading results showed no change for students scoring at the 90th percentile. Scores at age 13 have been flat since 1980. High school scores are actually down from their levels during the 1984-1992 period. The biggest decline at age 17 came for those at the 10th percentile (lowest scoring students).
Math overall scores
Math results, mirroring scores on NAEP state-level tests, rose at ages 9 and 13, but were flat at age 17, as they have been since 1990. Grade 9 scores rose from 1986-1990 and flattened for the next decade. They now have risen again from 1999 to 2004, by 9 points. Age 13 had been inching up since 1986, and then took a 5-point jump from 1999-2004. Age 17 has been flat since 1990. Age 9 math gains were a bit larger for lower to middle scorers than high-scorers, but the reverse was true at age 13.
A few comments:
Even if test-driven education "worked" for 9-year-olds, as the Bush administration has claimed, it would appear to be counterproductive at the high school level, which is where by far there has been the most high-stakes testing. Not only have the scores been flat, but the racial gaps have tended to widen at the high school level, and the dropout rate has increased according to several recent studies. Since No Child Left Behind (NCLB) did not become law until 2002 or come into effect until the 2002-03 school year, it is implausible to claim it induced the gains that did occur.
In addition, the gains may well come at a price: what has happened to untested subjects? In a few states, e.g., New York, scores declined on tests that don't have high stakes attached, and repeated surveys find less teaching in those subjects. An increase in the dropout rate appears to be a consequence as well. If the score gains are meaningful and they hold, that is good; whether they are worth the price should be publicly debated.
The racial gaps:
In reading, the black-white score gaps narrowed at ages 9, 13 and even 17, compared with 1999, which is the comparison the administration has touted and papers reported. The gaps, however, have tended to fluctuate, though not widely, from NAEP test to NAEP test. For example,
consecutively, starting with the 1984 test for age 9, the point size of the gap has been: 32, 29, 35, 33*, 33, 29, 35 (1999), and 26 in 2004. The gap for 2004 is smaller at age 13 (22 points) than at age 9 (26) and not much bigger at age 17 (29) than at age 9. The narrowest gap ever at age 13 was in 1988, grew slightly wider in 1990, then widened to around 30 points before dropping from 29 to 22 points from 99 to 04. Somewhat similarly, the high school gap had closed to 20 points in 1988 (from 50 points in 1980!), then expanded to 37 and fell to the 29-31 range since then.The reading score gaps between Hispanics and whites are pretty comparable to the black-white gaps (21, 24 and 29 points at ages 9, 13 and 17 in 2004). As with the black-white gap, the gaps at ages 13 and 17 were narrower in the late 1980s than they are now. From 1999 to 2004, Hispanics gained somewhat more than whites at age 9, but not at age 13 (both groups declined slightly), and at age 17 Hispanic scores dropped 7 points since 1999 while whites dropped 2 points.
In short, it is misleading to speak of closing the gap (never mind attributing it to NCLB) when the gaps at ages 13 and 17 have fluctuated.In math, the black-white gap narrowing at age 9 is significant only when one goes back to the 1982 scores. The gap was 25 points in 1986 and now is 23 points. The gaps are 27 and 28 points in 2004 at ages 13 and 17. At all ages, the biggest gap since 1982 was in 1999. At high school, the gap was 21 points in 1990, something of an anomaly year since before and after they ranged from 26 to 32 points. Similarly, at age 13 the gap in 1986 was 24 points, since then it mostly ranged from 27-29 points except for 23 in 1999. The 2004 gaps are notably smaller compared with 1982 and before at all age levels.
Comparing whites with Hispanics, in math, Hispanic scores did gain significantly more than white scores at age 9 but only by 1 point at age 13. The 18-point gap in 2004 is statistically narrower than the 26-point gap in 1999, but not statistically different in most of the preceding years back to 1973, during which time the size of the gap ranged from 21 to 23 points. At age 13, the current 23-point gap is within the 19-25-point size range going back to 1982. The 24-point current gap at age 17 is statistically different only from the 1973 gap of 33 points, and the gap has fluctuated from 21-24 points since 1986.
Overall, however, these data give no real credence to the claim that focusing on testing has closed the math gap since (for blacks) at ages 13 and 17, the gaps were narrower in 1986 (age 13), and 1990 (age 17). And even for the 9 year olds, the gap differences are not statistically significant until one goes back to 1982: the gap difference is only 2 points comparing 2004 with 1986 and 1994. Similarly, for Hispanics there is no credibility to the claim that NCLB has led to a gap closing since over time the gaps have fluctuated, sometimes as narrow or narrower as they now are. For the most part, gap closing can be claimed for 2004 only when compared with the low-scoring year of 1999.
Lastly, even if it can be shown that the focus on testing in two subjects led to increased scores at age 9 for reading and math and at age 13 for math alone, it does not mean that such gains could only have been attained through high-stakes testing. One danger of the test-based approach is that it is crowding out other ways of putting attention on student learning and professional development that should have more positive impacts on overall educational quality and equity.
Monty Neill, Ed.D., is Co-Executive Director of FairTest, the National Center for Fair & Open Testing, located in Cambridge, MA. Dr. Neill has directed FairTest's work on testing in the public schools since 1987. The FairTest website is http://www.fairtest.org/.
New Collection of Clickable Resources!Pencilsdown.org Bookstore
Fairtest.org
How Standardized Testing Damages Education
The Side Effects of Standardized Testing
What Should Parents Know Abut Standardized Testing in Schools?
Testing, Testing, Testing... From the Merrow Report on PBS
Roger Schank's Provocative Educational Outrage Columns
Posts
No comments:
Post a Comment