Jay Matthews argues in favor of national standards using the wide disparity between students labeled as proficient in the NAEP and in various state tests:
Sherman Dorn helpfully reminds Matthews that cut-score setting is not the same thing as standards setting. And further adds:
Maryland recently reported that 82 percent of fourth-graders scored proficient or better in reading on the state's test. The latest data from the National Assessment of Educational Progress, known as "the nation's report card," show 32 percent of Maryland fourth-graders at or above proficiency in reading.
Virginia announced last week that 86 percent of fourth-graders reached that level on its reading test, but the NAEP data show 37 percent at or above proficiency.
[A]ny attempt to use a test to "set standards" is getting things backwards. Don't we first decide what we want students to do?and
So if there were a national test every child takes, I predict that there would be a yawning gap between the test and any sense of real standards or expectations.This is, of course, a sucker's bet.
The current NAEP already reflects a yawning gap between the test and any sense of real standards or expectations.
Tom Loveless of the Brookings Institute recently took a look at the math portion of the NAEP. He coded each released arithmetic item according to Singapore's (who leads the world in math achievement) scope and sequence for its math program and determined how well kids answered the problems in the 8th grade NAEP. Here's the results:
|Grade Level||% of Questions||% Answered Correctly|
So, for example, in the 8th grade NAEP 16.3% of the test questions were at a 1st grade level and yet only 54% of the 8th graders taking the exam answered the question correctly.
This is what Loveless concludes based on his analysis of both the 4th and 8th grade NAEP tests:
A couple of things stand out in the fourth grade portion of Table 1-3. First, the problem solving items on NAEP are not very challengingÂat least not in the arithmetic required to answer them. Content taught in first and second grades is at least two years below grade level for fourth graders. But that is the level of difficulty of more than four out of ten (43.6%) problem solving items on NAEP. The second surprising finding is that even though the NAEP items are so easy,Loveless then analyzes the "algebra" strand of problems and finds them also lacking.
fourth graders do not do very well on them. The first and second grade items demand nothing more than being able to add and subtract whole numbers and knowing basic multiplication facts. Yet a majority of the nationÂs fourth graders miss the average item pitched at this level.
Even more dramatic findings are evident at eighth grade. The eighth grade items are only slightly more difficult than those on the fourth grade test (3.4 mean grade level). Almost four out of ten items (39.6%) address arithmetic skills taught in first and second gradeÂsix years below the grade level of eighth graders taking the test. Indeed, more than three-fourths of the items (33/43) are at least four years below grade level, taught in the fourth grade or lower. Yet the percentage of eighth graders answering problem solving items correctly is an unimpressive 41.4%. Problem solving items on the eighth grade NAEP only require knowledge of very simple arithmetic. Despite this, eighth graders have trouble getting them right.
Despite the simpler arithmetic on algebra items, fewer students answer the algebra questions correctly than the number sense questions, suggesting that some of what has been discovered here may be because of test design. It is clear that the algebra items are assessing something other than arithmetic. One assumes that the something else is algebraicÂindeed it seems to be quite challenging to most eighth graders. Nevertheless, really knowing algebra means being able to solve equations that contain more sophisticated forms of numbers than whole numbers. Anything less challenging is appropriating the term ÂalgebraÂ to convey a false sense of rigor to a pool of test items.It would appear that Matthews is barking up the wrong tree if he thinks that the NAEP is the gold standard or that national testing is the answer to anything.
What scares me though is not where the Board of Governors has set the cut scores for NAEP, but where the states are setting their cut scores (some 50 points higher). At least the NAEP cut scores accurately show that most students can't still can't add their way out of a paper bag.
It also makes me wonder why teachers and public education apologists raise such a big fuss over standardized testing. Just look how low the standards really are. Imagine if we raised the standards to accurately reflect what students really need to know to succeed academically.