April 2, 2008

Slate on NCLB

It's difficult to ascertain how much of this Slate article on fixing education policy is attributable to Slate's editors or the nominal author, Jim Ryan. Certainly, the misleading title "fixing it: Repairing some of the worst Bush administration screw-ups." can be blamed on the Slate editors. The Bush administration education policy is basically NCLB which is a bipartisan piece of legislation largely drafted by Democrats and left of center groups. So, classifying it as a "Bush screw-up" is a tad misleading. In fact, classifying it as a "screw-up" is also misleading. At worst, NCLB has perhaps been ineffective, but there is no evidence that it has made the already bad situation any worse.

Nonetheless, the body of the article is generally awful and blame can squarely be placed on its author for that. Let's dissect.

Most of the problems caused by the act stem from its ridiculous test-and-punish regime.

Actually, it's more like like a reward with federal largess, test to make sure federal funds are being used effectively, and punish schools that are not using the funds effectively regime. And, of course, it's a voluntary regime that states can opt out of if they so desire.

Specifically, the act promotes the heavy use and misuse not just of tests, but of stupid tests. This isn't a reason to abandon all testing; it is a reason, however, to come up with better tests and better ways to use those tests to judge schools.

The act does not promote the heavy use and misuse of stupid tests. Specifically, NCLB gives the states plenty of leeway to craft their own "challenging academic content standards" that must be "in academic subjects that--(I) specify what children are expected to know and be able to do; (II) contain coherent and rigorous content; and (III) encourage the teaching of advanced skills." (Section 1111(b)(1)(D)(i))) and "challenging student academic achievement standards" which must be "(I) [] aligned with the State’s academic content standards; (II) describe two levels of high achievement (proficient and advanced) that determine how well children are mastering the material in the State academic content standards; and (III) describe a third level of achievement (basic) to provide complete information about the progress of the lower-achieving children toward mastering the proficient and advanced levels of achievement" (Section 1111(b)(1)(D)(ii))). States must also enact an Accountability System (defined in 1111(b)(2)) that assures the state is making adequate yearly progress "toward enabling all public elementary school and secondary school students to meet the State’s student academic achievement standards, while working toward the goal of narrowing the achievement gaps in the State."

I'm not sure how anyone could read the statutory language as mandating the heavy use and misuse of stupid tests. But there we have it. Certainly, some states have crafted stupid standards and stupid tests, but NCLB didn't exactly mandate that they do any such thing. NCLB gives states plenty of leeway to fix their education problems in the best way they see fit or, as it turns out, avoid fixing their education problems by delaying the day of reckoning and/or setting low standards. But asking for more from NCLB would be even further intrusive which is something Ryan is definitely against already.

Current test results don't tell us all we need to know about schools. Far from it. Students are tested in reading and math and a little in science. Reading, math, and science are important, but so are social studies, history, literature, geography, art, and music. Instead of telling us how schools are doing in these other subjects, NCLB is turning them into endangered species by pushing schools—especially those that are struggling—to downplay if not ignore subjects not tested.

It is perhaps an exaggeration to analogize the situation to not worrying about the leaky faucets while the house is on fire, but not by much. Fixing reading and math should be a top priority, you kinda need those two things in order to learn the other things we also think are important, such as, history, geography, science, art, and music. In fact, there's good reason to believe that in order to fix reading comprehension problems you're going to go a long way toward fixing the content area subjects anyway. Moreover, if you want to get more information on student abilities in these non-tested areas, you're probably going to have to increase the number of tests to include these subjects, something Ryan is against.

Many tests that are given further narrow the focus of education by relying on multiple-choice questions that reward memorization and regurgitation rather than analytical and creative thinking.

Any poorly designed test can be used to "reward memorization and regurgitation." However, a well-designed multiple choice test will yield almost as good feedback as a well-designed fill-in-the-blanks or essay style test. The compromise is that the multiple choice tests are cheaper and easier to administer and are easier to grade. And, therefore the results can be gotten back to schools more quickly so that schools might act on the feedback in a more timely manner.

The second problem is that looking at just a sheet of test scores is a lousy way to judge school quality. Standardized test results tend to track socioeconomic status. As a teacher once remarked, the most accurate prediction you can make based on a student's test score is her parents' income. Teachers and schools with middle-class kids will invariably look better than those with poor kids if the only measure is how many students in a particular year pass a test.

This is why NCLB mandates disaggregating the data for various at-risk subgroups, so that more affluent school districts can't hide their failures. It's not a perfect system, but testing subgroups does catch many schools. For example, in Pennsylvania 446 out of 501 school districts have enough economically disadvantaged students to trigger the reporting requirements of NCLB. Similarly, 128 districts have a sufficient number of black student to trigger the requirement even though over 2/3rds of black students are concentrated in just 10 school districts.

And, not to put too fine a point on it, but this un-named teacher is wrong. Parental education levels and student IQ levels are better predictors than SES.

What we can't tell from scores alone, because they don't tell us where students started or how much they progressed over the year, is the value that a particular teacher or school has added to a student's education.

This only really matters if the schools are doing a lousy job getting kids up to speed in the first place. Schools have nearly four years to get kids to read and do basic math at a third grade level which is about twice the amount of time, even for low-performers, it should take if the school is performing with reasonable competence. After that, the school still has to make a year's worth of gains every school year or else students will fall behind. In other words, these "value-added" proposals only really matter at all today because many schools continue to do a lousy job in K-3 before the first NCLB test is even administered.

The third and most fundamental problem has to do with perverse incentives. Schools must show annual improvements on test scores or face increasingly severe sanctions and the stigma of being labeled as failing. NCLB couples this punitive scheme with utter laxity regarding the standards and tests themselves. States get to develop their own standards, create their own tests, and set their own passing rates. Imagine if the EPA told the auto industry it would be fined heavily for polluting too much but let automakers decide for themselves what counts as "too much" pollution. That's basically how NCLB works.

Again, I note the irony that Ryan started the article by complaining that NCLB is too intrusive, yet all of his complaints imply a solution that is even more intrusive. I also note the irony that NCLB allows states to set their own standards and testing instruments (as discussed above) and Ryan is complaining that its a bad thing that schools can't even meet these bogus standards. That should indicate the severity of the the underlying problem (schools don't know how to teach at-risk kids) really is. Imagine how punitive the measures would be if we held them to real standards. It's like blaming your scale which underweighs you by ten pounds for your being overweight by fifty pounds.

NCLB, despite lofty rhetoric to the contrary, is not about equalizing opportunities in poor and rich, city and suburban schools; it's about making sure kids can learn some of the basics. No less, for sure, but also no more.

You have to start somewhere right. Makes little sense to test a student on his ability to interpret a Shakespeare sonnet if that student can't read the thing in the first place.

OK, this post is getting too long. I'll save Ryan's equally-dopey proposed solutions for another post.


Spedvet said...

Take a gander at NCLB and the previous reauthorization of the ESEA (the IASA). Not much difference. You'll earn a golden banana if you tell your readers what the major difference is between the two laws. [And you guessed it, there is only one major difference.]

Try finding the widespread hatred of that law. Try. Search out all the writing of folks objecting to the Improving America's Schools Act. You won't find it.

That's because free money with no consequences for non-compliance is a both a beloved tradition and beloved entitlement for state and local depts of education.

These folks got real upset once they realized there'd be consequences for accepting those funds and being held to the promises they gave in accepting the money.

Notice that NO state didn't take the money.
ANY state was free not to take the money if it thought NCLB's requirements were so onerous.

SteveH said...

Good analysis (as usual) Ken.

My complaint of NCLB is that the minimum cutoff proficiency becomes a maximum goal for schools. Our schools get high marks on getting most kids over this low hurdle, but then call this an excellent education. They study the results of the tests and worry about making small relative changes. They never look at structural problems or assumptions. Our schools had a long-term strategic planning meeting a few years ago. These fundamental questions were completely off-the-table. They hired a facilitator to make sure of that. They are a "High Performing" school. The pressure is off, but few students ever make it to algebra in 8th grade.

By the way, In our state (with two others) the tests are developed and calibrated by teachers. The big news lately here is that only 22 percent of the 11th graders achieved proficiency in math. One educational columnist tried to do sample (released) problems from the test and then questioned whether this knowledge was necessary. I looked at the sample problems and all but 2 were algebra or lower level questions. A state education administrator acknowledged that the results were poor, but then questioned whether the government has the desire to provide the resources to fix the problem. In other words, it's not their fault.

My son scored very highly on the math portion of the test, but his school doesn't know (or care to know) that I've been supplementing with Singapore Math for years. For the reading comprehension part of the test he did OK, but I find out that the school never practices this skill. They don't like teaching to the test, so they take a very indirect route ... using posters, dioramas, and artifacts. This isn't about different learning modalities. It's about low expectations.

Now that my son is older, I can trust (a little more) what he says goes on (or not) in class. It's still hard to believe.