Let's Get Ready to RUMBLE

The AFTies call out Kevin Carey for making the recent claim that:

"...Rothstein's frequent assertions to the contrary--NCLB is not based on the premise that good schools can erase the achievement gap. A school can make AYP under NCLB and still have huge achievement gaps, as long it gets all students over that minimum standard."

The AFTies respond by quoting from the NCLB statute:

Where could Rothstein have gotten the idea that NCLB was an attempt to close the achievement gap? Maybe from the law itself. Here's what's at the top of the first page of The No Child Left Behind Act, as it was passed by Congress and signed by the President:

Public Law 107–110
107th Congress

An Act

To close the achievement gap with accountability, flexibility, and choice, so that

no child is left behind.

Carey makes a good point about the difference between closing the achievement gap and equalizing proficiency levels, but, given the actual text of the law, it's hard to dismiss Rothstein's claim that lawmakers created NCLB to close the achievement gap.

The AFTies delivered some good body blows there and almost smelled victory for a moment when they went and found the phrase "close the achievement gap" in the text of the statute, but then they went and snatched defeat from the jaws of victory by stating that "but, given the actual text of the law, it's hard to dismiss Rothstein's claim that lawmakers created NCLB to close the achievement gap."

Actually, it is easy to dismiss Rothstein's claim depending upon how "achievement gap" is defined in the statute. Kevin realized this and counterpunched:

AFTie John can't figure out the difference between the title of a law and what the law actually says.

[T]he phrase "close the achievement gap" can mean equally legitimate but very different things. It can mean "erase all academic performance differences between poor and non-poor students," or it can mean "make sure that both poor and non-poor students reach a defined (and in most states, not particularly high) level of achievement."

Since Congress chose the latter definition when the wrote the actual provisions of NCLB, it seems safe to assume that they also had that definition in mind when they referred to closing the achievement gap in the title.

Kevin managed to damage the AFTies badly, but isn't able to deliver the knockout blow. First, the term "achievement gap" is used no less than a dozen times in the text of the statute, not just the title. Second, Kevin should have provided textual support for his definition of "close the achievement gap" because he left himself open to this withering attack from Sherman "Crusher" Dorn:

In a word, dear readers, this is baloney (technical educational policy term). The mechanisms of AYP notwithstanding, rhetoric about NCLB has consistently been about the achievement gap. I'm not sure if Carey is echoing Fordham's Michael Petrilli (see my earlier entry on that), but when anyone starts to lowball expectations, it's, well, it's, well, ... soft bigotry? I'll avoid the purple prose and just note that defenders of the AYP mechanism and high-stakes testing are engaging in this rhetorical dance because NCLB and most accountability frameworks avoid concrete discussions of standards. We must have them, proficiency must be defined, but your everyday Joe wouldn't know what that means. Heck, I don't.

I'm not sure if we're headed towards the worst possible outcome of oversold education reforms (regression towards deterministic views of human capacity), but when defenders of high-stakes accountability start backpedaling as fast as they can from the rhetorical framework that's been the political underpinnings of NCLB, it's not good news.

That's the equivalent of a rhetorical roundhouse from Sherman who was clearly going for the knockout punch. Unfortunately Sherman misses the mark for the same reason Kevin did--failing to determine how the statute defines "achievement gap."

So why don't we go ahead and do that so we can declare a winner.

Right out of the box, we run into trouble because we quickly discover that Congress did not explicitly define "achievement gap." All we have to go on is references like "narrowing achievement gaps in accordance with section 1111(b)" throughout the text. So, clearly, we need to go to section 1111 to see if it sheds some light.

Section 1111 deals with "State Plans." Each State that desires NCLB funding must submit a plan (1111(a)) that will adopted challenging academic content standards (defined in 1111(b)(1)(D)(i)) and challenging student academic achievement standards (defined in 1111(b)(1)(D)(ii))to be used by the state to carry out the law.

The challenging academic content standards must be "in academic subjects that--(I) specify what children are expected to know and be able to do; (II) contain coherent and rigorous content; and (III) encourage the teaching of advanced skills."

The challenging student academic achievement standards must be (I) [] aligned with the State’s academic content standards; (II) describe two levels of high achievement (proficient and advanced) that determine how well children are mastering the material in the State academic content standards; and (III) describe a third level of achievement (basic) to provide complete information about the progress of the lower-achieving children toward mastering the proficient and advanced levels of achievement.

Note that the states must set both a proficient and an advanced standard to determine if students are mastering the academic content standards in addition to a basic standard to provide information about students who haven't yet met the proficient or advanced standards.

Now we can turn our attention to the Accountability system (defined in 1111(b)(2)) which must be enacted by each State to ensure that the state is making "adequate yearly progress." Now we're getting to the good stuff.

AYP is descibed in section 1111(b)(2)(B):

Each State plan shall demonstrate, based on academic assessments described in paragraph (3), and in accordance with this paragraph, what constitutes adequate yearly progress of the State ... toward enabling all public elementary school and secondary school students to meet the State’s student academic achievement standards, while working toward the goal of narrowing the achievement gaps in the State ...

So the states are supposed to be making adequate yearly progress (AYP) toward meeting the academic achievement standards and working toward the goal of narrowing the achievement gaps. So, as Sherman has already identified (but curtly dismissed), the notion of "narrowing the achievement gaps" is tied up in the notion of AYP. There are no separate standards for determine whether the achievement gaps have been narrowed. Presumably, if States are making AYP, they should also be narrowing the achievement gap. Let's see if this is in fact the case. To do that we need to take a look at AYP.

AYP is defined in 1111(b)(2)(C) and is intended to measure the "progress of public elementary schools, secondary schools and local educational agencies and the State based primarily on the academic assessments" (section 1111(b)(2)(C)(iv)). To effect the AYP, States are required to establish a starting point (section 1111(b)(2)(E)), a timeline (section 1111(b)(2)(F)), and separate measurable annual objectives (section 1111(b)(G)) for continuous and substantial improvement.

Starting Point. Section 1111(b)(2)(E):

Each State, using data for the 2001–2002 school year, shall establish the starting point for measuring, under subparagraphs (G) and (H), the percentage of students meeting or exceeding the State’s proficient level of academic achievement on the State assessments under paragraph (3) and pursuant to the timeline described in subparagraph (F).

Timeline. Section 1111(b)(2)(F):

Each State shall establish a timeline for adequate yearly progress. The timeline shall ensure that not later than 12 years after the end of the 2001–2002 school year, all students ... will meet or exceed the State’s proficient level of academic achievement on the State assessments under paragraph (3).

Measurable Objectives. Section 1111(b)(2)(G):

Each State shall establish statewide annual measurable objectives ... for meeting the requirements of this paragraph, and which ... (iii) shall identify a single minimum percentage of students who are required to meet or exceed the proficient level on the academic assessments ... and (iv) shall ensure that all students will meet or exceed the State’s proficient level of academic achievement on the State assessments within the State’s timeline under subparagraph (F)

So what does all this mean?

The primary criterion for determining if States are making AYP and satisfying the requirements of NCLB is determining whether students are meeting or exceeding the proficient level that they've set for themselves. Students are only required to meet or exceed a single cut score. (The advanced level never enters into the equation.) If the necessary number of students meet this criterion, then they've complied with NCLB. This mechanism is the political comprise that Congress settled on.

All that's left to do is to determine if the achievement gap will necessarily be narrowed as more and more students become proficient. The answer is yes it will. Let's take a look at a few graphs I blatantly stole borrowed from La Griffe.
The top distribution represents white student academic achievement. The bottom distribution represents black student academic achievement. Presently, blacks perform about a standard deviation below whites. The result is an achievement gap between the two groups. But the achievement gap itself is a statistical artifact which varies depending upon where the proficiency cut-score is located.

For example, if we put the cut-score at passing level 1 as shown in the above graph then the white failure rate will be 16% and the black failure rate is 50%, leaving us with a 34 point achievement gap. Now let's change the cut-score at passing level 2 , the white failure rate becomes 2% and the black pass rate becomes 16%, leaving us with only a 14 point achievement gap. Got that? Now take a look at this graph of how the achievement gap varies depending upon white pass rate.


So as we start to approach a point in which either all white students are either passing or failing the assessments, the achievement gap narrows until it practically disappears. Of course blacks will still be performing at about a standard deviation below whites, but the differential will be masked as long as either most students are passing or failing the assessment instruments.

Another way of saying this is that a combination of fiddling with cut scores and raising student achievement across the board will increase the number of students who are "proficient," thus narrowing the achievement gap. That's all that NCLB requires.

So it would appear that Kevin Carey holds on and wins the fight.

I disagree with Sherman that Kevin has backpedaled or otherwise lowballed expectations. NCLB says what it says, overheated politcal bloviating notwithstanding. In adition, as I've pointed out, increasing student achievement across the board will in fact narrow the achievement gap, assuming that States set the cut-scores sufficiently low.