October 25, 2009

A Good Example of Why Education Professors Shouldn't Blog

Education Optimist, Sara Goldrick-Rab, must not get embarrassed very easily judging by this post criticizing a mostly dopey Kristof column on education and poverty:


Social science researchers across the nation are scratching their heads. Where in the world did Kristof get this one? For decades, solid analyses have demonstrated that while aspects of schooling can be important in improving student outcomes and alleviating the effects of poverty, the effects of factors schools cannot and do not control are much greater (for a place to start, read Doug Downey's work). Kristof emphasizes teachers and improving teacher quality by taking on the teachers' unions because he reads the data to mean that "research has underscored that what matters most in education - more than class size or spending or anything - is access to good teachers." Simply put, wrong. Access to good teachers is the most important factor affecting student achievement that is under schools' control (or as many put it, the most important school-level factor). What matters most in educational outcomes is the poverty felt by students' families. And to my knowledge, no study has ever rigorously compared the effectiveness of interventions based on cash transfers, housing subsidies, and teacher quality improvement-- what's needed to reach the kind of conclusion with which Kristof drives his argument. At the same time, a simple glance at the relative effects of programs like Moving to Opportunity, New Hope, etc which target poverty itself rather than how adults interact with children from poverty (the aim of improving teacher quality), should convince anyone than his target is misplaced.
(emphasis added)


That is one densely packed paragraph of bad reasoning.  One more poorly reasoned sentence and it might have collapsed upon itself into a educational black hole, if you will.  Fortunately the following hasty call for censorship, another Goldrick-Rab staple, was placed in the next paragraph.
 
Experts who think daily (Ed -- how about the ones that only think fortnightly?) about how to end poverty could, and undoubtedly will, inform the next steps taken by Democrats. Dems should listen to them, and not to Kristof.
 
First of all, there are few if any solid analyses of the effects of poverty on educational outcomes.  I don't think there is any scientifically sound research upon which anyone could draw the causal conclusion that alleviating poverty has educationally significant effects on student outcomes.  But why don't we take a quick glance at the two pseudo-research studies Goldrick-Rab cites:
 
The Moving to Opportunity (MTO) for Fair Housing Demonstration Program interim report found:
 
MTO had no detectable effects on the math and reading achievement of children.

OK.  Let's move on to the New Hope Project:

One of the most striking findings from the earlier evaluation reports was New Hope’s positive effects on children’s academic achievement at the two-year mark, in the form of increased teacher-rated academic skills, and at the five-year mark, in the form of higher standardized reading test scores (these tests were not administered at Year 2) and higher parent-reported grades in reading. However, these effects did not persist to Year 8, at least for the full child sample, although there were continued small effects on reading test performance for boys. No effects on math test performance were found. Overall, there was a tendency for impacts to be greater for boys than for girls. (emphasis added)
(See p. 29 and Table 5)

Do you remember the scene in the movie My Cousin Vinny where Vinny's girlfriend is looking at all the bad pictures his girlfriend has taken to help him win the lawsuit against his cousin and his friend that he's trying to win? 

Okay, you're helping. We'll use your pictures. Ah! These *are* gonna be - you know, I'm sorry, these are going to be a help. I should have looked at these pictures before. I like this, uh, this is our first hotel room, right? That'll intimidate Trotter. Here's one of me from behind. And I didn't think I could feel worse than I did a couple of seconds ago. Thank you. Ah, here's a good one of the tire marks. Could we get any farther away? Where'd you shoot this, from up in a tree? What's this over here? It's dog shit. Dog shit! That's great! Dog shit, what a clue! Why didn't I think of that? Here's one of me reading. Terrific. I should've asked you along time ago for these pictures. Holy shit, you got it, honey! You did it! The case cracker, me in the shower! Ha ha! I love this! That's it!


The MTO and New Hope studies are the case crackers of poverty interventions on education outcomes.

Both studies show what most studies typically show for poverty interventions on education outcomes:  small or undetectable effects that tend not to persist past adolescence.

And, here we have Goldrick-Rab citing them as conclusive proof of just the opposite.  Simply amazing. The woman has no shame.  Where's Bracey's rotten apple award when you need it most?

6 comments:

Stephen Downes said...

> I don't think there is any scientifically sound research upon which anyone could draw the causal conclusion that alleviating poverty has educationally significant effects on student outcomes.

The repetition of this point on your part is beginning to sound like global-warming or evolution denial.

Study after study makes the point, and your repetition of a narrowly-based (and most certainly not evidence-based, as you have exactly zero counterexamples to support your case) reflects only a minority, fringe opinion.

It is irresponsible and rude to accuse a university professor (who has, you know, actually studied the field) of being "mostly dopey" for making the point.

(I make this comment not to convince you, because you steadfastly refuse to reason honestly on these points, but so your readers are aware of the status of your views as opposed to what is accepted knowledge in the field).

KDeRosa said...

Study after study shows a correlation between SES and student outcomes. Study after study has also failed to shown causation. Poverty interventions, like the two cited, have consistently failed to improve student outcomes in an educationally significant way. This would lead most honest observers to conclude that either you have the causation backwards of there is a third factor that is affecting both SES and student outcomes. Gee what might that be?

As far as counter-examples go, both the MTA and New Hope studies are counterexamples. In both studies, there was a poverty based intervention which utterly failed to improve student outcomes. Ooops. If you and Goldrick-Rab's theory was correct the inteventions should have been smashing successes and yet they weren't.

It is irresponsible and rude to accuse a university professor (who has, you know, actually studied the field) of being "mostly dopey" for making the point.

Here's what I wrote: "Education Optimist, Sara Goldrick-Rab, must not get embarrassed very easily judging by this post criticizing a mostly dopey Kristof column on education and poverty:"

Clearly, "mostly dopey" was referring to the Kristof column and not Goldrick-Rab's post.

Simple reading comprehension is proving to be a consistent problem of yours, Stephen.

(I make this comment not to convince you, because you steadfastly refuse to reason honestly on these points, but so your readers are aware of the status of your views as opposed to what is accepted knowledge in the field).

I doubt any readers who are familiar with your antics in our previous argument are going to take anything you write seriously. Your faith in your own reputation is either hopelessly niave or comically sad. I haven't decided which.

Dick Schutz said...

Two Wrongs (Kristof and Goldrick-Rab) don't make a Right. They just generate another dopey Downes' comment.

Why the belief that SES is causal is so deep and wide is perplexing and astounding. The only explanation I can come up with is that it lets publishers, professors and other "authorities", who ARE causally responsible, off the hook.

There are numerous "proofs" of correlation between SES and various health/illness conditions. But no one draws the loony tunes conclusions that SES is the cause or that the conditions would be fixed if poverty were reduced.

"Some" professors shouldn't blog; from my perspective they shouldn't even teach. But there ARE at least a few worthy prof/bloggers. And it seems to me the Internet keeps the profs more honest than the Ivory Tower does.

When are you going to tell us the dinosaur story?

KDeRosa said...

Good points, Dick.

Clearly, I'm stalling on the Dinosaurs and the other half of the insurance post. Good things come to those that wait.

Corey Bunje Bower said...

There's at least one study that finds that an exogenous source of income -- lottery winnings -- led to better health

Corey Bunje Bower said...

I agree that there's no conclusive longitudinal evidence that explicitly shows that reducing poverty will help kids in school. But there's all sorts of tangential evidence on the effects of poverty and the relationship between some of those effects and school performance. You're right to be skeptical, but I think you overstate your position.

Also, you should know that:

New Hope lasted 3 years, the 8 year study was done 5 years after its conclusion. That test scores weren't higher isn't really evidence that those reforms don't matter, it's evidence that 3 years of those reforms isn't powerful in that manner 5 years later. Besides, even 5 years after the experiment, kids in the treatment group were less likely to receive poor grades or attend remedial summer school and reported higher levels of school engagement, higher school expectations, and more hope for the future.

MTO was disappointing for a lot of people, but the results can be fairly characterized as mixed. Students in some sub-groups (e.g. African-Americans and Baltimore residents) did do better on standardized tests. And there were other positive outcomes -- mother's mental health, for example. Besides, there's also a problem with the scope of the reform. Only 47% of treatment group households actually moved to a lower-poverty neighborhood, and many subsequently moved back to higher-poverty neighborhoods. Besides, moving to a different neighborhood doesn't really effect the poverty level of a family.

Lastly, studies on Gautreaux, which would certainly be included in the et cetera part of her list, have found some huge results. For example, one study found that students who moved to the suburbs were four times less likely to drop out of school, twice as likely to enroll in college, and seven times as likely to enroll in a four-year college.

So, yes, there's no airtight evidence that ameliorating poverty will make kids do better in school. But there's also no airtight evidence that it won't. Given that SES encompasses so many areas of a child's life, it's actually fairly remarkable that changing just a few things for a rather short period of time sometimes has a positive effect.