October 18, 2010

I win Open Left’s Dunce Hat Award of the Week

Woo Hoo! I win.

I’d like to thank all my readers and, especially, all my commenters who make all of this possible.  Without you I wouldn’t have the will to persevere to do daily battle with the mighty intellects at blogs like Open Left.  It’s very difficult to display your ineptitude in the face of Open Left’s withering criticism without support.  And, it’s not just ordinary everyday ineptitude; it’s dunce hat award level ineptitude.  That must be especially difficult to read.  I also want to thank Open Left for offering such a prestigious award to us dunces.  So, again, thank you, thanks to all of you.

Apparently it was my post on poverty and student achievement that set me apart from lesser dunces.  Let’s go through Open Left’s analysis.

His claim, in a nutshell, is that the fact that Asian students, despite their poverty levels, have higher achievement than any other racial or ethnic group of students proves that there has to be another more important variable than poverty influencing student achievement.

Actually, that’s not my claim in or out of a nutshell.  What I said, in response to Parry’s comment, was:

SES and student achievement are correlated. (You can't tell the tightness of the fit with this graph.) But, as you indicate SES can clearly not be the only independent causal variable.

SInce we’re in the realm of correlational studies, the data isn’t capable of proving any hypothesis.  The data can only disprove a hypothesis.  In this case the data disproves the hypothesis that poverty is the sole independent causal variable.

"Asian children with parents having only a high school diploma performed better than black children with parents having graduate degrees," he points out.

However, instead of rejecting outright that there is a consistent positive relationship between parental income or parental levels of education and student achievement (a correlation that is undeniable), he instead insists that there is an "invisible variable" at work.

I don’t reject the correlation outright.  The correlation exists. What I reject is the drawing of the causal inference that poverty causes low student achievement.  The correlational data is incapable of proving causation regardless of how consistent or positive the correlation may be. (In actuality, the correlation is rather low, as I pointed out here. Less than 20% of the variance in student performance is accounted for in the variance of socio-economic status, at best.)

And, I didn’t insist there was an invisible variable, I merely indicated there might be a third variable in play.

In fact, SES doesn't have to be an independent variable at all; there could be a third variable(s) that drives both SES and student performance.

And, that’s only because when drawing causal inferences from correlational studies, there is always the possibility that there is a third variable in play or that or that the causality is backwards (the wet streets causes rain problem).

Open Left goes on to conjecture:

And what is that invisible factor? he only hints at.

A few days later and many times previous I explicitly indicated what researchers conjecture those third variables might be:

But maybe, it’s that affluent kids possess traits like self-discipline, the ability to defer gratification, and cognitive ability that will allow them to stay out of poverty and do well in school.

Open Left also conjectures:

But even if he is right - that there is something about what happens in Asian households (parenting habits, for instance, or nutrition) that matters more than poverty - that only proves that the determining factor is still outside the control of schools. Unless of course he wants schools telling parents how to raise children.

Open Left is so blinded by their own ideology, they can’t even draw fair inferences from what I wrote.  There is no implication in my post that Asian performance is attributable to what goes on in Asian households.  It could be that Asian students possess traits like self-discipline, the ability to defer gratification, and cognitive ability that allow them to access the education being offered in their schools better.  And as far as Asian parenting styles go I pointed out Stanley Sue and Sumie Okazaki’s paper, "Asian-American educational achievements: A phenomenon in search of an explanation" that noted

the parenting styles and values found in East Asian-American homes tend to correlate with lower test scores when they are found in white homes.

So much for that correlation as well.

Open Left concludes:

The difference between school-based factors that influence achievement and factors outside of schools' control is a distinction that in no way does the ed reform movement want to discuss. And they'll go to any lengths to avoid that discussion.

I’m not the one handing out dunce hat awards in an effort to “avoid discussion” now am I?  I’m not the one with only correlational data and a bunch of failed out-of-school interventions.  And, I certainly have enough humility to know what isn’t known about what affects student achievement and not to hand out dunce hat award on dubious (and fairly debatable) premises.

Out of school factors certainly exist, we’re just still waiting for someone to accurately identify what they are and to establish an intervention that consistently works to ameliorate their effects before we shift resources away from school-based interventions which have been shown to increase student performance.

Is that really too much to ask?

1 comment:

Jason said...

Perhaps most offensive to me is not their total misunderstanding of causal research, but their claims earlier on that the real research base in education has only confirmed progressive philosophy.

Sorry-- but edu-philosophers, though they contribute quite a bit to the conversation, are not the source of research-based methods for school improvement. Practically by definition philosophy is non-experimental (although folks blend and cross every line these days, which is a good thing).