SAT Scores and Family Income

Here we go again.




This graph of family income vs. SAT sub-test scores from the College Board (via the NYT's Eeconomix) has been getting a lot of play lately.




Many interpreted the graph as saying that raising family income causes an increase in SAT sub-test scores. (The Economix's sloppy "boost" language didn't help here. Nor did the left-unexplained R² of 0.95, an artifact of how the graph was constructed, rather than as a measure of the actual variance between family income and test scores which is far lower.)

Faithful readers will already know that the underlying data for the graph is incapable of supporting a causal analysis of the relationship. It only supports descriptive and correlational analyses. Yet that didn't stop many from making the causal leap in logic by opining how obvious it is that all we need to do to produce better students is to provide their family with more money (preferably lots more money) and, thus, solving all our education woes.

Economist Greg Mankiw points out the danger (and ridiculousness) of claiming causality from mere correlational data.

Suppose we were to graph average SAT scores by the number of bathrooms a student has in his or her family home. That curve would also likely slope upward. (After all, people with more money buy larger homes with more bathrooms.) But it would be a mistake to conclude that installing an extra toilet raises yours kids' SAT scores.

It's also a mistake to conclude that winning Powerball or getting a governmental handout is going to raise SAT scores as well. Suggest the former and you'll attract funny glances; suggest the latter and half the edusphere and three quarters of edu-journalists will call you profound. Yet both are un-empiric and equally silly.

This is especially so considering there is ample correlative and experimental evidence indicating that there is a third variable driving the test score - family income/number of bathrooms and similar income-derived correlations. Take a look at the correlation between IQ test scores and biological father's income in this graph.



The graph shows the correlation for both ordinary children and adopted children. The correlations are remarkably similar, yet the contribution of the biological father of adopted children is mostly limited to some sperm. That's some mighty potent sperm. Seems like we'd get a better bang for our buck if we assured that fathers have plenty of money instead of wasting it on kids once they are conceived.

I can waste the better part of the day dredging up silly correlations. Some of them might confirm your biases and opinions on how to improve education outcomes. You'll like those. Others will go against the grain of your biases and opinions. You won't like those and you'll probably think of a way to discount the results. In both cases you're not thinking scientifically. And that's the real problem in education.