Obama effect for reals according to NYT

Sam Dillon of the NY Times breathlessly reports some real educational magic today:

[The]performance gap between African-Americans and whites on a 20-question test administered before Mr. Obama’s nomination all but disappeared when the exam was administered after his acceptance speech and again after the presidential election.

The inspiring role model that Mr. Obama projected helped blacks overcome anxieties about racial stereotypes that had been shown, in earlier research, to lower the test-taking proficiency of African-Americans, the researchers conclude in a report summarizing their results.

This one was so hot the Times couldn't wait for peer review. No need to waste time with that. The Obama effect is for real and it must be reported right this second.

In fairness, Dillion does mention that the study had not undergone peer review, but only provides an incomplete and misleading explanation of prior research on stereotype threat which might have tipped readers off as to the dubiousness of this latest study.

Here are the money grafs from the Wikipedia entry on stereotype threat:

Furthermore, while Sackett et al. do not dispute the fact that stereotype threat has a real, measurable effect on test scores, they posit that in the part of the experiment where Steele and Aronson removed the stereotype threat, the achievement gap which did remain correlated closely with the existing African American - White achievement gap on large-scale standardized testing such as the SAT. In their own words:

Thus, rather than showing that eliminating threat eliminates the large score gap on standardized tests, the research actually shows something very different. Specifically, absent stereotype threat, the African American-White difference is just what one would expect based on the African American-White difference in SAT scores, whereas in the presence of stereotype threat, the difference is larger than would be expected based on the difference in SAT scores.

In subsequent correspondence between Sackett et al. and Steele and Aronson, Sackett et al. wrote that "They [Steele and Aronson] agree that it is a misinterpretation of the Steele and Aronson (1995) results to conclude that eliminating stereotype threat eliminates the African American-White test-score gap."

In the past researchers have been able to depress scores by introducing a stereotype threat (basically, the researchers told the test subjects that they were part of a group that were dummies). Removing the threat only brought scores back up to historic averages. They have not been able, however, to actually increase scores above historic averages.

This new research, in contrast, supposedly shows that test scores can be increased by such a large amount (somewhere between 0.5 to 1.0 standard deviations) to wipe out the achievement gap that exists between blacks and whites. Educationally speaking, that's a giant effect size and a truly unprecedented result (if true). Indeed, the Obama effect must be extraordinary to achieve such a result.

It's basically the educational equivalent of cold fusion. The Times apparently forgot about that lesson in journalistic humility. And so much for acknowledging legitimate opposing viewpoints that might cast some doubts on these extraordinary, unprecedented findings. Get a load of the expert the Times dredged up, replete with some nice spin supplied by the Times.

“It’s a nice piece of work,” said G. Gage Kingsbury, a testing expert who is a director at the Northwest Evaluation Association, who read the study on Thursday.

But Dr. Kingsbury wondered whether the Obama effect would extend beyond the election, or prove transitory. “I’d want to see another study replicating their results before I get too excited about it,” he said.

Kingsbury's wants to see replication -- a prefectly reasonable response. And the Times spins the failure to achieve replication as possibly being cause by "transitory" effects. The expert provides no indication that the results might be in doubt (in fact, he praises the study as "a nice piece of work") and the Times provides no indication that any doubt exists.

The impression I get from the Times article is that the Obama effect is for real, pending peer review, but might fade due to its transitory nature. The Obama legacy is already being written.

Keep your eyes on test results this summer. If we are to believe the Times the achievement gap should be eradicated due to the Obama effect. NCLB will turn out to be a smashing success. And, the world will be a happier place. Unless those nasty transitory effects dash our hopes once again.