Vae Victis*

Educators have much to be embarrassed about, not the least of which is their inability to adequately educate most children. One only needs to look at sample questions from any standardized test and the percentage of students who answer incorrectly to get a get a good understanding of the magnitude of their failure. This failure is most profound at the elementary school level-- a time in which children should be learning all the basic skills they need to succeed academically in later years. Today, educators are better at making excuses and inventing phony disabilities than they are at making educated students.

I can write such things without fear of criticism because they are true. Back in the 70's we spent a half a billion dollars proving they were true. And they remain true today. Nothing has changed. The song remains the same.

Back in the 70's we conducted a research-based contest involving thousands of students learning in actual classrooms. We assembled all our high-falutin educators, gave them lots of money, and asked them to teach children in the manner of their choosing and to the best of their ability.

The experiment was Project Follow Through.

We gave educators everything they asked for. Not only did we give educators over an extra $2000 per pupil (in today's dollars) we also provided comprehensive services, which included breakfast, lunch, medical, and dental care, and social services, to each student taking part in the experiment. There would be no hungry students with toothaches, tummy aches, and bad parents to foul-up the data.

Our educators came back with exactly the sort of educational programs you would expect from them. One group came back with a whole language program; another came back with a discovery learning constructivist program. A third came back with an open education model, while a fourth came back with a parental involvement model. (A description of all the major models can be found here.) In short, all the fashionable nonsense that passes for education today took part in the contest.

Also included in the experiment was a program designed by a non-educator with no formal education training other than teaching preschoolers. This program was Direct Instruction (DI).

Each program would be implemented in real schools in actual classrooms with actual teachers. Each sponsor got two years to implement and stabilize their program at their sites. The third cohort through the programs would be tested in grades K-3. That's four years exposure to the intervention.

Most of you have heard the results, but I suspect that many of you do not know the magnitude or the lopsidedness of the results. Engelmann, the co-creator of the DI program, describes the results as follows:

The evaluation had three categories: basic skills, cognitive (higher-order thinking) skills, and affective responses.

The basic skills consisted of those things that could be taught by rote—spelling, word identification, math facts and computation, punctuation, capitalization, and word usage. DI was first of all sponsors in basic skills...Only two other sponsors had a positive average. The remaining models scored deep in the negative numbers, which means they were soundly outperformed by [the control group]

DI was not expected to outperform the other models on “cognitive” skills, which require higher-order thinking, or on measures of “responsibility.” Cognitive skills were assumed to be those that could not be presented as rote, but required some form of process or “scaffolding” of one skill on another to draw a conclusion or figure out the answer. In reading, children were tested on main ideas, word meaning based on context, and inferences. Math problem solving and math concepts evaluated children’s higher-order skills in math.

Not only was the DI model number one on these cognitive skills; it was the only model that had positive scores for all three higher-order categories: reading, math concepts and math problem solving. DI had a higher average score on the cognitive skills than it did for the basic skills...

Not only were we first in adjusted scores and first in percentile scores for basic skills, cognitive skills, and perceptions children had of themselves, we were first in spelling, first with sites that had a Headstart preschool, first in sites that started in K, and first in sites that started in grade one. Our third-graders who went through only three years (grades 1-3) were, on average, over a year ahead of children in other models who went through four years—grades K-3. We were first with Native Americans, first with non-English speakers, first in rural areas, first in urban areas, first with whites, first with blacks, first with the lowest disadvantaged children and first with high performers.

From The Outrage of Project Follow Through, chapter 5, p. 4-8. For a more thorough description of the results see here, here, and here.

Clearly, these results were a profound embarrassment to our educators. Not only did a program developed by a non-educator completely and utterly trounce the educators' programs, but often their programs were beaten, and beaten badly, by the control group which received a more traditional education.

The bitterness still exists to this day. The DI program is so hated by educators they have erased it from their collective memory banks. It is a painful reminder of their professional incompetence. It dispels all their unscientific "theories" and unfounded opinions. It shows that they are a sham.

For educators, Project Follow Through was a total loss. They lost in every academic subject tested. They lost in teaching basic skills, which was expected. But they also lost in teaching higher order skills and in fostering student self esteem, which was unexpected. They lost in teaching low performers. But they also lost teaching high performers! They lost teaching everybody everything. It was a humiliating defeat.

And, they didn't just lose by a little bit. They lost by a lot and by a lot I mean an obscene amount. For most measures, the DI program beat the control group by an effect size of a standard deviation (This means a student performing at the 20th percentile would be performing at the 50th percentile after the DI intervention). Many of the educator's programs, in fact, underperformed the control group. Ouch!

Today, educators get giddy as schoolgirls when research comes back showing an educationally insignificant effect size of a quarter standard deviation, like results of The Star Project (class size reduction). The DI program beat them by 4x this amount. Yet, you'll never hear a good word coming from an educator when you mention DI.

The performance of the DI program in Project Follow Through dispels all the popular notions educators hold about education. When you hear an educator spout opinions on what he thinks is necessary to improve education, you can rest easy knowing that such bromides were tested in Project Follow Through.

We need more money:

The performance of the sponsors clearly debunked the notion that greater funding would produce positive results. All sponsors had the same amount of funding, which was more than a Title-1 program received. DI performed well in this context; however, the same level of funding did not result in significant improvement for the other models.

You got more money in Project Follow Through and it didn't make a whit of difference in your performance. Not only did you lose, you underperformed the control group.

Poverty kids have too many external factors affecting their performance. We need to fix those before we can educate them.

For all programs there were comprehensive services, which included breakfast, lunch, medical, and dental care, and social services. In this context, the only reasonable cause for the failure of other models was that they used inferior programs and techniques.

The kids in the DI program had the same external factors affecting them, and yet the DI program improved the academic performance by a significant amount despite the presence of these factors.

Kids have different learning styles. One program isn't suitable for all kids.

DI outcomes also debunked the myth that different programs are appropriate for children with different learning styles. The DI results were achieved with the same programs for all children, not one approach for higher performers and another for lower performers, or one for non-English speakers and other for English speakers. The programs were designed for any child who had the skills needed to perform at the beginning of a particular program. If the child is able to master the first lesson, she has the skills needed to master the next lesson and all subsequent lessons. The only variable is the rate at which children proceed through the lessons. That rate is based solely on the performance of the children. If it takes more repetition to achieve mastery, we provide more repetition, without prejudice.

I could go on all day dispelling education myths--half a billion dollars buys a lot of data-- but I think you get the point by now.

In the years subsequent to the Project Follow Through, educators haven't improved their programs by one iota. Student performance remains stagnated at 1970's levels. The only thing they've been successful at was in burying the Project Follow Through data with the help of Jimmy Carter--history's greatest monster-- and his awful presidential administration. (Ironically enough Project Follow Through was initiated by Democrats and then killed by Democrats when the results did not jive with the interests of their special interest supporters.)

In the meantime, the DI people have quietly gone about the business of improving their program while continuing to be shunned by educators. Today, a well implemented K-5 DI school (which only teaches the scripted DI programs for half the day) can get students performing at less than the 20th percentile, children that educators write off as ineducable, up to the performance level of children in affluent schools (See the 2003 results for what was once the worst performing school in the inner city of Baltimore). They do this in a normal school day, in a normal school year, without cherry picking students, without giving homework, without narrowing the curriculum, without sacrificing recess, art class or music class, while actively discouraging the school from teaching to the state tests, by virtually eliminating the need for special education classes, and at the same funding levels available to most schools.

Are there any myths left to dispel?

*Vae Victis -- Woe to the Vanquished