The Last Word (Hopefully) on Class Size Reduction

Never willing to let well enough alone, Mike in Texas, dredged up the class-size debate in the comments of this post again. I'm weary of the need to rehash these arguments over and over in comments battles so I'm going to make this post my definitive class size rebuttal post and just link to it when the need arises.

Executive Summary

Class size reduction in and of itself is not a proven technique for raising student performance. Class size reduction in combination with other factors does form a necessary factor in successful programs that do raise student performance. In any event, the research on classs size generally stinks; it is riddled with methodological shortcomings and typically shows small effect sizes. Given this and the high expense reducing class sizes entail along with the present teacher shortage, reducing class size does not represent a good investment for raising student performance.

I. The Hanushek Krueger meta-analyses

Both Hanushek and Krueger performed a meta-analysis on the extant class size research. For policy purposes, their conclusions are the same.

A. Hanushek (The Evidence on Class Size) (The Class Size Debate)

In a meta-analysis of 59 studies yielding 277 estimates of the effect of class size in student achievement, Hanushek found that 14.8% of these estimates were positive and significant. That is, students in smaller classes showed significantly higher achievement than their counterparts in larger classes. The remaining estimates were either insignificant (no difference in achievement – 71.9%) or negative and significant (smaller classes had lower achievement — 13.4%).

B. Krueger (Economic Considerations and Class Size) (The Class Size Debate)

Krueger reanalyzed the studies in Hanushek's meta-analysis using three alternate methods of analysis, only one of which is not controversial. Krueger found that 25.5% of these estimates were positive and significant. The remaining estimates were either insignificant (61.2%) or negative and significant (10.3%).

Much like hanushek found, a majority of studeies had insignificant or negative results.

C. Effect Sizes

The Hanushek-Krueger debate is all but academic. Here is the important finding by Hanushek that makes the debate academic:

More importantly, the estimated magnitudes are very small. A class size reduction of 10 students, which approximately cuts average class size in half and represents a 2½ standard deviation movement, is never estimated to yield more than 0.12 standard deviations improvement in student achievement for the results that are statistically significant. When results are separated for students eligible for free or reduced lunches, the performance of disadvantaged students is found to be more sensitive to class size: A 10 student reduction in class size reductions could yield as much as 0.19 standard deviations (in fifth grade math performance). Estimated class size effects for students ineligible for free or reduced lunch are, however, less than half the size of those for disadvantaged students and are more frequently insignificant.

This finding is not in dispute.

It's generally recognized that effect sizes smaller than about 0.25 standard deviations are not educationally significant. In other words, in the real world it is not worth pursuing an intervention whose research only procurred having such a tiny effect size. To put it in persepective, in theory your typical Title I school performing at the 20th percentile (80% of the kids are not at grade level) would only be boosted to the 28th percentile (72% of the kids below grade level) with the application of an intervention having a 0.20 effect size. In reality the increase would be less than that.

II. Project Star

The big kahuna of misrepresented, misinterpreted, and misued education research -- Project Star.

Project Star should be the poster child of what ails education research. To educrats Project Star is now slogan -- a talisman they can wave that proves that class size reduction is the panacea for our education woes. Hardly.

Let's start with the methodological flaws in the research which are signficant:

1. that not all students started the experiment at the same time, because kindergarten was not mandatory or universal in Tennessee;
2. sizable attrition occurred over the course of the experiment because of mobility and other factors, and this attrition was likely not random;
3. parents,teachers, and schools knew they were part of an experiment and, because of pressures from parents, part of the experiment was compromised by re-assignments of students;
4. no achievement tests were given before kindergarten, making it difficult to analyze whether elements of the random-assignment process contributed to any subsequently observed achievement differences;
5. approximately 6 percent of the students were transferred across treatment groups at the end of the first year of the experiment; and
6. there was some drift from the target class sizes of 15 and 22 so that there is actually a distribution of realized class size outcomes over time in both treatment groups.

Each of these issues has been raised by the initial researchers (e.g., Finn and Achilles, 1990) and by later interpreters of the results (e.g., Mosteller (1995) and Krueger (1997)), but the experimental data do not provide information that permits fully ascertaining the effects of such possible problems.

These methodological flaws remain unrebutted and go right to the heart of whether project STAR is even valid research. But, let's not get caught in that thicket. We don't need to even go there. Once again small effect sizes come to our rescue.

The Lasting Benefits Study, which has traced students after the end of the STAR experiment, showed that students from the small K-3 classes maintained most of the prior differences through the sixth grade (Nye et al., 1993). Comparisons of small versus regular classrooms yielded effect si

zes on the norm-referenced third grade tests of 0.24 and 0.21 for reading and math, respectively (Word et al., 1990). In the sixth grade, three years after the end of any differential resources for the two groups, the effect sizes for comparisons of students previously in small versus regular classrooms were 0.21 and 0.16 for reading and math, respectively (Nye et al., 1993). In other words, the differentials in performance found at kindergarten remain essentially unchanged by third grade after class size reductions of one-third were continuously applied (see figures 6 and 7) and remain largely unchanged by sixth grade after class size returned to its prior levels for another three years. This latter finding leads to rejection of the fall-back model and indicates that class size reductions after kindergarten have little potential effect on achievement

Once again we are confronted with educationally insignificant effect sizes. And even these miniscule results required reducig class sizes down to 13-17 students. And, don't forget about all the methodological flaws casting serious doubts on even these reults.

See tables 2 and 3 (pp. 20-22) of this study for a complete list of effect sizes obtained in STAR.

III. Student Achievement Guarantee in Education (SAGE)

Think of SAGE as STARs ugly step-sister. Although, it is commonly believed that SAGE evaluated the effects of reducing class sizes, it did not. Class size reduction was only one component of the intervention. Here are some of the other components:

1. a longer school day and increased collaboration with community organizations
2. a more rigorous academic curriculum
3. staff development and accountability mechanisms
   

So which of these factors contributed to increased student achievement? We don't know. If you want to implement the SAGE intervention, it's a package deal. You have to do class size reduction plus the three other not-insignificant components.

So, what's your reward at the end of the day? You guessed it -- the usual, small effect sizes:

Effect Sizes After Three Years in SAGE
Math: 0.193
Reading: 0.136
Language Arts: 0.350
Total: 0.213

Once again we have mostly educationally insignificant effect sizes.

Conclusion

The research, such that it is, simply does not suppport the commonly held notion that reducing class sizes will have an educationally significant effect on student performance. There are many other reasons why teachers are giddy for reduced class sizes, but improved student performance is not one of them.

This seems counterinuitive, even paradoxical. I agree. Let me offer an explanation. Ineffective teaching practices are relatively immune to class size changes. The system is just as broken with 15 kids in the class as it is with 30 kids. Throwing a few more kids in a classroom isn't going to break the sytem any more than its already broken. Teachers generally present material inefficiently and without regard to whether students are actually learning. Certainly some kids are learning the material -- the same top 25% who've always learned no matter how poorly the material is presented. Gaining a few extra a minutes a day of teacher time through reducing class size isn't enough to significantly alter this dynamic. As a result failure persists.

On the other hand, effective teaching practices are very sensitive to class size issues. I'll discuss those conditions in part II along with why I ultimately conclude that teacher presentation size is a critical component to increasing student achievement in highly effective instructional programs at least at the elementary school level.

Take me right to Part II.