This report follows an alarming pattern in which the WWC allows the evaluation of the research to include a testing instrument developed by the authors of the education program as a valid measure of success. Out of the 35 studies reviewed, only one (Ross, Nunnery, & Goldfeder,
2004) met WWC standards. And guess what:
The STAR Early Literacy test and STAR reading test are the only outcomes reported in the study. The STAR tests are developed and distributed by Renaissance Learning, which also distributes Accelerated Reader/Reading Renaissance.
The test is a computer-adaptive, norm-referenced test that measures student reading comprehension. It is designed for students who have at least a 100-word reading vocabulary and can be used with all students in grades 1–12. Students read passages of text and fill in key missing words from a set of options (modified cloze procedure).
What's surprising is that even using their own testing instrument, Accelerated Reader didn't perform all that well.
Comprehension. Ross, Nunnery, & Goldfeder (2004) reported a positive and statistically significant effect of Accelerated Reader/Reading Renaissance on third grade student performance on the reading comprehension measure (Star Reading test). In WWC computations, this positive effect was not statistically significant, but considered substantively important according to WWC criteria (an effect size greater than 0.25).
General reading achievement. Ross, Nunnery, & Goldfeder (2004) showed that Accelerated Reader/Reading Renaissance had positive and statistically significant effects on the general reading measure (Star Early Literacy test) for kindergarten, first, and second grade students. According to WWC analysis, the average effect size across grade levels was statistically significant.
With respect to general reading achievement, to get statistically significant results, the WWC had to average the performance of grades K-3. None of the grades individually had sufficient students to achieve a statistically significant result.
Yet on the basis of this one small study using a questionable testing instrument, the WWC concluded that Accelerated Reader has "potentially positive effects" for reading comprehension and general reading, earning the coveted green box and +?.
This seems like extremely flimsy evidence to me and it seems to send a message to publishers on how to cook the books to get the thumbs up from the WWC.