Think again.
The Feds implemented Reading First to force educators to adopt Reading Programs based on scientific research. The Feds offered educators lots of grant money provided they adopt reading programs that were consistent with the research on reading. To effect Reading First, the feds:
sponsored three major reading academies, the Secretary’s Reading Leadership Academies (RLAs). The RLAs were held in Washington, D.C., in January and February 2002, and hosted policymakers and key education leaders from every state and territory in the nation. The academies were designed to help state leaders gear up for the implementation of Reading First, the Department’s program to improve the quality of reading instruction in kindergarten through third grade.The RLA's included a session entitled “Theory to Practice: A Panel of Practitioners.” in which:
The speakers discussed how implementing a scientifically based reading program had brought about great improvements in the reading skills of their kindergarten through third grade students.
The a majority of the panel consisted of principals who had implemented either the Direct Instruction (DI) reading program or the Open Court reading program, two of only three programs that have been research validated.
After the RLA sessions the "policymakers and key education leaders from every state and territory in the nation" had an opportunity to comment on the RLA sessions by filling out evaluation forms.
Normally, such evaluation forms are maintained in confidence and I suspect that the attendees never expected that their comments would come to light. But then a little thing happened on the way to the teachers' lounge ....
The Department of Education's internal auditing department, the Office of the Inspector General (OIG) audited the Reading First program. (I'll have a post on the merits of this latest OIG audit in an upcoming post (quick take: it is laughable).) As part of that audit, they reviewed the evaluation forms submitted by the policymakers and key education leaders from every state and territory in the nation. And now as part of the OIG's Final Audit Report February 2007 you can see some of their comments in all their glory. See pages 25-39.
A few points immediately jump out. Many of these policymakers and key educators:
1. have not accepted the reading research and are not willing to abandon their beloved whole language programs.
2. were a hostile audience.
3. intensely hate DI and open court, i.e., the reading programs that have been validated by reading research.
4. were conspiring behind the scenes to give the impression that DoE was trying to force them to adopt specific curricula.
5. Know all the cliches very well.
These are THE state level policy makers and education leaders, not a bunch of powerless teachers or academic ideologues. These are the people who make the education policy in your state. From, these comments, it is clear that they won't be abandoning their beloved whole language anytime soon. At least not willingly.
And the research for reading is much further advanced than it is for math. Consider this a preview of the math wars five years hence.
31 comments:
I have to say that I like DI in some ways (for the reasons you're always talking about, the fact that it's extremely regimented, doesn't allow for bad teachers, etc), but I don't like some of the theory it's based on or much of the reading materials. I know it's the "only program that's validated by research," but I suspect that one of the main reasons it wins out is because the other programs just don't have such strict rules for implementation. From what I've read (which is admittedly only a few papers and a bit of Zig's book), it doesn't put enough emphasis on context for words that kids are learning. Rebuttal?
Eve, I hope you can explain what you mean. I teach a REACH program which is a DI curriculum. Before reading a story, students are exposed to many of the words that they will be reading in the story.
I completely disagree that DI "doesn't put enough emphasis on context for words that kids are learning," as words are taught prior to students reading a story. In previous programs that I've taught from, we were told to help kids "infer" from the context what the words were or what they meant. Can you imagine the struggle this can cause for ELL or students below grade level?
Some time this week, I'm going to have a post on my blog that outlines a typical REACH/DI lesson. This will give you a better understanding of how and what is taught.
ms-teacher,
I am looking forward to your post. I am really looking forward to an inside look at di. I would especially be interested all the little details that you have to pay attention to, to ensure that the lessons are given effectively. Also, how do you ensure mastery, what are your standards. Perhaps you can also let us know how you "own" the lesson. In otherwords, I imagine there is at least a bit of flexibility built in to the program that allows your individuality to shine.
Stuff like this makes me glad I have my own copy of "Teach Your Child to Read in 100 Easy Lessons" by Engelmann et al. I didn't even know what DI was when I used that book; all I knew was that I liked how the introduction says (if I may paraphrase) "We started with the assumption that if the kids did poorly, it was our fault, or the fault of the materials, NOT the kids' fault."
As for the criticism of DI as being too regimented too scripted too leafy too lumpy, I have several things to say. First, even if you are using a DI reading program alongside a DI math program, that's still less than an hour a day of programmed instructional materials; that leaves plenty of time to be creative in art and music and science and social studies and PE and playtime. School is not about entertaining educators; it is about educating children.
Furthermore, somehow actors follow scripts and do not complain that their creativity is stiffled. Can you imagine a 24 year old actor, fresh out of college, announcing to his director that Hamlet's lines needed juicing up? Just like an actor's script, the DI lesson script must be looked at (if not completely memorized) beforehand; unless the instructor is blandly reading the red words off the page, personality will come through.
And finally, there is the fact that the script works. It frees up the instructor to deal with actual problems, rather than reinvent the wheel each and every lesson plan, hoping this one rolls better than that one last Tuesday. I would have thought instructors would welcome something that both works and makes their lives easier, but clearly I am wrong.
So why don't we go ahead and actually take a look at a real live DI lesson. This one is from Reading Mastery III. The vocabulary exercise is exercise 3. Many of the words are then used in the reading passage.
Is there a better way to teach vocabulary?
Ok D-EdReckoning,
Why don't you update us on your money pit.
Anyways... how long would it take to complete that lesson?
The money pit is never done.
The teaching part takes 20-30 minutes and the indpendendent work part takes another 20-30 minutes depending upon my son's focus.
Spelling takes another 20 minutes, math 30 minutes, and writing takes 30 minutes.
So what I'm getting from the example of the lesson you provided, Ken, is that the kids are asked to memorize and spell the words repeatedly (I'm assuming that's what "until firm" means) before they're even told what the words mean. I'm not saying that these kids should be expected to infer right off the bat what the words are; I completely agree that for low-functioning kids that can be nigh impossible. But I think there can be a grey area between constructivism and direct instruction.
Up here in Canada the bible is something called "Strategies that Work"; it's by Stephanie Harvey and Anne Goudvis, and it follows the Balanced Reading model described by Isabel Beck in the early 70s. It's still relatively teacher directed, but there's a greater focus right away on what certain words mean, and the focus is on automaticity just like Beck, Engelmann and Bruer all agree is necessary for reading mastery.
Eve, that is the decoding part of the lesson. The students are given columns of words to read in isolation, before they see them in connected text.
The difficult words in the first column are typically modeled by the teacher. The words in the remaining columns typically do not have to modelled since they are more familiar to the students.
First the whole class reads each column of words until the entire class can read each column, not just single words, accurately. They do this to avoid the memorization effect. Once the class is firm on all the columns of words, then individual students are selected to read ane ntire column of words. This is a delayed test for decoding ability.
Then comes the vocabulary exercise in which the meaning of a few words are taught.
Then the students are given a connected text passage to read. In the exercise the teacher keeps track of decoding errors and a high criterion is expected, i.e, > 90% accuracy.
Every tenth lesson, the students are given a timed fluency test on a passage they have read.
The last exercise is the independent workbook exercises.
That is pretty much how each lesson goes.
It's still relatively teacher directed, but there's a greater focus right away on what certain words mean
But that is exactly the problem with balanced lit. They confuse meaning with decoding and overemphasize decoding stratagies that focus on extracting meaning from the text. Decoding doesn't work this way.
Often such techniques are merely unproductive for higher performers but they are frequently deadly for lower performers who are more likely to get confused.
So, I agree, the better balanced lit programs do focus on mastery (my son is in one such program); however, they are still problematic.
The stories in my son's balanced lit class are at the same lexile rating as the stories he reads in RM III. but, he has a much easier time reading the RM books because the words used are carefully controlled, even theough many of the words are phonetically irregular. In contrast, the balanced lit books are not carefully controlled for decodability and he has a more difficult time decoding the irregular words since they are unfamiliar. It is the equivalent of throwing the student into the deep end and hopeingthey can tread water long enough until they learn to swim. In RM, the kids enter on the shallow end and gradually make their way to the deep end already swimming.
First of all, I'm really enjoying this discussion. I'm trying to become familiar with a few different reading models so I can analyze them fairly, so I've been really enjoying reading your blog for a positive perspective on DI.
I agree that properly controlling the words in stories is important when it comes to learning phonics, but I'm more concerned with the initial vocabulary lesson. When you're teaching the kids to read particular words, I think DI doesn't put enough focus on metacognition and high level reasoning. The way it goes about teaching words is too abstract. As a contrast, here's a test program by Beck and McKeown from the early '90s:
The program required 4th graders to learn 8-10 words per week within a single meaning family. Examples of these words are “accomplice,” “virtuoso,” “rival,” “miser,” “philanthropist,” “novice,” “hermit,” and “tyrant.” Instruction took place 30 minutes a day, 5 days a week and involved associating words and meanings, playing word games, making up sentences, and reasoning with words (“Would an accomplice be more likely to squeal to the police in return for not having to go to jail, rob a bank by himself, or enjoy baby-sitting?” “Could an accomplice be a novice?”). Every week ended with a multiple-choice test.
Granted, I don't think that's enough words per week to build a sufficient vocabulary, but that's not bad for 30 minutes a day. I have a better example based on this (reciprocal teaching, Palincsar & Brown, 1982) lying around, but I don't have the patience to type it all out at the moment.
I just realized I forgot to respond to your first point (about how decoding works). I agree that DI is potentially great for "intensive" kids, but slowing down the pace for everyone means that the faster kids lose out, especially in the comprehension category. I think the biggest reason for backlash against DI is that it claims to be a solution for every child, and while that's a great prospect, it means that gifted kids spend their days snoring through the lessons. Like the pigs say, everyone is equal, but some people are more equal than others. If every child in the US is taught the same way, you'll bring some kids up to average, but there's the risk you'll also bring kids down to average.
eve,
Direct Instruction implimentation is based on ability grouping.
High Ability, (gifted students) will be able to skip many exercises if they show mastery and move on at their own pace.
The problem is not that DI will slow down high ability students, but the opposite. Because DI allows students to progress through the lessons once they meet mastery, there is nothing to slow them down.
D-Ed has posted on it before, but a quick lazy search didn't readily turn up the info. I am sure he will pipe in.
Eve, let me answer your earlier post first.
The full DI curriculum can be taught in half a day, that leaves the other half of the day for teaching history, science, art, PE, and extra vocabulary like the course you mention.
My experience with my son, who'd I classify as a higher performer, is that his oral vocabulary is more advanced than his decoding ability., so he has extra capacity tolearn more vocabulary, facts, and unerlying concepts than are presented in the DI reading program. But that is not to say that he doesn't benefit from the vocabulary instruction he gets in the DI program. He has learned many words in DI so far and he has learned them all well.
One other quick point, be careful about the peceived need for deep thinking and problem solving in instruction. Often such strategies force the student to spin his wheels searching the problem solving space for answers making the student's limited working memory unavailable for effecting changes to long term memory, i.e., learning. See Why Minimal Guidance During Instruction Does Not Work.
I also think that if the DI people thought the activities you describe resulted in better learning for the instructional time expended, they would have included them since their goal is to teach to mastery as quickly as possible. Food for thought.
Now onto point two.
DI is not meant to be taught to heterogeneous groups. The idea is that students of similar ability will be flexibly grouped together and taught as quickly as they can learn. The lessons are meant to be briskly paced and engaging. See here.
DI's main benefit to high performers (because let's face it these kids will learn in whatever curricula you put them in) is the ability to accelerate their performance very rapidly, increasing their math and language skills so that they are capable of doing algebra and reading difficult texts at an earlier age than is typical. It is the expsoure to these advance learning environments will these kids will find the best challenge for their ability.
I agree that DI is potentially great for "intensive" kids, but slowing down the pace for everyone means that the faster kids lose out, especially in the comprehension category. I think the biggest reason for backlash against DI is that it claims to be a solution for every child, and while that's a great prospect, it means that gifted kids spend their days snoring through the lessons.
I wasn't taught by DI and I spent my days snoring through the lessons. At the very worst, this is not a problem specific to DI.
As others have said, DI depends on separating kids by ability, so the smarter kids are accelerated, which is at least a partial solution.
May I ask what non-DI curriculum you have found that has evidence that:
- smart kids aren't snoring through classes
- smart kids (or any group of kids) are gaining more in comprehension than kids taught through DI?
"Often such strategies force the student to spin his wheels searching the problem solving space for answers making the student's limited working memory unavailable for effecting changes to long term memory"
Are you saying that if kids think hard about something, they won't remember it? If anything, with regular exercise in searching that problem solving space, the search would become more refined and automatic, eventually freeing up working memory.
I'm not talking about spinning wheels, I'm talking about creating a denser and more connected semantic space that helps in future vocabulary retrieval. And if the kids pick up some extra metacognitive strategies along the way, that's great. I don't see enough evidence of DI attempting to develop that rich semantic space, or to teach metacognitive strategies.
I think one thing that behaviourists like Engelmann ignore is that every kid is coming in with a different knowledge base, so a one-size-fits-all decoding program sort of falls apart at the encoding phase. I think that's the main issue I take with DI; it's not all decoding!
Are you saying that if kids think hard about something, they won't remember it?
He's saying they won't remember the solution once they've come up with it.
I don't see enough evidence of DI attempting to develop that rich semantic space, or to teach metacognitive strategies.
According to Project Follow Through, kids taught by DI performed better on cognitive skills. See http://psych.athabascau.ca/html/387/OpenModules/Engelmann/evidence.html. The DI develops that rich semantic space and metacognitive strategies by teaching children explicitly so the children have a large based of knowledge to draw on.
May I ask what evidence you have that another curriculum teaches cognitive skills more effectively than DI?
I think one thing that behaviourists like Engelmann ignore is that every kid is coming in with a different knowledge base, so a one-size-fits-all decoding program sort of falls apart at the encoding phase.
Engelmann doesn't ignore that kids are coming in with a different knowledge base. Kids are tested on first entering the school and placed according to their knowledge base. So a kid who already knows the meanings of the colour words and how to touch something without picking it up doesn't go through those classes.
May I also ask you which curriculum you have found that teaches gifted children more effectively and keeps then from snoring in class?
Eve, I hear you talking the cog sci lingo but that's level one research in education. Good to base a theory on, but doesn't always translate into practice or instructional srategies.
Let's look at some level three research from Project Follow Through. As luck would have it, they tested cogntive skills in PFT. Here's Engelmann's summary:
DI was not expected to outperform the other models on “cognitive” skills, which require higher-order thinking, or on measures of “responsibility.” Cognitive skills were assumed to be those that could not be presented as rote, but required some form of process or “scaffolding” of one skill on another to draw a conclusion or figure out the answer. In reading, children were tested on main ideas, word meaning based on context, and inferences. Math problem solving and math concepts evaluated children’s higher-order skills in math. Not only was the DI model number one on these cognitive skills; it was the only model that had positive scores for all three higher-order categories: reading,
math concepts and math problem solving.
"He's saying they won't remember the solution once they've come up with it."
They might not, but the intent of teaching is not to get kids to memorize what 234 x 7384 is, or what's the distance from LA to Michigan. The intent is to teach skills, am I right?
"Interventions for Students With Learning Disabilities: A Meta-Analysis of Treatment Outcomes" talks about the Palincsar model I mentioned earlier. It states that a combined approach (one that integrates both top-down and bottom-up processes) is more successful than direct instruction or strategy instruction alone. It's at Google Books here: http://books.google.com/books?id=yEEEhcsV7mQC
From another paper: "Follow-up tests were administered to 700 participating students at the end of fifth and sixth grades. Despite the positive effects of the early intervention, student scores slipped below the national norms in the upper grades. Becker and Engelmann noted that students’ decoding skills remained strong, however reading comprehension dropped to the 25th percentile. The researchers concluded that basic vocabulary deficits were the main cause of students’ reading comprehension difficulties, a problem that was exacerbated by the escalating demands of reading material in the upper grades." Whether that's been tested, I'm not sure. What I do know is that the Matthew Effect should have taken over by then, if the kids are as proficient by grade 4 as Engelmann says they are. If they can't infer vocabulary through context by grade 4, that's too much hand-holding. At some point, you have to take the training wheels off.
http://scholar.google.com/scholar?hl=en&lr=&safe=off&q=cache:OUMt67GhPjcJ:mitchellprion.com/Courses/LublinerDiss.pdf
"Reciprocal Teaching: A Review of the Research" by Barak Rosenshine and Carla Meister is another fantastic resource. It states effect sizes of +.32 on standardized tests and +.88 on researcher generated tests of comprehension. One thing that's mentioned (which I think is quite important) is that there were no evaluations of the quality of reciprocal teaching in any of the studies reviewed. That's one advantage that DI has over reciprocal teaching, and that's a methodology I'd like to see implemented with any and all programs that are compared to DI. It's a bit unfair to say that DI beats the competition if the competition didn't have a chance to implement properly.
In "Experimental Intervention Research on Students with Learning Disabilities: A
Meta-Analysis of Treatment Outcomes" by Swanson & Hoskyn, an interesting thing to note is that strategy instruction beats out direct instruction at .72 to .68 when effects of inflation are removed (compared to 1.07 and .91). (That's on page 304; read through it if you like, because I'm not typing it all out!)
And just because one particular comment got under my skin: I followed your lead on the cog sci lingo, Mr. Working Memory :)
Sorry, one of those google cache links disappeared: it's here.
conspiring behind the scenes to give the impression that DoE was trying to force them to adopt specific curricula.
I've not kept up with your blog (sorry); I don't find allegation self-evident. Can you cite something specific? (I rather like seeing educators skeptical of silver bullets.)
Consider this a preview of the math wars five years hence.
Lets hope not. Perhaps the STEM (SciTechEngrMath) boosters could counter this: http://www.wheresthemath.com/
They might not, but the intent of teaching is not to get kids to memorize what 234 x 7384 is, or what's the distance from LA to Michigan. The intent is to teach skills, am I right?
This is a strawman. No one is arguing that that teaching is defined as memorizing measningless facts. To object of education is not to merely "teach skills." The object of education is for students to learn skills which involves some change to student long term memory. If the student cannot recall the skills taught then the child hasn't learned it. Being able to succesfully search a problem space to solve a problem does not necessarily lead to that solution transfering to long term memory.
I also think we have a definitional problem here. You've cited a bunch of research papers comparing direct instruction to something else. In contrast, we are discussing Direct Instruction which is a specific direct instruction program.
Moreover, you're citing a bunch of level one research and some unreplicated level two research. This type of research, such that it is, is all over the board. It's not difficult to show that one intervention performs about the same as the control group or that the intervention has had some small positive effect under research conditions. Where is the replicated level two and level three research?
The best we can say about teaching reading comprehension is that we don't know exactly how to teach it, so we do things that we think increase comprehension.
Placing third or fourth grade DI kids into a traditinal classroom and then measuring their comprehension abilities a few years laterd oesn't tell us much about the effect od DI on comprehension due to the intervening non-DI instruction. We should not expect low performing kids to build their background knowledge and vocabulary in later grades if that knowledge is not taught or id the skills they learned are noit systemeatically built on. We do have evidence that DI third graders decode and comprehend better than other third graders and we have some evidence that DI six graders also comprehend better tahn control groups, though I don't think the evidence is quite yet dispositive.
CLearly what I have not seen yet are the recipricol techniques you are advocatying having a positive effect on comprehension in iddle school grades when compared to a superior academic program like DI. Maybe it's better, maybe it isn't. We don't know because no one has proven it yet.
Hi Eric,
This is what I've read posted by some of the RF panelists on listservs. They claim that the states were trying to game the system. States would probe DoE in their applications trying to determine avenues they might exploit to get their pet programs funded under RF. So if state A found a successful exploit , you could be sure you'd find that vulnerability exploited in the revised applications of states B and C soon thereafter.
There's a big difference between the quest for the mythical educational silver bullet and instructional programs that have been validated to work better than your typical reading program.
One more point, Eve.
According to Rosenshine, the palincsar model you are advocating appears to contain many of the same componenets as the DI model, scaffolding, guided practice, model-lead format, etc.
You should probably checkout levels 4-6 of the DI programs where the instruction of reading comprehension is taught. FOr all you know, the DI model might already use similar techniques used in the Paincsar model.
"He's saying they won't remember the solution once they've come up with it."
They might not, but the intent of teaching is not to get kids to memorize what 234 x 7384 is, or what's the distance from LA to Michigan. The intent is to teach skills, am I right?
Yes - like how to calculate 234 x 7384 - or more precisely how to calculate the products of any two real numbers.
Or how to use Pythagorus's theorem to measure the straight-line distance between two points so they could calculate the distance between LA and Michigan by knowing each cities' longitude and latitude (ignoring for the moment that LA and Michigan are on the surface of a globe).
The evidence from cognitive science seems to be that if you have children try to solve problems like 234 x 7384 by discovery learning, they may eventually solve the problem but they're less likely to remember the method used to solve it. The mental thing that seems to be going on here is that the student remembers all the methods they tried and failed about as well as the one that worked.
"According to Rosenshine, the palincsar model you are advocating appears to contain many of the same componenets as the DI model, scaffolding, guided practice, model-lead format, etc.
You should probably checkout levels 4-6 of the DI programs where the instruction of reading comprehension is taught. FOr all you know, the DI model might already use similar techniques used in the Paincsar model."
Oh, I have no doubt. Like I said, there are some things that I like about the DI model. I think a middle point between the two could be quite promising, which I said right at the beginning. I think we're both arguing different shades of grey :)
Tracy - Palincsar's model doesn't involve discovery learning at all. Strategies are modelled first and responsibility is gradually transferred to students.
My understanding is that reading strategies are relatively easy to teach to students efficiently. Students know how to employ startegies when they know the vocabulary and underlying concepts. It is when they don't know tis that they stumble.
I've also noticed that these techniques are being embraced by the whole language cultists which means they will surely be misused as some substitute for teaching decoding properly.
Tracy - Palincsar's model doesn't involve discovery learning at all. Strategies are modelled first and responsibility is gradually transferred to students.
I think the conversation is shifting on me - are you now arguing the merits of Palinscar's model compared to DI?
Anyway, I wouldn't be surprised if there are improvements to be made on the DI curriculum, or even completely different methods that are better. What makes DI different is that it was highly informed by testing and improvement (rather than writing a book and assuming all the methods would work), and Engelmann figured out methods of ensuring it was reliabily replicated.
I don't know of another curriculum that has been tested so thoroughly but of course that doesn't mean that there isn't one. Engelmann seems to have been using testing and quality control methods whose principles are similar to ones in engineering. I don't think these methods are patented, and I see nothing to stop developers of another curriculum undertaking the same process. Just whatever they come up with should at least have such good test results as DI.
(Of course there are some differences between engineering tests and testing children. For example, it's very hard to get an ethical committee to approve dropping small children from various heights to see how shockproof they are, :) but there's no problems with doing that to a cellphone. I'm talking about applying similar principles, not treating kids exactly like silicon chips.)
I looked at that DI lesson. Whew, it is mind-numbing. No wonder EVERYBODY hates it. It is horrid.
Thomas
Yes, Thomas, almost as horrid as those legions non- and poor readers all the lollipop and rainbow programs create.
Post a Comment