August 4, 2006

Research -- Such That It Is

Following up on my last post on Kid Writing, let's now look its research base or lack thereof.

The first sign that things may be amiss is, well, the sign over to the right that is posted prominently on the front page of the Kid Writing site.

Here's what the text on the bottom reads:

If you have done any action or scientific research on the Efficacy of Kid Writing on children's literacy achievement, please contact Eileen Feldgus...

Contact me too because I am dying to see what is likely the only scrap of scientific research on the abomination known as Kid Writing.

Some of you (the ones who've actually followed the links to the Kid Writing site) are probably saying how can there be no scientific research on Kid Writing? Right on the front page, there's a research link that brings me to a page that boldly proclaims:

Current Research is Supportive of
the Kid Writing Approach

And, below that we have two "research papers" that appear to be the research base for Kid Writing:
  1. Joint Position Statement of the NAEYC and IRA
  2. New Standards Primary Literacy Committee (PDF)
(NB: both the links on the Kid Writing site are broken, but nothing can escape google-fu.)

There's also a link to the oddly titled Action Research on Kid Writing by Mary Ellen McCloy (doc) on the front page.

Thing is -- none of this is even close to being research. And, more importantly, none of the "research" is on Kid Writing.

The first paper is a position paper by an advocacy group co-authored by the International Reading Association, a whole language shill group. Research is cited in the paper, but none of it is on Kid Writing -- it's all on other people's reading programs. This is the current scam du jour in education research and I'll cover it in more detail in my next post.

The second is a paper on early reading that, like the first, is neither research or specific to Kid Writing.

The third shows some promise -- it's actually about Kid Writing. However, it's not research -- it's based on a survey given to students, parents, and teachers about their feelings toward Kid Writing. The first paragraph tells you all you need to know about the seriousness of the scholarship:

The word kindergarten was created from two German words that mean children'’s garden. As a gardener tends to his garden with patience and care, kindergarten intends to treat young children gently, like flowers ready to unfold, so they can develop to their fullest when they are ready. If you hear the word kindergarten, your mind may conjure wonderful images of children frolicking and playing. There is often an element of surprise to hear that these five year olds follow an intensive curriculum.
I also spotted a fancy bar graph that shows:

An evaluation of the individual student's progress was determined using the Conventions of Writing Developmental Scale (Kid Writing, pp.169-179, appendix D). Using the students'’ portfolios and present journals, thirty-four students from two classes were rated on their writing in September as compared to June, as indicated in Figure 1.

Bonus points to the first person who can tell me what's wrong with this "research."

That's it. That is the extent of the research on the "research based" Kid Writing. Mind you, this program is over ten years old and was actively used in both authors' classrooms. Yet, no one ever thought to measure student learning using a conventional standardized measure of literacy.

Sadly, this is the norm for most educational programs in use today. This is the standard scholarship. This is the reason you get edu-jargon instead of serious research. This is why student performance is so low. And, this is why we have NCLB and Reading First.

Continue on to the Exciting conclusion to our story in Part III: Kid Writing: Education's Scam du Jour.


Peter Campbell said...

So you're suggesting that NCLB and Reading First are examples of good research???

Holy carumba!

If you have the funds and the clout, you can conduct a research study that proves that the moon is made of cheese. If you have lots of funds and lots of clout, you can get that study published and out in the media. And if you are the National Reading Panel (NRP) and are appointed by The National Institute of Child Health and Human Development (NICHD) to conduct a "research-based" inquiry on "scientifically proven" ways to teach reading, then you can have that study form the foundation of educational policy for an entire country.

The National Reading Panel came to the conclusion that "phonics instruction appears to contribute only weakly, if at all, in helping poor readers apply [decoding skills] to read text and to spell words." (quoted in Garan, Elaine. 2002. Resisting Reading Mandates: How to Triumph with the Truth. Heinemann. Portsmouth, NH., p. 47; taken from the NRP Report of the Subgroups, Chapter 2, p. 116) But, thanks to those wonderful public relations people from Widemeyer Communications, the Washington PR firm hired by McGraw-Hill to promote Open Court in Texas and to write the Summary Booklet and produce the promotional video that explains the NRP's "research," phonics has become (once again!) The Next Big Thing.

It's a no-brainer that Reid Lyon, a guy that worked for the NICHD (the agency that sponsored the NRP), a guy that served as Bush's "reading guru" while Bush was governor of Texas, and a guy that preaches "explicit instruction in phonemic awareness, phonics, guided repeated reading to improve reading fluency, and direct instruction in vocabulary and reading comprehension strategies" may have had something to do with the fact that the NRP's recommendations were largely copied and pasted into NCLB's Reading First program.

See my post on this.

Laura said...

It's called marketing. The kind of research that you advocate is, of course, more accurate. But it is less sexy and sells far less. It does little or nothing to appeal to buyers. It bores them first.

Just because they have not invested the time and fundage in this unprofitable proof does not preclude the program working. It just means there's no proof one way or the other so far, other than your horror and others' experience.

SteveH said...

"If you have the funds and the clout, you can conduct a research study that proves that the moon is made of cheese."

It's a mistake to view everyone as being either on the Democratic or the Republican bus. It's also a mistake to say that all research is only used to prove one's opinion. (although that is a common technique) Difficult as it is, reliable and independent research can and has been done.

However, either you accept certain standards for research (where you can rely on the results) or you don't. Schools can't have it both ways. If a proper education is simply a matter of opinion, then the only option is full parental choice.

I'm willing to accept the position that good research is either too difficult to do, or is tainted by assumptions and biases. This means that schools can no longer talk about "authentic education" or "best practices". They would have to tell parents that the education they provide is simply their own opinion. This would be a breath of fresh air. Of course, this would have to lead to full parental choice - unrestricted charter schools and full vouchers.

Valid research that you can rely on or full choice? Of course, this says nothing about curricula and expectations. Our public school might use valid, research-based techniques for teaching math (they don't), but they only provide an algebra-lite course in 8th grade. Research tells you nothing about assumptions and expectations. That is the job of the parent. Research is a distraction if schools and parents cannot agree upon what constitutes a proper education.

Unfortunately, there are those who view all issues in terms of right wing or left wing boogeymen. Excuse me if I don't play along.

allen said...

There seems to be a bit of confusion. NCLB is a political action, the result of a compromise among various parties. Like any law, it doesn't require research and there have been various laws enacted which makes that abundently clear . The NCLB was enacted to bring some semblence of accountability to the spending of federal dollars in the public education system. I don't believe it's necessary to research the validity of the assumption that with accountability comes responsibility.

To stay on topic, the new flavor of the month, Kid Writing, pretty clearly has no basis in rigorous, or any other kind, of research. And really, why would it? The people who have to be convinced of the value of the program have no interest in whether it teaches kids to write, read or balance a ball on their nose. They have many responsibilities and teaching kids to write, or teaching them anything, is just one of those responsibilities and, on the evidence, not the most important.

The complaints about Kid Writing, and the same complaint can be made of many exciting educational innovations that litter the scrap-heap of public education history, isn't that the research that underpins them is of poor quality but that in general, it doesn't exist. Kid Writing is just the latest in a long line of educational adventures that's based not on any demonstrations of efficacy but on an appeal to conceit.

If efficacy were actually a concern then rigorous, and defensible, measurement would have occurred before unleashing Kid Writing on an unsuspecting public. Like drug testing regimes used by the FDA, Kid Writing would have gone through various stages of testing - teaching rats to write using journaling then proceeding to gauge effectiveness by teaching chimpanzees to write followed by carefully conducted small-scale pilot studies culminating in large-scale testing. But no, attempting to gauge the effectiveness of the program is an afterthought.

KDeRosa said...

Hi Peter.

No I'm not. As Allen setas forth above, both NCLB and Reading First are laws, not research. However, Reading First is somewhat responsible for not weeding out bad research like that relied upon in Kid Writing. I plan on discussing this point ion my next post.

The NRP's findings were as problematic as the Reading First initiative. they suffer from teh same problem. However, let's be clear, the metastudy conducted by NRP was based on valid research, though some of the conclusions NRP study extended beyond any fair inference that could have been drawn from the underlying research. Nonetheless, the research underlying the NRP report is readily available, so your sweeping generalization of bias is weak considering you haven't pointed out any specific instances of bias or any faults in the underlying research.

Your quote relates to the teaching of phonics after the second grade, not to the efficacy of teaching phonics before that point. Again, this is possibly an example of over reaching by NRP. And, your critism is a similar overbraod distortion of the research, but in the opposite direction.

So while you make some valid criticisms of NRP and Reading First, I disagree with your ultimate conclusion.

KDeRosa said...

Laura, I'm wondering if the research related to a potentially life-saving medicine you were taking would you have the same cavalier attitude.

Learning how to read is most likely the single-most important thing children are supposed to be taught in school. Not learning how to read proficiently will pretty much ruin any chance these kids have at further education.

KDeRosa said...

I'm going to agree with Steve, the necessity and provision of a quality education transcends political ideologies.