Rope-a-Dope

In the Dallas Morning News, Sandy Kress takes on Charles Murray's (in)famous WSJ Op-ed article and proclaims:

Charles Murray's essay reminds me of the effort of the boxer who never lands a punch but hopes to impress the judges by dazzling dancing and ducking. The problem: He is a sitting duck for the knockout punches. Here they come.

Murray may be against the rhetorical ropes, but I think it's more a matter of Murray giving Kress the rope-a-dope than Kress setting Murray up for a knockout.

Kress takes issue with Murray's assertion that NCLB "has not had a significant impact on overall test scores and has not narrowed the racial and socioeconomic gaps" and points out that there has been a statistically significant increase in NAEP scores:

On fourth-grade math national scores from 2000 to '05, the white-black gap closed from 30 points to 26 points, the white-Hispanic gap has closed from 26 to 21 points, and the nonpoor-poor gap has closed from 26 to 22 points.

In fourth-grade reading, the gap over the same period for white-black has closed from 34 points to 29 points; for white-Hispanic, from 35 to 26 points; and for nonpoor-poor, from 34 points to 27 points.

All these gap-closing statistics are statistically significant. Clearly there's much more progress to achieve, but, contrary to Mr. Murray's assertions, the gains since 2000 have been of historic proportions.

Kress may have won the opening battle, but it is likely to be a Pyrrhic victory. Murray will likely win the war in the long run. And, the reason has to do with the highly-controversial concept of IQ and racial inequalities and the nature vs. nurture debate.

Unfortunately for Kress, the weight of scientific authority is increasingly falling on the IQ is mostly determined by nature/genetics and is mostly immutable side of the argument. This is not fashionable among the largely liberal intelligentsia and it certainly isn't carved in stone just yet. Faced with dwindling scientific support, the nurture side of the argument has resorted to merely denying the entire concept of IQ and painting the nature side as racists and Nazis.

Because of this Murray is dancing around the issue without bringing up IQ. And, Kress is taking advantage of transient state increases in minority achievement.

Let's look at what's going on.

In recent years, schools have made a grudging effort to improve teaching of traditionally low performing students. And, as the NAEP scores are showing, that effort is starting to pay off. But, gains remain small (though statistically significant, but that doesn't necessarily make them "significant"). We know from Project Follow through and subsequent research shows us that on average low-performing kids can achieve like your average kid with better instruction.

This result, however, is dependent upon not improving the instruction given to the average kids. The average kids were the control group.

My theory is that this is what is causing the test score increase we see today. Political pressure has caused instruction to improve slightly for low performers. But the focus is almost exclusively on low performers and schools serving large percentages of low performers. It is business as usual with the average and higher performers. They are presently passing the low standards tests already, so they're still getting the same instructional witch's brew they've been getting.

But what if we started to improve instruction for the average and high performers as well? Guess what? Their tests scores will start going up as well. Check out Fig. 5 on Page 5 of this article.

The achievement of all kids can be improved if we improve their instruction. In fact there's reason to believe that the achievement gap will, if anything, increase, with better instruction:

With more emphasis on the higher performers, their performance could have been accelerated more dramatically.

It all comes back to IQ. On average, higher IQ kids will find a way to learn more if instruction is held constant. As a result, there will always be an achievement gap at least in the near future.

So what we have now is a short term transient situation which is the result of unequal educational improvement. We will eventually reach steady state again.

What Murray was really saying is that you can raise achievement (at least temporarily), but it's much more difficult to increase IQ. Eventually, the inability to change IQ will result in an achievement gap. The only way to eliminate the achievement gap is to mask it with "pass percentages, not mean scores."

I'm not vouching for the rest of Murray's argument, but this part he's gotten right.

Taking Advantage of the Poor

The Richard Rothstein poverty meme is the hot topic in the edusphere.

I'll paraphrase Rothstein's argument as:

The effects of poverty are the primary cause for low student achievement among poor kids. And while schools can be improved, substantial improvement in the achievement of poor kids is not possible until the effects of poverty are eliminated.

Would you be surprised if I told you that this argument has resonated with many educators and other soft-headed education thinkers? You shouldn't. That's because the implied solution to Rothstein's argument invokes the educrat's favorite solution to all education problems:

1. have taxpayers throw money at the problem and
2. require them to do nothing.
   

Then there's the extra bonus of providing a launch platform for attacking NCLB fo rdiscriminating against poor kids.

The question remains: is Rothstein's argument valid? Let's find out.

Let's get beyond the fact that the premise (the effects of poverty are the primary cause for low student achievement among poor kids) remains unsubstantiated. Let's, instead, focus on the term "poverty" so we can see what it really means to be living in poverty in the U.S.

According to the census, in 2005 a family of four could have an income as high as $19,806 and still be living below the poverty threshold. Not a princely sum by any stretch of the imagination, but not exactly destitution either. And wait until you get a load of what isn't included in "income." Income doesn't include:

noncash benefits such as food stamps (about $2,200), housing assistance (about $5,400), Medicaid (about $6,000 for a family of four), the State Children’s Health Insurance Program (SCHIP) (about $1,000 per child), energy assistance (about $400), the school lunch and breakfast programs (as much as $600 per child), and the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) (about $400 per person). It also does not count refundable tax credits such as the Earned Income Tax Credit (EITC) (about $1,700), because they are “post-tax.” (All figures are average benefit amounts in 2002 regardless of family size, unless otherwise noted.) (Source)

It also doesn't include such assets as homes:

according to American Housing Survey for 2001, about 46 percent of poor households owned a home, with the median value being about $86,600 (about 70 percent of the median value for all homes), and the equity value being about $52,800. (About 58 percent had paid off their mortgage.)

Nor does it include the income of cohabitors and nonfamily household members.

Then we have a few of these charming factoids to put "poverty" into better perspective:

• Seventy percent of "poor" households own a car; 27 percent own two or more cars.
• Ninety-seven percent have a color television. Nearly half own two or more televisions.
• Nearly three-quarters have a VCR; more than one in five has two VCRs.
• Two-thirds of "poor" households have air conditioning. By contrast, 30 years ago, only 36 percent of the entire U.S. population enjoyed air conditioning.
• Sixty-four percent of the "poor" own microwave ovens, half have a stereo system, and over a quarter have an automatic dishwasher.
  
• As a group, the "poor" are far from being chronically hungry and malnourished. In fact, poor persons are more likely to be overweight than are middle-class persons.
• Nearly half of poor adult women are overweight. Despite frequent charges of widespread hunger in the United States, 84 percent of the "poor" report their families have "enough" food to eat; 13 percent state they "sometimes" do not have enough to eat, and 3 percent say they "often" do not have enough to eat.
• The average consumption of protein, vitamins, and minerals is virtually the same for poor and middle-class children, and in most cases is well above recommended norms.
  
• Poor children actually consume more meat than do higher-income children and have average protein intakes that are 100 percent above recommended levels.

Most of the rest of the world would dream of living in this kind of poverty. In fact, our neighbors to the south are beating down our door to get a little American-style poverty. Hardly paints the picture of Dickensonian abject poverty does it?

But for a small fraction at the very bottom, it is the height of absurdity to think that our poor suffer from the effects of poverty of the variety that Rothstein is describing. This is not to say that they don't face more challenges and have more problems than your average middle class family. But lack of money, and all the things it buys, is certainly not the root cause of their academic woes. These people don't have enough money to be rich, but they have too much to be truly considered poor, much less suffer from the destitution brought on by real grinding poverty.

Our definition of poverty is ridiculously overbroad and includes way too many people who are not poor in the traditional sense. It's like when recent studies claimed that bullying is rampant in our schools and then included under the rubric of bullying not only physical behavior (hitting), but also verbal behavior (name-calling, threats), and phsycological behavior (rumors, shunning-exclusion). Basically, it took a lot of harmless and minor things ordinary children do and turned them into examples of bullying. And by doing so, it greatly overstated the problem of bullying.

This is the same problem that infects any serious discussion on poverty and the plight of the poor. By vastly overstating who is poor, we diminish the effects anti-poverty measures will have on those that aren't truly poor. And, to think that the eradication of this inflated notion of poverty is going to have a significant effect on curing the existing academic failure affecting our schools is foolish, at best, and dangerous, at worst.

It is merely the lastest convenient excuse for those that want to maintain the status quo.

AFTie Ed Hearts Rothstein

I'm going to go ahead and disagree with AFTie Ed who disagrees with Kevin Carey who disagrees with Richard Rothstein who argues that:

reforms aimed at education alone are doomed to come up short, unless they are tied to changes in economic and social policies to lessen the gaps children face outside the classroom.

A lack of affordable housing makes poorer children more transient, and so more prone to switch schools midyear, losing progress. Higher rates of lead poisoning, asthma and inadequate pediatric care also fuel low achievement, along with something as basic as the lack of eyeglasses. Even the way middle- and lower-class parents read to their children is different, he writes, making learning more fun and creative for wealthier children.

"“I would never say public schools can'’t do better,"” Mr. Rothstein said. "“I'’d say they can'’t do much better,"” unless lawmakers address the social ills caused by poverty.

After an introductory paragraph based entirely on leftie talking points (which even most Democrats probably don't truly believe anymore), Ed settles into his main argument:

I taught low-income emotionally disturbed adolescents. Among the shining jewels of my memory are the moments where I made a difference in these kids' lives. But I can point to as many moments where I felt like I was shoveling sand against the tide. Some of that tide came from school based problems like a lack of textbooks and good professional development. But more of it was built into the challenges that my kids faced every day. And in recent years that tide seems to be running harder. Improving healthcare, public safety, and housing and wage standards would have made a world of difference to these kids.

I'm going to go out on a limb and assume that Ed didn't do his teaching during the great depression or in rural Mississippi, so it's a good bet that any school that didn't have sufficient "textbooks and good professional development" was due to the school's own fiscal mismanagement and not through any real lack of funding.

And, while no one doubts that poor (or what most of the rest of the world would consider to be rich) kids face extra challenges, the notion that the tide is "running harder" today only exists in the make-believe land of leftie talking points and not in the real world, highly-misleading linked-to poverty articles notwithstanding.

My main beef is with Ed's last sentence. There is absolutely to credible evidence that improving "healthcare, public safety, and housing and wage standards" would have made one whit of difference with respect to raising student achievement. All of these things have improved dramatically since, say, the early 70s and student performance, as measured by NAEP has hardly budged.

This idea, of course, comes from Rothstein who basically pulled it out of his ass and started parading it around like it was somehow fact. It is one of those fashionable ideas that seem so compelling, kinda like the concept of Santa Claus, until you realize that Santa Claus doesn't exist.

Then Ed writes:

Rothstein argues that standards, discipline, accountability, improved professional development and smaller classes do matter, but that we can'’t really solve the problems that poverty injects into education without focusing on poverty itself.

Ok, let's break this down. Let's take the last part first: "we can'’t really solve the problems that poverty injects into education without focusing on poverty itself." As I set forth above, Rothstein pulled this out of his ass. We spent almost a billion dollars on Project Follow Through proving that you can indeed solve the "problems poverty injects into education" simply by improving how you teach these kids. In fact, the AFT actually endorses the instructional program that prevailed in Project Follow Through, no doubt because it is, in the AFT's own words, "what works." Ed start reading your employer's own website.

Now let's deal with the first part: "Rothstein argues that standards, discipline, accountability, improved professional development and smaller classes do matter." To Rothstein, these are merely slogans that are sufficiently malleable to include all the fashionable ineffective crap that gets taught in our classrooms every day. You know, the stuff that has people howling for education reform. Rothstein and his amigo Kozol are usually cagey when it comes to any sort of specifics, but they occasionally slip and give us a peek at what they really think. It's not pretty.

Let's not mince words. Rothstein couldn't care less about "standards, discipline, accountability, improved professional development and smaller classes" as a means for reform; he's only for advocating for a socialist utopia in which no poverty exists, which technically is impossible due to how we define poverty. Education reform is merely a means to that end. In this respect education is just like the environment and global warming.

To continue:

Criticisms of this idea center on how it will harm the atmosphere of reform rather than its truth. Or that Rothstein's goal is to be an apologist, so the argument should be dismissed. That'’s education reform as jingoism. And its simply poor form to characterize Rothstein as saying instructional reform "should be ignored until some far-off day."

Actually, criticisms of this idea center on how it is so outright wrong and factually unsupportable which are at least two good reasons why it should be dismissed out of hand.

Ed winds it up by stating:

Without it, all the facts in this debate so far rest with Rothstein.

And here we come full circle because the only way the post makes sense is if we substitute "leftie talking points" for "facts" which we know from the first paragraph is exactly what Ed has in mind.

Update 1: Kevin responds to AFTie Ed with 56% more statistics. Of course, he does throw in this line "They could have done a lot more to help those poor children, but they by and large didn't make them poor to begin with." This administration, like almost every administration in the recent past, has done more than the previous adminsitration to fight poverty. So, while it is always true that we could have done a lot more, it's not like the current adminsitration has been pikers when it comes to the poor. Time to scrape off the Kerry-Edwards bumper sticker.

Update 2: Julie at School of Blog chimes in: "it is also obvious to all of us that we can eliminate a good chunk of that achievement gap if we start talking about the real problems, the ones that are even harder to fix than poor schools and inadequate teachers." I must be stupid because it's not obvious to me. Show me the money, Julie. Show me a real life example in which low-SES kids (controlled for IQ) were given money, health insurance, food, better housing, and whatever else you want to throw at them and the result was higher student achievement. I'll wait.

Some Light Reading

I've been meaning to write about a few topics, but what with the ongoing debate with anti-NCLBers and all and an anticipated heavy workload tomorrow I have a feeling I might not get to it. So I thought I'd leave you with some reading assignments you might find interesting:

The First is Why Minimal Guidance during Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching which is a very good debunking of constructivist nonsense and explains why they fail in practice. I especially like the "minimal guidance" name they use which is not only accurate but provides a good explanation why these teaching techniques are so popular with educators. Hint: it's the minimal work angle.

The above paper is based on cognitive load theory which is explained well in Research into Cognitive Load Theory and Instructional Design at UNSW. Another good weapon to add to your constructivist slaying arsenal.

Lastly, we have Amanda's Story which shows what can be accomplished with good instruction and a lot of hard work with a severely cognitively disabled girl (IQ of 62). It is a fitting conclusion to the above two articles because it shows what can be accomplished when you teach efficiently according to sound scientific principles instead of wasting time with disproved quackery.

The comments are open for anyone who wants to give me their impressions on these articles which I hope to get to this weekend.

I'll take Education for $500, Alex

"Toothaches, Global Warming, and the Iraq War."

I thought the topic Was Education, Alex?

It is. Do you want me to repeat the answer?

You better, Alex, I'm confused.

Ok, "Toothaches, Global Warming, and the Iraq War."

No idea, Alex.

Do any of the other contestants know the question?

Crickets, crickets, crickets.

The question was "What are three popular reasons anti-NCLB educrats give for being against standardized testing?"

No, really. I'm serious.

Go check out Professor Dana Rapp's arguments in the ongoing Edspresso debate going on this week. You'll especially like when the website editor, Ryan Boots, had to request that he stay a little more focused standardized testing arguments--to which the good professor responded that no, in fact, those were his arguments.

And, don't think for a minute that the professor is just some lone nutter. We have our own little NCLB/standardized testing debate going on here and here. And, guess what? Our anti-NCLBer is making the same arguments! Coincidence? I think not.

And, we have a little something Edspresso doesn't, our ant-NCLBer is hurling around such delightful epithets as "racist," "Nazi," and "Hitler." Beat that, Edspresso!

Update: And at least my anti-NCLBer showed up for work today.

When Anti-NLCBers Attack

Peter Campbell of Transform Education [Ed: Return to the Status Quo Ante NCLB Education would have been more accurate] responds to my post The Hive Mind's New Buzz in which I criticized the anti-NCLBer's new meme that "English language learners (kids whose primary language is not English) and learning disabled kids (kids of average intelligence who are underperforming) can't be taught to read at grade level in a timely manner" and, as such, NCLB should be repealed, burned, buried, and the ground salted so we can return to the halcyon days when kids didn't learn how to read and no one really cared (except for a few marginalized crackpots).

Peter posted a longish comment in which he disagreed with my tough-love [Ed: fair but firm] criticisms which I'm reposting here. Since I just learned how to change font colors, I'm going to put my rebuttals in blue. Anyway, here's Peter:

Since KDeRosa cited my blog post out of context, I thought I'd clarify this a bit more.

Let's play a game, shall we?

Goodie, I love games.

You've heard of the poker game "Texas Hold 'Em" that's sweeping the nation, right? Well, the game I want to play is called "Texas Test 'Em." Here's how it works.

1) You create a one-size-fits-all test for kids. This includes kids who can barely speak English and kids who cannot read at grade level due to a developmental disability, learning disability, etc.

In every class I've ever been in, they only gave one test to every students (one size fits all, if you will). They didn't give one test to the kid who didn't speak English well, another test to the kid who wasn't that smart, and another test for the rest of the kids. (This is not about giving accommodations to non-cognitively handicapped kids.) You had to pass the third grade reading test if you wanted to say you were proficient in the third grade reading standards set by the school. Otherwise you went to summer school, learned some more (hopefully), and then retook the test.

I don't know what all the fuss is all about under NCLB now that the states are setting the standards and giving the tests. Remember, if the schools were doing their job properly and not socially promoting kids who they hadn't been taught, we might not have NCLB today. Unless, of course, you believe that this was a plot hatched by Karl Rove to steal tax dollars and funnel them to his fat cat friends, in which case I'd suggest adjusting your tin-foil cap.

2) You tell the kids that fail the test that they are stupid, a.k.a. they "do not meet grade level expectations."

Hold on, cowboy, you're jumping to conclusions. Why are we blaming all this on the kids? Could it be that maybe, just maybe, the schools are at fault by not adequately teaching these difficult-to-teach kids. NCLB doesn't blame the kids. NCLB blames the schools. I believe that that is where the responsibility rightly belongs. Some believe the responsibility should be shared between the schools and the students (and their parents). But, almost no one believes that kids are solely to blame with the antics that pass for teaching today. But I digress, certainly if a student has not passed the test, which presumably is aligned with the state standards, it's a good indication that he has not met the state's grade level expectations. The test merely tells us that there is a problem with the education at the school. Don't shoot the mssenger.

3) In some states, e.g., Florida and Texas, you make these stupid kids repeat the grade they failed because this is good for stupid kids.

Repeating a grade may not be the best thing for failing students (the research, such that it is, is mixed and inconclusive), but, the alternative, social promotion, hasn't worked too well either now has it? Maybe this whole grouping by age idea instead of grouping by ability was a bad idea from the get go. But again, why are you just blaming the students? Show me a failing student and I'll show you a kid who was in all likelihood inadequately taught.

4) Keep repeating this cycle until all the stupid students have dropped out or have been imprisoned.

Yes, it is awful when schools fail to adequately teach their students, especially when so much money is spent on education. Many people viewed this as a bad thing. That's why we have NCLB -- flawed perhaps, but better than what we had before.

I love this game!

Hey, I thought we were going to play a game. That wasn't a game. That was a series of weak arguments based on a faulty premise. I feel gypped.

Yes, you've read the research that shows that holding kids back and making them repeat a grade actually increases the likelihood that they will eventually drop out of school,

What happened to steps one and two? Step One: Teach kids poorly. Step Two: kids don't learn and fail. Step Three: Kids held back. Maybe if we worked on step one a bit more, we'd reach step three a lot less often. And, what about all the research that says that kids who don't learn in school are the ones who tend to drop out. It's the failure to learn that's causing the problem, not the failed test. This is one of those root cause issues that the left thinks is so important.

and you've read the research that shows that the vast majority of prison inmates dropped out of school,

See above re root causes. And bear in mind that correlation isn't causation.

and you've read the research that shows that per pupil expenses in this country are about $10K per student whereas per prisoner expenses in this country are about $30K, but you think that most of this crap published by educrats is just more liberal whining.

Liberal whining, perhaps. Irrelevant, definitely. And, I think you lost you're liberal creds when you started calling kids stupid, you meanie.

And, yes, you know that a kid with an IEP has been diagnosed [giggle] as such after thorough evaluation by a team of trained experts [giggle] who know a thing or two about learning disabilities [that they likely caused or made up] , and you know that if the IEP says that a 4th grader reads at a 1st grade level [because his school failed to teach him in grades K through 4 inclusive] that that means that this child reads at a 1st grade level, and you know that if this child takes a 4th grade reading test without any kind of accommodations [i.e., someone answering the questions for him] that this child will more than likely fail this test, you still want to fail this stupid kid [there you go again] because he needs it and it's good for him.

No, I want to fail his school for failing to teach him. Then if the school doesn't shape up, I want to close the school so it doesn't damage any more kids.

After all, you believe that all children can learn [Indeed I do, though you apparently don't], and that all kids -- ALL of them -- should be held to high standards.

By "ALL" you must mean 99% of students, because 1% are already excluded from "the high standards" under NCLB and get to take an alternative assessment, i.e., held to a low standard. We may disagree whether 1% is too low, but I'd rather err on the side of excluding as few kids as possible who might be capable of meeting the "high standards" (which, in actuality, are already pitifully low).

Now, if you would please do me one more favor. Please translate several short passages from the ancient Persian philosopher Rumi from Farsi to English. I recognize that this might be challenging. But I believe that we should hold you to high standards.

No problem, just send over a highly-qualified teacher in translating complex passages from Farsi to English who is willing and able to teach me for a year and I'll get right on it. Oops, I forgot that you didn't know that there was this magical "teaching step" that is supposed to come in between the setting of the standard (in this case learning to translate Farsi) and passing the test (actually translating a passage). I understand the misconception though because what passes for teaching in most schools is mostly non-existent.

And since we're asking for favors, could you do me a favor and translate the latin term "non sequitur" into English for me?

If you object to this proposal, then you are a racist bigot [I was getting worried, I thought you'd never pull out the race card. I'm so relieved] who does not believe that all educational policy experts like yourself can learn. You should really dream bigger, friend.

Lucky for me I didn't object then. And, watch who you call friend, pal.

Hopefully, Peter realizes that we kid because we love. But seriously, Peter, you need to examine some of the assumptions you're basing your arguments on.

Update: the comments are flying fast and furious over at the original post.

Education's Scam du Jour

This is the stirring conclusion of a multi-part post. Although I'd like the think all my posts make sense in and of themselves, you might be better off starting at Part I. We'll wait for you to catch up. Promise.)

In this post, we were discussing the research base (or lack thereof) for Kid Writing and I indicated that:

The first paper is a position paper by an advocacy group co-authored by the International Reading Association, a whole language shill group. Research is cited in the paper, but none of it is on Kid Writing -- it's all on other people's reading programs. This is the current scam du jour in education research and I'll cover it in more detail in my next post.

Such is the sad state of education research today. When a program claims to be "research based" what the program's developers mean is that the program contains bits and pieces of amorphous concepts (i.e., phonics, phonemic awareness, fluency, and the like) that were present in real researched based programs.

Let me explain by using the National Reading Panel as an example.

The National Reading Panel was convened to search through the existing research on early reading and identify specific programs that worked. The result was Report of the National Reading Panel "Teaching Children to Read."

The first thing the NRP did was to throw out all the non-scientific research. This eliminated about 90% of the research available for review. This alone speaks volumes about the quality of education research.

Having whittled down the research to exclude the crap, the NRP next tried to identify the components of the more effective beginning-reading programs. As I'll describe shortly, this was a fool's errand. One conclusion of the NRP was that the effective programs have common features: phonics, phonemic awareness, decodable text, oral practice formats, and a few others.
This is bad enough, but then the NRP made an even worse error, they formulated recommendations based on these common elements. The NRP's recommended that if a program included phonics, phonemic awareness, decodable text, oral practice formats, and so forth, it will also be highly effective. This is a logical fallacy.

Engelmann described the illogic well in his Education Week article "The Dalmatian and Its Spots" He also has a more in-depth treatment of it here (pdf) for you geeky science-types. I'll try to summarize it briefly.

Here's a non-education example of the NRP's faulty reasoning:

1. If a dog is a Dalmatian, it has spots.
2. Therefore, if a dog has spots, it is a Dalmatian.

Hopefully it is clear that the second statement does not follow from the first. Lots of dogs have spots -- English setters, some terriers, sheepdogs, and many mutts, not just Dalmations. Here is the NRP's parallel logic:

1. If a beginning-reading program is highly effective, it has various features: phonics, phonemic awareness, and so on.
2. Therefore, if a program has these features, it will be highly effective.

The NRP went looking for dalmations, identified spots as a common element (why not paws or snouts) and concluded that any dog with spots must be a Dalmation. Again, Engelman explains why the conclusion is faulty:

There is a lot more to a Dalmatian than having spots, and a lot more to programs that generate superior outcomes than having the features that are specified in recommendations. The additional features would include the amount of new material introduced on each lesson, the nature of the reviews that children receive, the ways in which the program tests mastery, the number of times something is presented in a structured context before it occurs in other contexts, and many more technical details about how the material is sequenced and field-tested.

Engelmann is saying that not only was the NRP's recommendation based on a logical fallacy, they didn't even identify the right "spots." You might not like Engelmann or his (highly effective) instructional programs, but I trust his identification of the "spots" in his own program over those identified by the NRP, none of whose members ever developed a highly effective beginning-reading program.

More pernicious is the fact that spots have been removed from the Dalmation; they're not Dalmation spots anymore, they're just spots. "So the analysis moves from a more careful articulation of each Dalmatian (effective program) to an elaboration of spots, now freed from the constraints of the effective program."

By removing the spots you start down a road to disaster. What if you didn't identify the right spots or missed a few? What if the arrangement and interconnection of the spots was important? Engelmann analogizes effective reading programs to complicated inventions, like the airplane, to drive the point home:

The Wright brothers had to orchestrate hundreds of specific details to make a flying machine. Yet, if any one of those pieces had been out of place or misconstructed, the machine would have failed. So it is with educational programs. You can't provide teachers with simplistic slogans and expect them to create programs that are highly successful because the criteria expressed through the slogans are far too general to result in effective programs. The field has to recognize that because highly effective educational programs are inventions, there is no intellectually honest way to describe their structure or why they are highly successful without presenting a myriad of criteria...

If you think about it, you see that the program has to be an orchestration of detail. If it weren't the moment-to-moment performance of the students would not be smooth and successful but lumpy, with no control of tiny details that could make it smooth.

As usual the education industry has ignored Engelmann and we are left with just spots. If you can say you have the right spots and pass the laugh test, you can claim that you're program is research based. Once you have the monicker "research based,' they'll let you pass through the flimsy Reading First gateway.

This is why a program like Kid Writing can claim it is a research-based program even though no research has ever been conducted on it. This is why Kid Writing can imply that it will be highly effective, when it has no evidence of success.

Thank the NRP.

Shorter Rapp

There's a new debate going on over at edspresso on standardized testing starting today.

Professor Dana Rapp takes up the anti-testing mantle first and to spare you the indignity of having to plow through the incoherent mess of an argument that it is, I'm going to give you the short Rapp version with my helpful color commentary.

According to Rapp, standardized testing is bad because it:

• eliminates recess
• links teacher salaries to test scores (like the unions would allow this)
• makes kids sick
• makes teachers dislike their jobs (see bullet two above)
• makes kids learn less
• made the baby Jesus cry
• is unreliable and/or unethical
• makes schools dehumanizing places
• evaporates enriching opportunities for students
• erodes local control of curriculum
• causes kids to be doped into educational submission by Big-Pharmacy
  

(Ok, I added one of the above bullet points (dare I say the most ridiculous one) see if you can guess which one.)

Left unsaid is whether it is the standardized tests that cause this parade of horribles or the schools' own bad decisions when the standardized tests showed that their students weren't learning anything.

But the parade's not over yet, here come the floats. In paragraph two Rapp turns his attention to the capitalist pigs who are profiting on the back of schools' failure.

Testing is a booming market where companies like McGraw-Hill and Harcourt-Brace are reaping record profits with the sale of the textbooks, tests, practice tests, and improvement kits.

Oh, the humanity. Textbook publishers are actually making profits selling ... textbooks. Have they no shame? My monocle almost popped out when I read that line. But it gets better:

As testing proceeds to earlier grades, even kindergarten, CEO'’s and industrial "“leaders"” can rest even more assured that future employees will not have the skills, knowledge, dispositions, and collective consciousness to recognize and act to change disparities of wealth, loss of jobs, lack of health care, and corporate corruption in the organizations in which they work.

Huh? So you mean to tell me that when my son took that fancy DIBELS test in kindergarten that it wasn't to see if he was at risk of reading failure as I so naively suspected at the time, but rather to assure that the plutocrats will be able to beat him out of health care twenty years hence? Lucid criticism or unintentionally hilarious? You be the judge.

Then we come to yet another parade of miserables (cut and pasted write from the socialist's party platform no doubt) in which we find out that standardized testing is neither Santa Claus nor the Tooth Fairy. A snippet will suffice:

Often over-shadowed by a focus on higher standards and test scores by politicians, who are anything but accountable themselves [Ed: except at, you know, the ballot box], are deeper social problems that increased test-scores will never address. At this time in the United States: 5% of the population has 95% of the wealth; middle and lower income citizens are working 10 hours more than 1990 (that is 180 hours a year away from families) for less money; U.S. citizens have nearly four weeks less vacation time on average than do people in almost all other industrialized countries; at least 25% of US citizens do not have health coverage; half the new jobs created are at minimum wage (that is $10,800 a year); one in five children in the U.S. go to school hungry; the U.S. is bankrupting it’s future generations by wracking up a record deficit; programs for the elderly are being cut; global warming and environmental degradation are more apparent by the day; and, Iraq and Afghanistan are out of control. [And soylent green is made from ... people !!!]

Reminds me of the Simpsons episode when Homer asks, "If they can send a man to the moon, why can't they make my shoes smell good?"

So in honor of Homer J Simpson let's summarize Rapp's "argument" (see, I know how to use ironic scare quotes too) as: Standardized testing: won't make your shoes smell good.

The Hive Mind's New Buzz

I'm starting to see the educrats loading up the anti-NCLB siege guns with new ammunition. Apparently, the old ammunition wasn't working.

There is definitely a new anti-NCLB meme forming in the edusphere. The hive has begun to swarm. (But who is the queen bee?) Here's one formulation courtesy of Transform Education:

I keep looking for NCLB's Achilles heel. I think it has two: (1) testing of Limited English Proficiency (LEP) students and (2) testing Learning Disabled (LD) kids at grade level. When I give my talks on education and NCLB, the majority of people I speak to are outraged that non-native speakers of English and kids who do not read at grade level are required to take the same test as their native speaking and non-LD peers. Doing so makes zero sense. People get this.

You can also see the meme playing out in the comments of this Education Wonks post on this press release by Secy of Education Spellings reminding educators that special education kids fall under NCLB too. I thought that was pretty obvious, after all they didn't call the law No Child Except Special Ones Left Behind.

In any event, the new meme is based on the underlying assumption that English language learners (kids whose primary language is not English) and learning disabled kids (kids of average intelligence who are underperforming) can't be taught to read at grade level in a timely manner.

Remember, under NCLB 99% of all students (1% can take an alternate assessment) tested (a few percent can be absent on the test day) must meet state standards. And bear in mind that most state's have already jiggered the tests and standards so that the only kids who aren't meeting state standards are the bottom 25% of the curve which consists mostly of special education and LEP students. But the well has finally run dry, state's can't manipulate the standards and tests any further to pass the lowest 25%. The only way to get these kids to pass is going to improve education -- something the educrats are either loathe or unable to do.

Of course, this new meme is all bunk. Reading instruction can be dramatically improved and, by doing so, would improve the ability of almost all kids across the board. Not surprisingly, these improved teaching methods work almost as well with kids who don't know English very well:

The evidence cited here is consistent with the conclusion reached by Fitzgerald (1995) that effective beginning reading programs for English language learners are likely to be similar to those for English proficient children, with appropriate adaptations to their language proficiency. The programs with the strongest evidence of effectiveness in this review are all programs that have also been found to be effective with students in general: Success for All (Slavin & Madden, 2000, 2001), Direct Instruction (Adams & Engelmann, 1996); Reading Recovery (Pinnell et al, 1994) [Ed: giggle], and phonetic tutoring (e.g., Wasik & Slavin, 1993). In fact, several of the studies evaluating Success for All (e.g., Nunnery et al. 1997; Livingston & Flaherty, 1997; Ross et al., 1998) as well as DI (Gunn et al., 2000), also included non-ELL students, and in each case those students also gained from the interventions, to about the same degree. The beginning reading programs with the strongest evidence of effectiveness in this review made use of systematic phonics, such as Success for All, Direct Instruction, and Jolly Phonics, but systematic phonics has been identified as a component of effective beginning reading programs for English proficient students as well (see National Reading Panel, 2000; Gersten & Geva, 2003).

I think what we're seeing here is the end-game for the anti-NCLBers. It's an eight year set-up. When 2014 rolls around and the bottom of the curve fails to become proficient through the nonfeasance and malfeasance of our educators, they're going to point to this meme, say "I told you so," and hope no one pokes beneath the patina of this nonsensical argument

This is one of the great deficiencies of NCLB, a product of political comprise; it's too toothless to force the educrats to change what they're doing once they decided they weren't going to play along with the new rules. The law was based on the premise that schools would change for the better and only a small percentage would fail. The law never contemplated that almost none of them would change and we would see mass failures in 2014.

There is solidarity in failure.

It is difficult to get a monopolist to change, especially when that monopolist if feeding at the public teat. Perhaps it is time to finally break the monopoly.

Research -- Such That It Is

Following up on my last post on Kid Writing, let's now look its research base or lack thereof.

The first sign that things may be amiss is, well, the sign over to the right that is posted prominently on the front page of the Kid Writing site.

Here's what the text on the bottom reads:

If you have done any action or scientific research on the Efficacy of Kid Writing on children's literacy achievement, please contact Eileen Feldgus...

Contact me too because I am dying to see what is likely the only scrap of scientific research on the abomination known as Kid Writing.

Some of you (the ones who've actually followed the links to the Kid Writing site) are probably saying how can there be no scientific research on Kid Writing? Right on the front page, there's a research link that brings me to a page that boldly proclaims:

Current Research is Supportive of the Kid Writing Approach

And, below that we have two "research papers" that appear to be the research base for Kid Writing:

1. Joint Position Statement of the NAEYC and IRA
2. New Standards Primary Literacy Committee (PDF)

(NB: both the links on the Kid Writing site are broken, but nothing can escape google-fu.)

There's also a link to the oddly titled Action Research on Kid Writing by Mary Ellen McCloy (doc) on the front page.

Thing is -- none of this is even close to being research. And, more importantly, none of the "research" is on Kid Writing.

The first paper is a position paper by an advocacy group co-authored by the International Reading Association, a whole language shill group. Research is cited in the paper, but none of it is on Kid Writing -- it's all on other people's reading programs. This is the current scam du jour in education research and I'll cover it in more detail in my next post.

The second is a paper on early reading that, like the first, is neither research or specific to Kid Writing.

The third shows some promise -- it's actually about Kid Writing. However, it's not research -- it's based on a survey given to students, parents, and teachers about their feelings toward Kid Writing. The first paragraph tells you all you need to know about the seriousness of the scholarship:

The word kindergarten was created from two German words that mean children'’s garden. As a gardener tends to his garden with patience and care, kindergarten intends to treat young children gently, like flowers ready to unfold, so they can develop to their fullest when they are ready. If you hear the word kindergarten, your mind may conjure wonderful images of children frolicking and playing. There is often an element of surprise to hear that these five year olds follow an intensive curriculum.

I also spotted a fancy bar graph that shows:

An evaluation of the individual student's progress was determined using the Conventions of Writing Developmental Scale (Kid Writing, pp.169-179, appendix D). Using the students'’ portfolios and present journals, thirty-four students from two classes were rated on their writing in September as compared to June, as indicated in Figure 1.

Bonus points to the first person who can tell me what's wrong with this "research."

That's it. That is the extent of the research on the "research based" Kid Writing. Mind you, this program is over ten years old and was actively used in both authors' classrooms. Yet, no one ever thought to measure student learning using a conventional standardized measure of literacy.

Sadly, this is the norm for most educational programs in use today. This is the standard scholarship. This is the reason you get edu-jargon instead of serious research. This is why student performance is so low. And, this is why we have NCLB and Reading First.

Continue on to the Exciting conclusion to our story in Part III: Kid Writing: Education's Scam du Jour.

Tower of EduBabble

Newoldschoolteacher is guest blogging at EduWonk this week and compares and contrasts the course title at KIPP Summit 2006 with those given at the April conference of the American Educational Research Association (AERA). First a few course offerings from the Kipp Summit:

• Basics on Advising College-Bound Students
• Analyzing Test Scores
• Activities and Questioning with Bloom's Taxonomy
• Informal Assessment of Reading Difficulties
  

Now a few from AERA:

• English-Language Learning in a 3-D Virtual Environment: Native/Non-Native Speaker Dyads Co-Questing in Quest Atlantis
• Resisting Resistance: Using Eco-Justice and Eco-Racism to Awaken Mindfulness, Compassion, and Wisdom in Preservice Teachers
• The Formation of the Subjectivity of Mail-Order Brides in Taiwan and Their Educational Strategies Toward Their Children
• 'Ho No Mo': A Qualitative Investigation of Adolescent Female Language Reclamation and Rejection.

Welcome to the world of impenetrable edu-jargon. Edu-jargon is not your garden variety jargon. Normally, jargon is used by practioners as a shorthand to simplify complex ideas or things to avoid excessive verbiage. In contrast, edu-jargon is used to obfuscate simple ideas using overly-complex and high-falutin terminology to give the impression that what you're saying is more important than it really is and to disguise soft-headed thinking.

I have a simple rule when it comes to edu-jargon:

Anyone who uses edu-jargon has nothing serious to say.

It's quite a time saver. Once I come across edu-jargon in a paper, I stop reading; it's a waste of time to read any further.

The rule doesn't just apply to the field of education -- it applies to all scholarly fields. The more "edu-jargon" a profession uses the less it has to say and the less usful it is to society. Compare the writings in a serious field like computer engineering which continues to pump out useful ideas and inventions and generally has made life better for all of us to the writings in any of the liberal arts which has failed to generate anything useful to society in the past 50 years at least.

Here is an excellent takedown of some constructivist nonsense by Martin Kozloff:

"From this perspective, learning is a constructive building process of meaning-making that results in reflective abstractions, producing symbols within a medium." (Fosnot, 1996, p. 27). "Reflective abstraction is the driving force of learning." (Fosnot, 1996, p. 29).

First, notice the circularity in the line, "learning is a constructive building process of meaning-making that results in reflective abstractions, producing symbols within a medium," followed by "Reflective abstraction is the driving force of learning." One moment reflective abstraction is the result of learning (meaning construction). The next moment reflective abstraction is the driving force behind learning. Well, which is it?

Second, the excerpt contains examples of reification: learning is not said to be like a building process; it is a building process. Likewise, reflection is said to be a driving force. But what is reflection? Reflection means talking to yourself. What kind of force is that? How can talking to yourself drive learning? Do you talk first and then learn? Nonsense! [But in the field of education, this sort of piffle is commonly seen as wisdom.]

Third, notice how the author connects phrases into what comes off sounding profound, but means nothing. What, after all, is "meaning-making"? Is it something persons do alongside acting? "I'm writing a paper. Occasionally I stop to make meaning." And the phrase "producing symbols in a medium" is simply incomprehensible. What medium? A dish of agar?

As they say, read the whole thing.

Kid Writing

(Or the latest way educators have managed to screw-up teaching kids how to read.)

Kid Writing is the latest reading fad to come down the pike. Have you seen it?

I have. And, it's not a pretty sight. So, gather round kiddies while I tell you a tale so horrifying that balanced literacy appears almost sane by comparison.

Kid Writing is a "system" for teaching kids how to read by allowing them to write without first teaching them how to read or write. This is the educational equivalent of teaching kids how to swim by throwing them into the deep end of the pool with a sack of lead strapped to their backs. Guess which kids don't float to the top.

With little formal instruction, kindergarteners are asked to write a story. In the beginning of the school year, this is usually what the teacher gets back:

Helpfully, the teacher translated the kid writing for us. (The ghost is scaring people.) I'm not sure whether this falls into the "emerging," "pictorial," or "pre-communicative" portion of the Kid Writing Assessment Scale. It strikes me as a bit cruel to have six year olds try to write without any instruction.

During the course of the year, the students are haphazardly presented various sounds they can use in their writing. Supposedly, this is phonics instruction (all reading programs say they have phonics nowadays). In any event, by the end of the year, it is hoped that the student will be writing like this:


Translated into English: I wish my mom would have a baby boy because there are already three girls in my house. This is poetically referred to as the "semi-phonetic" stage, or, more accurately, the "guessing stage."

Before I forget, none of the misspellings and ungrammatical phrasing is supposed to be corrected by the teacher. The kids are supposed to develop naturally. It's all so sweet and innocent. Until somebody gets hurt. Like poor little Andrew who is going to have to unlearn the misspellings "wold," "becaus," and "allredy" after repeatedly misspelling them all year long. (First law of teaching: never teach something that has to be untaught.)

Bear in mind, these kids are purportedly being taught to read; the writing skills are merely a bonus.

In case you haven't figured it out yet, this is an incredibly dopey way to teach children how to read. Putting it more technically, Kid Writing teaches kids how to decode, i.e., read, by teaching them how to encode, i.e., write or spell. As I stated a mere two sentences a go, but it bears repeating, this is a dopey way to teach writing.

Phonics for reading is a very different system of phonics than phonics for writing/spelling. Reading/decoding goes from letters to sounds; writing/spelling goes from sounds to letters. Ideally, beginning reading passages should be pretty regular in terms of sound/symbol correspondences (with the notable exception of irregular sight words), making it easier for the kids to learn how to read. But, spelling/writing, on the other hand, is quite a bit less regular. This is especially so when the kids get to pick what they want to write about, as in Kid Writing. This is why reading/decoding has traditionally been taught first (it's easier) and the less regular writing/encoding taught afterwards.

Maybe an example is in order. Let's use the long "e" sound. In reading, the following letter, letters, or pattern would cause you to say the long-e sound when you see them:

ee as in seed
ea as in speak
ie as in brief
ei as in receive
y as in study
e-consonant-e as in Pete

And so forth. When a student sees these letters, he knows he has to say the long "e" sound. However, for writing/spelling the student is presented with the long "e" sounds and then has to figure out which of the above letter, letters, or patterns to use when writing. The question is: how do I spell the long-e sound? The answer is: lots of ways. And therein lies the problem. With the long-e sound, phonics for spelling and phonics for reading are worlds apart.*

Anyone hawking a beginning reading program should know the difference. The authors of Kid Writing apparently do not.

Reading is difficult enough as it is. Why make it more difficult by teaching kids how to read by teaching them via the wrong, and unnecessarily complicated, rules? When things go awry for some kids, how do you diagnose the problem? Furthermore, with all this writing "instruction" going on, there is precious little time being spent actually reading, which is to say practicing to read.

Which brings me to my reason for writing this post in the first place. Kids Writing claims to be "research based." I told you this story was going to be terrifying. I'll explain this little nifty bit of tom-foolery in Part II after you've had time to regain your senses.

* Borrowed heavily from A View From Askance: Sometimes, Phonics Sucks.

Our saga continues in Part II: Kid Writing Reseacrh or lack Thereof .

Quote of the Day

If you've told a child a thousand times and he still does not understand, then it is not the child who is the slow learner.

Attr:Walter Barbee

Socialized Education Vs. Capitalism

Naturally, Socialism loses again, as it always does.

From the Detroit Free Press we learn that "[c]ash-strapped school districts across Michigan are increasingly turning to private companies to manage food, custodial and bus services"

And why do you think the Michigan schools are cash strapped? Because they are inefficient. Everything they do, costs more than it should. This is what happens when you don't have the discipline of the market to keep expenses in line. Let's let the Detroit Free Press tell the story:

The move can save money -- and in some cases allow districts to put more resources into their main mission of educating kids.

Sounds like a good thing so far. Cutting expenses on non-education services, frees up more money for education services. Win-win.

But it can also bring controversy. Outsourcing recently spurred outrage in Garden City, where an effort is under way to recall five of the seven Garden City Board of Education members who voted last month to outsource custodial and food service jobs...

The Garden City Board of Education expects to save $2.2 million to $2.3 million over the next three years, said Superintendent Richard Witkowski.

That's not an insignificant sum. Gee I wonder where this outrage is coming?

"We had to look for some options that would reduce our costs considerably," Witkowski said last week. "I feel for those people who have lost their positions."

One of those people is Liz Jefferson, 31, of Garden City who has worked in the district for 12 years, about eight of them as a custodian. She is among the 33 employees -- 29 custodians and 4 food service workers -- to lose their jobs.

So, the outrage is coming from the workers who lost their sinecures jobs. I'm shocked.

Though district employees whose jobs were lost to outside companies may later find work with those companies, they often find the same jobs they once held come with lower salaries and fewer benefits.

In Garden City, for instance, custodial workers pulled salaries of $17 to $21.67 an hour. Grand Rapids Building Services, which will manage custodial work, has proposed salaries of $10 to $13 an hour.

I suppose I'd be upset too if I lost my cushy job that was paying nearly twice the going rate. I wonder if these custodial workers got the same obscene benefits package that the teachers get?

Let's skip down a bit and find out about the Avondale school district's flirtation with privatitation.

Outsourcing food service three years ago to Chartwells has worked so well in the Avondale School District in Auburn Hills that it since has outsourced the maintenance of its bus fleet, and in June the Board of Education also voted to outsource custodial work.

"As districts look for ways to provide service more efficiently, more and more districts will look to outsourcing as a viable alternative," said Timothy Loock, outgoing assistant superintendent for business services in Avondale.

Avondale went that route because it was losing money on its food service program, Loock said. The last year the district ran it, there was a $104,000 loss. After Chartwells' first year, the district made a $15,000 profit. At the end of the 2005-06 school year, the district made a $50,000 profit.

So when the school ran food services they lost $100k, now that these services have been outsourced, the school district pockets $50k. Left unsaid is whether the quality of the food improved as well. No doubt it did. The Philadelphia Zoo outsourced their food services recently to private companies, such as Nathan's and Starbucks. Not unsurprisingly, the customer service and the quality of the food (it's now edible) improved dramatically. But I digress.

How did they do it?

"Primarily through lower labor costs," Loock said. "The food service staff as employed by Chartwells are paid more closely to the market for that type of labor."

White said school districts are more frequently considering outsourcing because of the tough economic times they face.

Of course, they face these "tough economic times" because they can't effectively control their spending and the public is starting to balk at paying skyrocketing taxes. They must have thought that taxdollars were like the loaves and the fishes.

"School boards are stuck between a rock and a hard spot on this. They're looking for ways they can save money so they can continue to keep the same number of people in the classroom," White said.

He should know. White also is on the school board for the Clinton Educational Service Agency, the intermediate school district that serves Clinton County in mid-Michigan. Six years ago, the board sold its bus fleet and hired a private company to provide bus service. The move wasn't made to save money, but to allow the district to focus on other areas.

"It's one of the most painful things I've ever done," White said.

Welcome to the real world, jerky. These are the same kind of tough decisions managers in the private sector have to make all the time. We'd all like to have our bloated 1950's era support staff back and be able to pay employees far above the prevailing wage and abide by all the archaic union rules, but companies (at least the ones without a government monopoly) that continue to do such things quickly become defunct companies with unemployed employees.

Union officials aren't happy.

Are they ever? Having already lost most of the private sector with their stuck in the 1930's mentality, they are now beginning to lose their last remaining toehold -- the public sector. Listen to the old-timey rhetoric:

"Our members are being displaced, losing their jobs, and in some case losing pensions," said Rosemary Carey, a communications consultant for the Michigan Education Association. The state's largest teacher's union has 160,000 members.

Does anyone still care about defined pension plans? You know, one of those pensions subject to employer (or in this case union) mismanagement. It's 2006, Rosemary, people have 401k's. You don't lose those when you change jobs.

They say communities are affected too, because the jobs being outsourced tend to be held by longtime district residents. Meanwhile, they say, the private companies tend to hire from outside the communities and the lower salaries tend to result in heavy turnover.

"You've got the possibility of strangers being hired and being gone within a few months because they're not making enough money," Carey said.

Longtime district residents that are milking the district dry through over-priced jobs. Remember the point made up-front -- every dollar that is being overpaid for these services is a dollar that isn't making it into the classroom.

High turnover is never a good think for any company, because it is costly. Companies balance pay scales to avoid excessive turnover. This is one reason why the outsourced company in the story is paying 2 -3 times the minimum wage for low-skilled custodial jobs.

Now all we have to do is find a way to outsource the teachers and, more importantly, the school administration, and maybe we'll get back to providing a decent education again.

As I pointed out in my previous post, the City of Philadelphia could slash their school budget in half by outsourcing their schools to the archdiocese of Philadelphia.

The $13,000 Question

While researching yesterday's post, I discovered (to my amazement) that the Philadelphia School System wastes spends about $13,000 per student for education. This is about 13% higher than the state average expenditures for school districts in Pennsylvania. So much for the underfunded schools meme.

As a point of comparison, the tuition for attending a Catholic high school in Philadelphia will be only $4,380 for the 2006-2007 school year. Not exactly a princely sum even if you include the inevitable subsidies the Archdiocese gives to their schools. And, I'm sure the hallways in the catholic schools don't reek of urine as they do in many of the public schools.

Let's see where the Philadelphia School System spends all this money (from School Matters for 2004-2005).

Total Expenditures were $12,761. This is the number I used; but, when most education commentators write about school expenses they use Operating Expenditures, which for the Philadelphia School System were $8,551. Using Operating Expenditures, instead of Total Expenditures, makes it appear that the cost of public education is cheaper than it really is; in Philadelphia School System's case it undervalues the cost to taxpayers by a third. Quite a neat trick.

Total Expenditures includes such thing as capital expenditures (buildings and equipment), interest payments, and payments to outside schools (usually for education special education students with real handicaps--as opposed to the phony school-caused "specific learning disabilities"). These are, of course, real expenses that need to be paid. If you don't believe me, try not paying your mortgage for a few months and see upset your bank becomes.

Next we have Instructional Expenditures at $4,779 which is only about 56% of Operating Expenses (37% of Total Expenditures). The rest of the Operating Expenses go to transporting and feeding the kids, utility expenses, and maintaining the bloated bureaucracy.

For every dollar out of the taxpayer's pocket, only 37 cents makes it to instruction.

This is fiscal mismanagement on a grand scale. You hear a lot about the military's $600 toilet seats from the press, not as much about the $13,000 educations in which less than a third of the students (who didn't drop out) actually learned. At least the toilet seat accomplished its intended function.

The Art of Spin

spin -- "a usually pejorative term signifying a heavily biased portrayal in one's own favor of an event or situation. While traditional public relations may also rely on creative presentation of the facts, 'spin' often, though not always, implies disingenuous, deceptive and/or highly manipulative tactics."

That's from Wikipedia. And, here's an example from the Philadelphia Inquirer of the all-too-typical spin we get when education journalists report test score results, especially when those test score results are from urban areas.

The headline blares: City schools show gains for 5th year.

The subhead screams: PSSA test scores marked progress, especially at the high school level, but there were still trouble spots.This is highly misleading, to say the least.

Philadelphia schools may have "show[ed] gains" for five consecutive years which "marked progress"; however, what we don't learn until paragraph two is that "fewer than half of the district's students in the tested grades [are] performing at advanced or proficient levels."

And, we don't learn until paragraph ten that "fewer than half," in Inquirer lingo, means:

"only a third of the district's 11th graders perform at advanced or proficient levels in reading, and nearly 27 percent in math."

This is called "burying the lead" in journalist-speak.

Yes, Philadelphia schools have made some small gains in recent years (whether those gains are real or a matter of fiddling with the tests is unknown); but, is that really what the story is about?

Isn't the real story that despite five years of "gains" only 33% (in reading) and 27% (in math) of Philadelphia students passed the test. These pass rates are abysmal by any standard. They are especially abysmal when you consider how easy the Pennsylvania test really is.

The headline "City schools show gains for 5th year" doesn't do the facts justice.

How about:

Despite small gains, less than a third of City School students in 11th grade (at least the ones who haven't dropped out yet) pass PSSA test.

Or better yet:

Despite spending over $13k per student, less than a third of 11th grade City School students pass PSSA test, up slightly from last year.

A little longish perhaps, but at least it's accurate.

Light Blogging

Sorry for the light blogging, kids. I've been on vacation in sunny Nevis for the past week. Should be back on my game shortly.

Not So Stinky Research Part II

(This is part II of this post. Part I can be found here.)

In the last post, we discussed how we'd need to increase student performance by at least a full standard deviation in order to comply with NCLB. Bear in mind that this is considered to be a large effect size improvement, which are very rare in education research.

So are there any current educational interventions that are capable of improving performance by such a large amount?

Let's limit the discussion to research that has at least a moderate effect size (> 0.5 SD), statistically significant results, and at least three well-designed studies.

Believe it or not there are a few interventions that meet this criteria.

Fortunately, the American Institutes for Research has already done much of the hard work for us by evaluating the research for many popular elementary school interventions. AIR published the results in its November 2005 paper CSRQ Center Report on Elementary School Comprehensive School Reform Models. I went through the report and found three interventions that met my criteria.


Accelerated Schools

From the CSRQ Center Report:

The CSRQ Center reviewed 37 quantitative studies for effects of Accelerated Schools on student achievement. Three studies met CSRQ Center standards for rigor of research design. The Center considers the findings of these three studies conclusive, which means that the Center has confidence in the results of these studies. About one third of the results reported in these studies demonstrated a positive impact of Accelerated Schools on student achievement, and the average effect size for these significant results is +0.76.

The AS studies can be found in Appendix A of the report.


Success For All

From the CSRQ Center Report:

The CSRQ Center reviewed 115 quantitative studies for effects of SFA on student achievement. Thirty-three of these studies met CSRQ Center standards for rigor of research design. Upon review, the Center considers the findings of 31 of these studies conclusive, meaning the Center has confidence in the results reported. The findings of the other 2 studies are considered suggestive, which means the Center has limited confidence in them. Overall, the 33 studies report a mix of results showing positive effects and no effect of SFA; of the 91 separate achievement test findings across the 33 studies, just over half (52%) demonstrate a statistically significant positive impact. The average effect size of these positive effects is +0.63.

The SFA studies can be found in Appendix U of the report.


Direct Instruction

From the CSRQ Center Report:

The CSRQ Center reviewed 56 quantitative studies for effects of DI on student achievement. Twelve studies met CSRQ Center standards for rigor of research design. The Center considers the findings of 10 of these studies conclusive, which means that the Center has confidence in the results reported. The findings of two studies are considered suggestive, which means the Center has limited confidence in the results. The findings in the conclusive and suggestive studies showed mixed results: some studies demonstrated a positive impact of DI on student achievement and other studies showed no significant effects. About 58% of the findings reported in the studies that met standards demonstrated positive effects; the average effect size of those significant findings was +0.69.

The DI studies can be found in Appendix K of the report.

I also know that Gary Adams performed a meta-analysis on the DI research and examined 34 rigorous studies. Here's what he has written about the results.

On pages 48 and 51, the meta-analysis shows that 17 studies lasted less than a year and 17 lasted over a year. The effect size can be calculated per comparison and per study but all of the results show large effect sizes: .95 for studies less than a year and .78 for studies more than a year... On page 44, the age of the publications was analyzed (1972–1980: 6 studies, 1981–1990: 22 studies, 1991–1996: 6 studies) and all of the effect sizes were large (.73, .87, 1.00, respectively).

Fifteen of the studies were conducted by researchers who have been somehow connected with Direct Instruction. In contrast, the majority of the studies (18 studies) were conducted by non-DI-connected researchers. The effect size for studies by DI-connected researchers was .99—a large effect size. The effect size for studies by non-DI-connected researchers was .76—also a large effect size.

So, there are at least three educational interventions that are capable of achieving moderate to large effect size improvements on student performance, at least in the elementary grades.

Of course, all three of these interventions completely overhaul how schools are run, changing almost every aspect of the school. Maybe such radical change is necessary.

Does anyone else know of any other valid educational research with similar moderate to large effect sizes? The comments are open.

Research that Isn't So Stinky

Is there any decent Ed research asks John from AFT? The simple answer is yes but not much of it.

John also helpfully sets out a good definition for effect sizes for social science research:

[I]f the difference between sample averages is no more than two-tenths of a standard deviation, the difference should be regarded as small; a difference of half a standard deviation should be regarded as moderate; and a difference of eight-tenths of a standard deviation or larger should be regarded as a large difference.

These are the generally accepted standards for evaluating social science research.

Your typical Title I school is performing at the 20th percentile. Only 20% of its students perform above the 50th percentile. This low performance places the typical Title I school about 0.84 standard deviation below the median school (50th percentile). So in order to improve the performance of this school so that it performs like a mainstream school, it would need to raise its performance by 0.84 standard deviation. This is a large effect size.

So now let's pretend that there were some easily implementable educational intervention that could raise the performance of all schools by 0.84 SDs. Even with such a large improvement, about 20% of students will perform below the present 50th percentile . In Title I schools, half the students would still not make the cut! In order to raise performance so that only about 5% of students perform below the 50th precentile, we'd need some combination of lowering standards and raising performance by another 0.80 SD. Most states have already dutifully complied by lowering standards quite a bit under NCLB.

Realistically, schools are going to have to raise performance by at least a full standard deviation (and count on some help from the states) in order to comply with NCLB. And even then, the Title I schools will still be failing since they will have about 20% of students not making the cut. This is the most unfair aspect of NCLB, but until we're ready to admit that student IQ affects student performance such policy discussions will be off limits.

We'll take a look at some of the existing Ed research in part II of this post to see if we know of any educational interventions that are even capable of increasing student performance in the neighborhood that we need to comply with NCLB.

Education Research Stinks

I almost fell over as I read this editorial in the Atlanta Journal-Constitution. The editorial discusses Ed research and how lousy it is:

The public risks whiplash keeping up with the latest twists and turns in the education research. One day, block scheduling is a fresh approach that all high schools ought to adopt. The next, it's derided as stale and moldy.

...

These conflicts reflect a seldom acknowledged truth in education: There's a lot of uncertainty about what works and what doesn't.

In the meantime, schools invest millions of dollars in innovations and reforms that sound impressive, but don't amount to much more than a new coat of paint on a rickety old house. And when those reforms don't deliver the promised results, schools cling to them anyway because they've spent too much money and time to walk away.

Wow. That's pretty good. A journalist actually understanding the quality of research. How often do you see newspapers acknowledge that research isn't sometimes all that good. Usually, they just accept the researcher's conclusion uncritically, especially if it coincides with their ideological agenda, and play-up whatever data is handed to them.

The emphasis on student achievement in the federal No Child Left Behind Act and the requirement to use data to substantiate outcomes are prompting researchers to devise more reliable ways to capture effectiveness.

"The biggest revolution caused by No Child Left Behind is the revolution in education research," says Georgia State University's Gary Henry, a scholar in educational policy and evaluation. "We are getting better at figuring out what works. But what we are seeing is almost nothing that has a very large effect."

Even when the research shows a gain, it's a very small gain produced under the best of circumstances. That's because most reforms only tug at the edges and don't address central flaws in public education: A teacher's track record of improving student performance is hidden from public view, and that performance is not used as a factor in teacher salaries.

Now go back and re-read the part I emphasized; it's probably the most important thing you'll read this week. If you're a journalist or edu-blogger, understanding this little truism will result in you making fewer silly statements, such as touting your favorite edu-theory which inevitably lacks any indicia of success.

Here are a few simple rules to live by:

1. If the results of an experiment are not statistically significant (p ≤ 0.05), then the results are not reliable. If the researcher fails to indicate the p value it is safe to assume the results are not statistically significant. However, a p value of less than 0.05 does not transform a statistical association into a causal association. It just means that there is the requisite level of confidence that the observed statistical difference between the comparison groups are not due to chance. Whether the statistical association is in fact a causal association will depend on additional factors, such as a well designed experiment.

2. If the effect size is less than 0.25 standard deviation, the results are generally not considered to be educationally significant. This just means that we're don't want to waste good money pursuing educational interventions with small effect sizes because invariably the effect size of the intervention in actual use will be less than the results observed in the lab. To put this in perspective, an intervention with a ¼ SD effect size will only raise the performance of a typical Title I school from the 20th percentile to the 28th percentile. It takes an effect size of about one standard deviation to raise their performance to the average school--the 50th percentile. In education research, effect sizes larger than ¼ standard deviation are rare.

3. Poorly designed experiments are neither science nor reliable. The big shortcomings here are lack of comparison groups, lack of randomization or pre-testing of the experimental groups, not accounting for attrition efects, control group participation in the intervention, the presence of selection or confirmation bias, and other methodological flaws. To put it bluntly, most education researchers do not know how to conduct valid experiments.

4. Results cannot be extrapolated beyond the parameters of the experiment. This one is a biggie. Experiments are only valid for the conditions under which the experiment is conducted. So, if the experiment properly concludes that reducing class sizes to 13 students may benefit kindergarten age students, it is improper to extrapolate those results and claim that similar class size reduction might benefit all K-8 students or that a class size reduction to 20 students would have a similiar effect. Such conclusions are not supported by the evidence.

There is very little education research (and subsequent advocacy) that meets all these criteria. Typically, when any serious meta-analysis of education research is conducted, about 90% of it has to be discarded as lacking. Journalists and edu-bloggers should be very leery relying on shaking education research. You don't want to make the make the mistake the new think Tank Review Project made this week by relying on shaky research to make over-hyped points criticisizingthe Reason Foundation's universal preschool report and wind-up the laughing stock of the edu-sphere.

Follow-up Post: Ed Research that Isn't So Stinky