July 9, 2009

More Evidence that Good Instruction Can make a Big Difference

Last spring I reported on the Gering School District (Nebraska) which had taken their Reading First grant back in 2004 and, with the help of NIFDI, implemented the full-immersion Direct Instruction for Reading and Language Arts back in 2004 district wide.

Let's see what the results look like four years later now that that first cohort has reached 3rd grade. We can also take a look at the performance of the 4th - 6th grades who participated in the program for some of their school years. Also, we can take a look at the 7th and 8th grades that received a remedial version of the reading program only.

As you can see from my fancy Excel 2007 graph below, Gering students are now performing better than the Nebraska state average.



The blue bars on the left represents the mean performance of all students in Nebraska on the Nebraska state NCLB test (a test with an artificially high pass rate to be sure, but we are concerned here with relative performance). Nebraska is about 70% white and 25% Hispanic so the average performance of all students comprises mostly white students.

The red bars represent the performance of all Gering students which is higher than the Nebraska average in all grades. This is significant because the demographics of Gering are below the Nebraska average. (Adults are less educated and there is more poverty.)

Now let's take a look at the subgroup data to see who is making the progress.

The green and purple bars represent the performance of Hispanic and white students respectively. Both groups are performing better than the state average for all students in all grades but grade 5 where Hispanics perform slightly below. Gering Hispanics are actually performing better than Gering white students in grade 7 which is something you rarely see on a district wide basis and/or without selective admissions.

The light blue bars (on the right) represent the performance of students receiving free and reduced price lunches -- a proxy for poverty. Poor students in Gering perform better than the Nebraska average for all students (not just poor ones) with the except of grades 5 and 7. This is something you don't see everyday. I've run quite a few analyses like this for different states and I have yet to find an entire school district whose low-SES students performed above the state average for all students.

Apparently the effects of poverty and being a non-Asian minority can be mitigated through good instruction alone.

I challenge anyone to produce similar effects for their favored non-instructional intervention (on a District-wide basis without selective admissions) that produce effects similar to those found in Gering in grades 3-8.

I'll drill down deeper into the data in a future post.

18 comments:

Robin said...

The peer reviewed studies, national reports, and practice guides in reading, language arts and math are consistent in calling for "direct, explicit and systematic instruction" for "at risk" kids.

Despite this convergence of evidence inquiry learning and whole language techniques remain the norm in too many schools and districts.

Gering truly shows the way to close the achievement gap so that everyone benefits. It also shows what can be done without outside or at home tutoring moderating the detrimental effects of poor or no instruction at school.

Kathy said...

Does anyone on this site know what the Harlem Children's Zone uses for reading instruction?

http://www.hcz.org/our-results

I can't find anything on the Harlem site explaining reading programs, or any specific reading data. They do say they are getting kids to grade level in reading and credit the holistic approach.

Does anyone have know of specific test data from this project?

thank you

Kathy

KDeRosa said...

I have heard much about the HCZ, but have seen little hard data.

This article makes it seem like HCZ has a long way to go to improve student achievement.

HCZ boats on their site that the elementary school is doing great, but it seems to me that they are working the children as hard as they are doing in a KIPP school who don't need to provide the social services to obtain good results. Also, it's a charter school so there's a selection bias problem just like KIPP has.

Of course, people are already trying to replicate it despite the lack of success because non-instructional interventions are very fashionable.

palisadesk said...

Re Harlem Children's Zone, I've read that they get children to grade level in math and have raised achievement (but not to grade level) in reading and other ELA.


Does anyone have know of specific test data from this project?

See here:

2009 Harvard study

Kathy said...

Thanks will check out the links. I am currently arguing with a teacher on another blog. He keeps telling me we need a holistic approach to improve reading. I need some facts. He also mentioned the Bolder project. I found a ton of blogs on this site to debunk their claims. I have sent him a ton of data, the recent post on Gering but changing minds on this topic is really hard.

My district is currently in a large debate about school funding for the low performing schools. The CEO of the district is claiming kids are not learning because resources are not high enough in some schools. She wants to have the ability to move teachers to any school as she sees fit.

Whether you agree with her ideas or not, it is still the same old same old stuff- more money, better teachers needed etc.

Kathy

Kathy said...

American Educator devotes its entire summer 2009 issue to this very topic, the holistic approach is needed to raise test scores.

The shortened URL:

http://tiny.cc/V5s07

If that does not work here is the full URL. Sorry I don't know how to embed the URL.

http://www.aft.org/pubs-reports/american_educator/issues/summer2009/index.htm

Robin said...

Does "holistic" mean that you take the terminology from the National Reading Panel report and apply it to the practices we used to describe as Whole Language?

KDeRosa said...

The Fryers paper while not exactly science does provide some insight as to what is going on in the HCZ.

The HCZ students spend between 50% and 100% more time in school depending upon how far behind they are. Attending school for twice as much time with resonable instruction is going to have positive results. That's a large part of how KIPP gets it done.

Fryers believes and the data suggests that what is happening in-school is teh driving force behind the results.

Fryers also points out how the out-of-school programs can account for only a modest effect on performance.

Holistic my ass. This is good old fashioned time on task.

KDeRosa said...

This article explains how to code a hyperlink in html.

Christopher Paslay said...

Let me start by agreeing with the premise of this post: Good instruction CAN make a big difference. Some instructional strategies work better than others, and teachers, parents, schools, etc. should fight to implement the ones that work best, and abandon those that don’t. I say this as a teacher, someone who’s taught in a public school for 12 years. I am a much better teacher than I was 10 years ago, and this is due in big part to my methods of instruction, which I’ve honed over the past decade. So I like the premise of this post. However, after perusing this blog briefly (I scanned several of the articles on the main page and a few archived posts that Kathy linked me to), it appears that the idea of holistic education isn’t taken very seriously. Movements like Broader Bolder and The Harlem Children’s Zone have been scoffed at here. DeRosa commented that “people are already trying to replicate it [HCZ] despite the lack of success because non-instructional interventions are very fashionable.” I’ve read other posts here that marginalize a student’s socioeconomic conditions and parental involvement. I’ve read still others (Kathy provided me with the links), that claim reducing class size doesn’t make a difference in itself (The folly of class size reduction), that home environment isn’t a factor in education (Achievement gaps can be eliminated by schools alone), and that teachers are to blame for failing schools (The root cause of educational failure—bad teaching). Here are two observations I’d like to make from my limited exposure to this blog and its ideas: One—Why is it that when someone mentions holistic education, they are seen as making excuses? No one’s denying that teachers (and good instruction) are very, very important. But if a teacher never has the chance to implement this instruction (because a kid is chronically absent; or can’t see the board; or doesn’t have medication for ADHD; or doesn’t understand how to problem solve non-violently; or can’t control his anger; or is hungry; or has no place to live so he can’t do homework; or his parents make him watch his siblings so his misses school and assignments; etc.), the teacher can’t even GET to the instruction in the first place. Anyone who’s taught in an inner-city public school understands that BEFORE INSTRUCTION COMES CLASSROOM MANAGEMNT. You can’t walk until you crawl. Management stems from solid teaching, but a big portion (at least 50%) comes from the child’s background, parents, SES, all of these things. Asking for a more holistic approach is not making excuses. Anyone who’s actually taught in a classroom knows this. When teachers fight for holistic education, we are not saying good teaching and instruction aren’t important. We are saying we need the help of others to make schools work better: parents; education policy writers; teacher colleges; communities; social services; etc. I’m not sure if Mr. DeRosa is a school teacher, but my guess is that he’s not. If he were he’d know that class size is a BIG factor in learning, despite what some social scientist purported in some study. 20 students are easier to teach than 30, hands down. You have more time to teach, you can get to know them better, you can manage the class better. Education is the only field where everyone and their mother is an expert. It doesn’t matter that those who criticize from the sidelines have ZERO formal college training in education, or any experience teaching in a classroom. Sure, many people back their opinions with studies, but statistics can be made to show that an elephant can hang from a cliff with his tail tied to a daisy. And ironically enough, the very people closest to the classroom and education itself (the teachers themselves) have been marginalized, viewed as ignorant, incompetent, lazy, and the main reason why kids don’t learn. There needn’t be an “us verses them” mentality. Teachers don’t need to be bashed and disrespected. We need to find a middle ground, and combine good instruction with holistic resources.

Anonymous said...

Please explain the effectiveness of DoDEA schools.

http://yglesias.thinkprogress.org/archives/2009/07/americas-best-schools.php

KDeRosa said...

You take an IQ test ABVAB, etc)before they let you in the military. This gets rid of much of the bottom end of the curve. In other words, there is a selection bias problem.

Anonymous said...

"You take an IQ test ... in the military."

In other words IQ is heritable, so screening out stupid parents screens out stupid kids. By the same logic, stupid parents have low socioeconomic status, but economic interventions will not change their stupid kids. The main solution appears to be segregating classes by ability.

Given your emphasis on good direct instruction, I was hoping that military school benefitted from uniformity of curriculum, avoidance of fads, and better discipline.

Thanks,

Steve Heston

Independent George said...

You know what else really helps people read? Paragraph breaks.

Mikethelawstudent said...

I find that character education is highly effective especially with kids who are currently falling through the cracks.

Mike

CHARACTER EDUCATION KANSAS

Florida resident said...

Form Wikipedia:
Gering, Nebraska:
As of the census[1] of 2000, there were 7,751 people, 3,173 households, and 2,170 families residing in the city. The population density was 2,067.8 people per square mile (798.0/km²). There were 3,332 housing units at an average density of 888.9/sq mi (343.1/km²). The racial makeup of the city was 91.48% White, 0.13% African American, 1.14% Native American, 0.25% Asian, 0.04% Pacific Islander, 5.56% from other races, and 1.41% from two or more races. Hispanic or Latino of any race were 13.40% of the population.
So about 1,000 of hispanic population (as of year 2000). Let us estimate the # of kids as 30% of Hispanic population: about 300 kids.
So, if all of them are in that statistics, that is great.

silly girl said...

There are differences is what a given individual can learn. Understanding that students do not have uniform capacity to learn helps you to set appropriate learning goals. Most children start in kindergarten and by about 3rd grade, there is reliable standardized data as to which students have strong academic ability and which may have disabilities. Students with low performance are often referred to Special Ed. for evaluation. Special Ed tests the student's IQ and then compares his achievement to his ability. If his achievement tests (ITBS or Stanford 10 etc.) match what his IQ would predict, he is considered to be learning commensurate with his ability. If they don't match and his IQ predicts that he should do better, then he qualifies for more intervention because he is underperforming given his abilities.

Likewise many schools have programs for talented students. What is really hard for teachers is that they often do not have access to documentation as to what abilities their students may have. The teacher is going nuts trying to meet the needs of children ranging in IQ from 80 to 130 or more. It is hard for the teacher not to feel defeated unless she has accepted that she isn't responsible for the academic ability the student was born with. There is no magic bullet or instructional method. All instructional methods are more effective for higher ability students.

FYI, I read that the HCZ school brought kids up to an average of the 39%tile on nationally normed tests, which are more rigorous than the state competency exams. While that may represent a significant improvement in particular student population, it is not miraculous. It probably just puts the students over some artificial achievement threshold set by the state education agency.

The greatest failure of school administration from the school level all the way up to the DOE is their dishonest attitude that all kids can achieve an arbitrarily set goal. This is not true. No method on earth will get 90% of kids to pass Algebra. Not even if they get to take it three times. Likewise about 10% of kids could learn it on their own just by reading it from the book and asking a few questions from time to time. Life is not fair. Trying to make all students the same is not fair to them, and is not possible. We all need to grow up and face it. From the content of this blog, it seems that the author and commenters sincerely care about serving students' needs. I spent years in public education working with students to meet their goals, not my goals. Often students are more practical and reasonable than those trying to tell them what their goals should be.

Anonymous said...

The Joanne Jacobs website recently had a thread on military schools and military kids in public schools. The overall comments agreed that the military families (including single parents) stressed self-control, discipline, good manners, hard work and the importance of education; to the point that teachers in schools near military facilities could identify the military kids very quickly. It's what used to be called the Puritan/Protestant work ethic and the behaviors were explicitly taught in both public and private schools for well over 100 years, whether so named or not.

The above is not to deny that the military screens out the bottom segment. The selection bias is even stronger for military parents with school-age kids, since they are disproportionately senior petty officers, chiefs or officers. Unsatisfactory junior enlisted personnel are returned to civilian status before they have school-age kids.