Hoffman and the Rule of Holes

Tom Hoffman is having a grand ol' time digging himself deeper into a rhetorical hole.

Let's start with the argument Hoffman apparently considers to be the knock-out blow.

DeRosa's critique of the assignment is based on how he imagines the Dred Scott decision ought to be taught in a US History class. Had he asked before writing his missive, or bothered to read the History and Social Studies section of the Curriculum Guide, he would have known that his entire frame for critiquing the assignment was incorrect, because this was not an assignment for a US History class (taken in 11th grade at SLA), but an African-American History class.

I'm not sure why Tom thinks I didn't know that the project was for an African-American history class considering this statement from my initial post.

The project comes from page 10 of the Family Handbook and pertains to African-American History.

I've taken the liberty of highlighting the relevant portion for Tom's benefit. Apparently, my "entire frame of critiquing" wasn't correct after all.

In this context, what is important is the decision's impact on African-Americans and the abolitionist movement, not the balance of power in the great game between the North and the South in which the African-Americans are seen as mere pawns. Perhaps in 11th grade US History, the pre-war balance of power dynamic will be emphasized.

I agree. That's why I gave the following as an example of analysis showing deep understanding for a high school student.

[T]he Dred Scott decision is important because it upset the political compromise at the federal level (the Missouri Compromise and the Kansas-Nebraska Act) which served to limit the spread of slavery.

Again I've emphasized the relevant portion for Tom's benefit. The balance of power issue remains an important issue for African-Americans history since it affected the growth of slavery. I'm thinking the growth of slavery might have "impacted" African-Americans.

I provided yet another reason for why the balance of power issue was important to the slavery issue:

As long as the Senate was gridlocked, the North would not be able to pass a constitutional amendment banning slavery in all the states.

Again, I've scaffolded the passage for Tom's benefit. I'm thinking the balance of power issue was a little more important than merely a "great game between the North and the South in which the African-Americans are seen as mere pawns."

Tom continues:

I would note that Chris Lehmann told me that he left a comment on Ken's blog explaining this oversight on Ken's part, but for whatever reason, that comment has not been published as of this date.

Unlike Tom, I don't moderate comments and I only delete spam comments. If Chris' comment didn't post it's either because Chris did something wrong or blogger ate the comment. Most likely it was the latter.

I'm thinking at this point Chris is glad the comment didn't make it through.

The knock-out punch missed its mark. Let's see if Tom's remaining argument lands.

Beyond Ken's unhappiness of the framing of the decision and the assignment, his criticism of the student work itself is not based on any knowledge of the kind of work 14 year olds typically do. As a piece of writing, the letter in question would stand up admirably against the anchor papers used in any 9th grade writing assessment in the country, if not the world. DeRosa never questions the accuracy of the student's historical information.

Actually, I did question the efficacy of petitioning Southern Democrats for redress. That seemed to be on the wrong side of a few historical facts. However. the primary deficiency in the student project was that the student didn't give us much to work with, hence my characterization of the project as "superficial understanding." This contradicted the claims made by SLA in the family handbook, and I quote:

Teachers in each course ask the question – “What are the enduring understandings students should have when they leave this class?” Teachers then create projects that can only be completed by showing both the skills and knowledge that are deemed to be critical to master the subject and demonstrate that deep level of understanding.

...

At SLA, there may be multiple assessments – including quizzes and tests – along the way, but the primary assessment of student learning is through their projects.

This was supposed to be an exemplary project, yet it contradicted SLA's assertion that it showed deep understanding of the subject matter and mastery of skills. I argued that it showed superficial understanding because it "fail[ed] to cover any of the important issues presented by the decision, the historical context of the case, and why the case is historically important."

Tom's point with respect to what a 14 year old should be expected to know is relevant. Tom cites an AP prep book for U.S. History that provides far more detail on the Dred Scott decision (p. 132)and the relevant history leading up to that decision that my analysis. I also cited a middle school history text which gave about the same level of analysis that I provided. What is clear is that the student example project provided far less analysis than either the middle school analysis, my analysis, or the AP prep book analysis.

That's two misses.

My advice to Tom: stop digging.

Still Waiting on Broader, Bolder

I'm still waiting on someone fom Broader, Bolder to offer some evidence supporting the effectiveness of their call to expand public education to cover a myriad of social services.

This week both Diane Ravitch and Randi Weingarten offer tepid defenses over at The Education Gadfly.

Let's take Ravitch's defense first:

I care as much about academic achievement as Checker or anyone else in the world, but I don't see any contradiction between caring about academic achievement and caring about children's health and well-being.

The issue isn't about who cares about children's health and well-being. The issue is whether public schools, who are by and large failing at their primary task of education, should take on the additional responsibilities of caring about children's health and well-being. You could care very much about the health and well-being of children and NOT think it's a good idea to hand these services over to our public schools.

The argument seems to be that since children attend school every day (cough, cough) that social services could be easily provided at school. Then why not hand over these responsibilities to the post office. After all, they make house calls six days a week regardless of the rain, snow, heat, or gloom of night. They could give the kids a quick vision screen and drop off any drug prescriptions.

Will it help or harm children's academic achievement--most especially children who are living in poverty--if they have access to good pre-K programs?

The extant evidence suggests that pre-k programs will have no or a negligible effect on academic performance. The yoke is on you to show that there will not only be an educationally significant effect on academic achievement but also that the benefit will persist when the public schools do the provisioning.

Will it help or harm children's academic achievement--most especially the neediest children--if they have access to good medical care, with dental treatment, vision screening, and the like? Will it help or harm children's academic achievement--the children whose lives are blighted by the burdens of poverty--to have access to high-quality after-school programs?

The evidence here is even more scant. I know these community service schools exist, so where is the evidence that they are raising academic achievement. Show me the money.

So, I explain my dissent briefly: One, what we are doing now--the standards & assessments & accountability strategy alone--bears little or no resemblance to genuine academic excellence.

But this doesn't mean that the Broader, Bolder way will be any improvement.

And two, children who come to school hungry and ill cannot learn no matter how often they are tested.

Last I checked, schools offer free and reduced price lunches to practically the entire left side of the socio-economic curve. If these kids are still hungry what makes you think that expanding these current programs will solve the problem, to the extent that there even is one. Again, where is the data?

And three, a good education must include attention not only to academics but to children's character, civic development, physical education, and physical health.

Schools are attempting to do most of this stuff already. Where is the evidence that it is working? Where is the evidence that providing more will bring about improvement?

All we seem to have is rhetoric. Show me the data.

Let's move on to Weingarten.

I'm stating the obvious when I say that No Child Left Behind's testing regime has left little time for these kinds of in-class activities.

By "the obvious," Weingarten appears to mean "with little evidentiary support."

What evidence we go have suggests that only about 16% of schools have reduced art and music time at all. And those that did reduce time in these areas only reduced time by less than an hour a week. Perhaps these schools were neglecting math and reading pre-NCLB. Do you know? No, you don't.

But teachers alone can't get kids all the way to proficiency, when disadvantaged children typically enter school already three years and 30 million words behind.

I hate throwing this word around, but this statement is a lie. The big lie, so to speak. Schools can substantially reduce all of the achievement gap that exists between low-SES and middle-class schools. We've known this for over thirty years now. The experiment has been replicated many times, most recently in Baltimore. Up to the fifth grade level as well.

[M]y message was twofold: first, let's put in place a federal education program that, unlike NCLB, provides space and opportunity for children to be taught a rich, well-rounded curriculum, with standards and accountability that support rather than undermine that curriculum; and second, let's--at the same time--try to address the outside factors like nutrition and health care that affect a child's ability to reach her full educational potential. And yes, I said that we also should try to help parents so they can better support their children's learning.

In Project Follow Through, comprehensive medical, dental, nutritional, and social services were provided to all of the thousands of students taking part in the experiment so these factors would not confound the results. Most of the interventions failed to achieve any student gains at all despite the provisioning of all these services. Many interventions performed below the performance of the control groups. Ooops.

The causal link has not been established. Repeating the rhetoric ad nauseum is not a substitute for data.

Ravitch knows better. Weingarten probably does as well, but there is self-interest at play.

I'll ask one more time. Show us the data. We're waiting.

The Myth of Fun and Interesting

Most educators have brought the myth that academic learning does not require discipline--that the best learning is easy and fun. They do not realize that it is fluent performance that is fun. The process of learning, of changing performance, is most often stressful and painful.

--Lindsey, O., (1992). Why aren't effective teaching tools widely adopted? Journal of Applied Behavioral Analysis, 22-26.

So begins chapter four of Vicki Snider's book Myths and Misconceptions about Teaching, What really happens in the classroom on the Myth of Fun and Interesting. This chapter is relevant to the ongoing debate over the educational value of the SLA student's project on the Dred Scott decision and problem-based education in general.

Snider describes the dangers of this myth well.

The myth of fun an interesting is an extension of the myth of process. When the process [of education] is emphasized, the entertainment value of a lesson and the students' level of engagement becomes the measure of the successful lesson. Teachers derive their reward and sense of satisfaction from creating fun and interesting lessons rather than from attaining specified learning outcomes.

...

There is some truth to the myth that learning should be fun. I can understand why teachers feel compelled to make lessons entertaining. I, too, like to design activities that engage students, create excitement, stimulate discussion, and make students laugh. I also know that these reinforcing moments do not necessarily guarantee that students have mastered the content. The exhilaration of my great lesson is more that offset by the letdown when I assess retention and application of skills and concepts.

Snider goes on to describe the potential harms that are done by the myth:

There are four harmful effects that result from overreliance on fun and interesting activities. First, fun activities lead to a lot of wasted instructional time. Second, activity-based instruction can make it difficult for learners to focus on what it is they are supposed to learn. Knowing what to pay attention to is called selective attention in the psychological literature and it is often a problem for young or naive learners or those with learning disabilities. Third, rather than increase motivation to learn, activities with a high entertainment value but a low content value may actually decrease the probability that a child will become a lifelong learner. Fourth, without effort and practice, individuals cannot master any intellectual or creative endeavor.

Snider makes an often overlooked, yet important point about lifetime learners:

People seldom decide to pursue a new intellectual area out of the blue; they become interested because they find themselves in a situation that reactivates some general knowledge that a teacher thought important years ago. If they have enough general knowledge, they can find out more through experience, by going to the library or looking on the Internet, or taking a class. The more specific knowledge they acquire, the more they are able to learn. Sometimes this positive reciprocal learning cycle leads to a depth of knowledge that allows a person to think critically and analytically. In other words, interest is the reward of learning, not the motivation for learning.

Lastly, Snider on the importance of developing fluency.

Fluency goes beyond accuracy. It is accuracy plus speed. Our traditional reliance on accuracy only, in the form of percentages, does not distinguish between students who have learned a skill, but still perform it with hesitation, and those that are fluent. failure to make these distinctions underlies many educational failures (Binder, Haughton, & Bateman, 2002). Students "progress by building one non-fluent skill on top of another until the whole skill set becomes too difficult to be enjoyable and students may respond to this stressful learning situation by becoming inattentive, misbehaving, or failing to complete homework and other assignments. All of these consequences are predictable. If the teacher responds by making learning more fun and interesting or by reducing accountability, the problem is solved in the short term, but the real issue remains unaddressed and the long-term result is low academic achievement.

Snider raises some important issues with PBL. Making school work fun and interesting is all well and good, but ultimately you have to look at what the student has actually learned to gauge the effectiveness of the teaching. The arguments I'm reading so far favoring PBL and SLA rely on some form of goal-post shifting with respect to what the student is expected to have learned. This isn't exactly a strong argument in favor or PBL and SLA if you know what I mean.

Teaching Content -- the Dred Scott Decision

We all can agree that history instruction should not be a parade of facts; however, facts will need to be learned, at least temporarily, to give the student something to think about. Here's one way to make that process less painful and boring for the student.

To facilitate learning, most of the instruction should relate to big ideas. Big ideas are the ideas that summarize content, are applicable again and again, can generate predictions and hypothesis, can be used to help structure the details of content, and help in remembering and reconstructing through inference content details. Most of the instruction should revolve around the big ideas. (Crawford 2004)

Students should be able to fluently articulate the knowledge they learn so that the knowledge can be used in higher-order activities, such as essays, comparisons, application, synthesis, evaluation, and the like. (Crawford 2004)

Here's an example of a big idea from a middle school history textbook that precedes the teaching of the Dred Scott decision.

The sectional disputes about slavery that began during the Constitutional Convention ended with two compromises. One compromise was that the Constitution said that slaves were to be counted for both representation in Congress and as property, but at three-fifths value. The other compromise kept Congress from stopping the slave trade until 1808, but allowed Congress Control over other aspects of trade. The northern and southern states had equal numbers of representatives in the Constitutional Convention and chose to reach a compromise. When each side is evenly matched, there is a balance of power. An example of balance of power is when two teams have the same number of players and the players have the same skill level. If one team gets a chance to put one extra-good player into the game, then the balance of power is upset and that team will probably win. That team will dominate.

University of Oregon (1995), Understanding U.S. History: Volume 1—Through the Civil War, Chapter 13: The Road to the Civil War pp. 318- 328.

The big idea of the unit is balance of power which gives the student something to think about as the material is learned and by doing so helps him integrate the material being learned.

By using the big idea of Balance of Power, the congressional compromises of the period, the Missouri compromise and the compromise of 1850, can be learned with some meaning, rather than as a mere parade of facts. For, example, here is how the Missouri compromise might be taught:

In 1819, the balance of power in the Senate was equal with 11 free states and 11 slave states. Adding new slave or free states to the United states could be thought of as adding extra players to a game—if one side got too many new players, the balance of power would shift to that side. But adding new players was exactly what was about to happen to the U.S. because the country was growing. It looked as if the balance of power might change.

In 1819, the people of the Missouri Territory asked to join the United States as a state. A territory was a region that was a part of the United States but not yet a state. Most of the white people living in Missouri wanted slavery. Northern Senators were against adding Missouri as a state because then there would be more slave states than free states with representation in the Senate. In 1819, the northern senators would not agree to let the South have a majority in the Senate. They argued for months against adding the Missouri Territory as a slave state.

After several months of debate, the people of Maine, which had been part of the state of Massachusetts, asked to join the United States as a free state. This opened the way for a compromise to be worked out. Henry Clay, a senator from Kentucky, proposed the compromise plan, called the Missouri Compromise. Henry Clay was one of the War Hawks who boasted before the War of 1812 that the Kentucky militia could conquer Canada. He also worked out a compromise for the Nullification Crisis.

Clay’s idea was to admit both states into the United States. One state, Maine, was free state which did not allow slavery. The other state, Missouri, did allow slavery. That maintained the balance of power.

The Missouri Compromise was a plan that drew a line across the Louisiana Purchase territory, south of which slavery would be allowed. Slavery would be outlawed in the rest of the Louisiana Purchase lands north of the imaginary line. The Missouri Compromise Line was drawn at the southern boundary of Missouri. Areas south of the Missouri Compromise line could become slaveholding states, but the land north of the line was to remain free from slavery. In 1820, the United States had not acquired the land west of the Louisiana Purchase. At the time it was passed, the Missouri Compromise established rules for all the land in the United States.

(Understanding U.S. History)

Now, see if you can answer the following question to see f you understand balance of power and how it affects the admission of territories as states.

Q: Arkansas was the next state to ask to join the United States. Arkansas was below the line of the Missouri Compromise, so it would be added as a slave state. What factors might influence Arkansas' admission as a state?

The same big idea can get you through the admission of Florida and Iowa, the election of President Polk, the concept of Manifest Destiny, the admission of California, New Mexico, Texas and the Oregon Territory, the Compromise of 1850, the South's attempt at seceding, the concept of popular sovereignty, and the passage of the Fugitive Slave Laws which culminate in the end of the balance of power era:

Although the Compromise of 1850 solved the problem of the southern states seceding from the United States, the solution was only temporary. Enforcing the Fugitive Slave Law and allowing popular sovereignty to decide about slavery only made the issues about slavery more difficult for Congress to solve. The ability of the Congress, the political parties, and the country as a whole to make any more compromises on the issue of slavery was coming to an end.

...

Until 1850, a balance of power between slave states and free states had existed. That balance of power resulted in compromises being made between the North and the South for many years. In the 1850s, three factors ended the ability of Congress to make compromises about slavery: (1) the idea of popular sovereignty, which began with the Compromise of 1850; (2) the Kansas-Nebraska Act of 1854, which led to violence and 200 deaths in Kansas; and (3) the Supreme Court’s Dred Scott decision in 1857.

(Understanding U.S. History)

The Dred Scott decision wasn't important because of the specific holding with respect to the slave Dred Scott (though I'm sure it was important to him), but was important becuase it eliminated Congress' ability to maintain the delicate balance of power that had existed and threatened to allow the unhindered growth of slavery:

The Dred Scott decision. The third factor that completely ended the ability of Congress to compromise on the issue of slavery was an 1857 Supreme Court ruling on slavery. The Court ruled that slaves had no right to sue in federal court because African Americans could not become citizens of the United States. The Supreme Court ruled that someone who owned slaves could keep slaves in any federal territory. The decision basically made slavery legal in any territory. The Supreme Court also ruled that there was nothing in the Constitution that allowed the federal Congress to outlaw slavery in any territory, and that only states could decide for themselves about slavery. That decision was the end of Congress’ ability to make compromises about slavery because Congress couldn’t pass any laws that limited slavery.

The Chief Justice who wrote the court’s decision was a southern Democrat who had always supported slavery. He felt that the decision in the Dred Scott case would end the debate over the expansion of slavery. He thought that once the court had decided to protect the property rights of slaveholders, the country would accept this decision. He was mistaken.

(Understanding U.S. History)

The challenge for techers is to teach all this material so that the students know it well enough to explain it without the referring to the textbook. This is the first prerequisite for knowledge to be used flexibly in higher order activities such as essays, comparisons, application, synthesis, evaluation, and the like.

There is no golden road to learning. Deep understanding in a domain (such as pre-civil war history) is not going to happen in a fact vacuum. You need something to think about first, before you can think about it deeply, i.e., understand the abstract functional relationships inherent in the material. If the student's "inquiry" hasn't led the student to learning the facts, at least temporarily, deep undersatnding isn't going to follow. This is what happened to the SLA student.

Tom Hoffman Attempts a Defense

Tom Hoffman of Tuttle SVC tries to defend SLA'a Dred Scott project that I recently critiqued. Hoffman attempts the old smoke and mirrors defense by attempting to portray the student's response as "deep understanding" of other stuff. You can be the judge of whether he's succeeded or not. I think he unwittingly proves my point as I indicate in the comments.

Hoffman's argument is another good window into the mind of the progressive educator. Deep understanding has been redefined to mean the amount of understanding you can achieve without knowing much content. To the rest of us, this is superficial understanding. History without historical facts or understanding of those facts. See the way that works?

I hope Hoffman will never try to "get my back" like he did here.

Chris Lehmann Responds

Chris Lehmann, Principal of Science Leadership Academy (SLA), responded to my criticism of one of SLA's projects in a recent post. Here is Chris's response with my comments.

One, let's start with the premise that you and I have fundamentally different views on educational philosophy, so I'd argue that you're predisposed to
disapprove of our school.

I'm not so sure about that. I assumed that our goals are similar: to maximize student learning. To the extent that I favor certain pedagogies over others, it is only because those pedagogies have evidence of superior results when we look at what the student has actually learned. But I am not beholden to any particular pedagogy for the sake of any particular educational philosophy.

Let's also state that, as you point out, we are completely transparent about our philosophy at SLA. Between my blog, the Family Night Book, the web site, etc, you can get a sense of what we believe pretty easily. Also, we encourage all
prospective students and parents to come spend a day at the school... not a special "everyone-visits" day, but any day. We want families to understand our educational philosophy because we want kids to make an informed decision about where they want to go to high school. So we're not trying to trick anyone into coming to SLA. Given that transparency, we've had a great interest in our school. We're also pleased with the academic results we've seen. So far, we are currently on track to have over a 90% four-year graduation rate from SLA, and by qualitative and quantitative metrics (attendance, course passing rates, PSAT scores, as well as early research by two PhD students), kids are doing very well.

I haven't seen any reported PSSA results for SLA yet. Will this year's results be reported?

So to the specific points you raise:

A) You show an interesting unwillingness to accept the scope of the piece or that the scope of the piece has merit. To do this work, the students had to examine primary source materials, learn about abolitionist societies, learn about the Dred Scott case through multiple lenses -- including the political one you favor. And then they had to create a piece of writing that a) showed historical understanding and b) made a decision to argue a point of view of one of the groups active at that time. All of that maps to the standards of the state for history.

Admittedly, the scope of the project is unclear based on what is provided in the handbook. I assumed, and I think fairly so, that the scope of a project for a high school history class would be a demonstration of historical understanding. Now, if the student expectations are diminished from this level, I'm curious to know in what way they are diminished and what is the expected pathway to this higher level since the curricular design is claimed to be backwards?

The handbook states that the project "can only be completed by showing both the skills and knowledge that are deemed to be critical to master the subject and demonstrate that deep level of understanding." What specific skills and specific knowledge were the students expected to have mastered? What was the level of specific understanding that was expected?

I do think that a high school student can be expected to perform the kind of analysis I provided if they are taught how to do it and taught the underlying content. In fact, I think middle-school student can be taught to this level. I'll provide an example in a future post.

My concern, and I think it is a valid concern, is that student learning is taking a backseat to the pedagogical fidelity. My opinion is that the student's understanding is banal and typical of what you see when the student lacks domain knowledge. (My third grade son comes home with stuff like this from school all the time even though he is capable of better.) My understanding is the theory is that it is better for the student to generate this knowledge on his own, rather than be told it. Fair enough. But the student's work product indicates that the required knowledge has not been generated, as the theory predicts, with the result being a superficial analysis. I just don't see how the student understanding can in anyway be considered to be deep at this point.

Moreover, during the course of the unit, students were engaged in a variety of instructional techniques. The assumptions you make about the kind of teaching that happens at SLA are wrong.

My assumptions were based on what was provided in the handbook and by what learning the student demonstrated. If I'm wrong, I'd like to understand how.

Moreover, even in my presentation that you cite, I talk about how traditional forms of assessment -- quizzes and tests -- have their place in our classrooms, they just are now lower on our hierarchy of assessment. Tests and quizzes are great ways to see if kids have learned how to handle skills and content in a narrowly defined context. What the student projects do is see if they can transfer those skills and content to a larger context.

The handbook states that "the primary assessment of student learning is through their projects" so I think it's fair to use the project, by itself, as the criterion of whether the learning meets the educational goals. I assumed that some teaching took place regarding the historic era in question and some background knowledge was taught, but I don't see how very much of this transfer took place.

B) I'd argue that [your] analysis of what was missing suggests your own bias toward what [you feel] is important to learn about that material. That's fine, the single greatest limiting factor in school is time. If you want to cover a great deal of material -- and even in "progressive" schools, history courses have a lot to cover, you are never going to get to every lens. And, by the way, while this assessment may not have asked kids to deal with the larger political lens, other work did. And yes, we do often question our balance with depth and breadth -- that's the question all good progressive schools should ask themselves, just as all good traditional schools should ask themselves about their balance of skills and content, information transferal and knowledge acquisition.

Again, the handbook indicates the goal was deep understanding and mastery of skills. I don't see either having taken place here at a ninth grade level. Which lens were expected if not the historic/political one considering this was a history class? I don't see depth nor breadth here, but then again I am not sure of the expectation which appears to be much less than what I consider to be high school level.

C) The level of analysis you suggest is warranted is collegiate -- or at least 11th or 12th grade -- in its complexity when this was a 9th grade piece of work. (We only had ninth graders when we published the book.) I think it holds up as a sophisticated, smart piece of work that shows an emergent sense of what it means to have a historical sense of the world.

Another way of saying this is that the student has not yet acquired an historic sense.

I'm going to post a middle school level text related to the same issue so we can judge the level of analysis we might expect. I get to hopefully by the weekend.

D) You and I have a fundamental disagreement over the value of skills v. content in student understanding. You believe that until the student has enough facts at their disposal, there is little benefit to asking them to "think like a historian." We believe that those skills develop with practice and that students' ability to develop the ability to apply an historical lens on the world requires frequent, guided, scaffolded practice. That's fine. We'll have to agree to disagree.

Again, the proof of the pudding is in the eating. I don't see how a series of superficial analyses is going to lead to expertise, nor I have I ever seen a deep analysis of an issue by one who was not in possession of a deep understanding of the underlying domain knowledge. Cog Sci tells us you don't acquire the latter until you've acquired the former. Nor is the student getting practice gaining expertise in this domain. The student isn't learning the underlying content. How is he going to learn how to analyze what he doesn't know?

Inquiry-driven learning doesn't mean you just set the kids off on Google. Guided inquiry means giving kids skills to access resources and make decisions on their own.

Has that been demonstrated here? I think just the opposite has been demonstrated. The resources weren't accessed and the right decisions were not made. Hallmarks of a novice.

I don't think you're going to read this comment and suddenly think, "Aha! I get it, SLA is wonderful!" But I also would hope that we could move beyond strawman arguments -- I'll promise not to argue that KIPP creates a bunch of automatons who merely can regurgitate what they have been told if you promise not to argue that SLA is some unstructured school where kids just are indoctrinated toward an ideological bias or kids merely learn some surface knowledge by surfing the web.

I do not assume that SLA is unstructured. I assume there is plenty of structure because no structure would be a disaster. I think that the problem is that the students' inquiry isn't leading to content knowledge and that the students' analysis suffers for it.

Regurgitation is not the goal, though inflexible knowledge is the typical starting point. However, not being able to even regurgitate is telling as well.

As always, I offer you the opportunity to come visit SLA. I don't think you'll like what you see there -- again, my goal is not to convince you that we're the one right school, but rather so you could see that there is more than one approach to schooling. I have no doubt that a thoughtful application of DI schoolwide can create an effective school where kids learn well. Can you entertain the same notion that an inquiry-driven approach that has been executed thoughtfully can do the same?

There is always the possibility, but I've yet to see the results with a population like that of SLA. I'm always looking to be convinced. Perhaps, there is a better example project for inclusion in next year's handbook.

Juxtaposition

Here's a nice juxtaposition from Teacher's Magazine's Blogboard on playing the teacher card*.

First we have Ryan's statement from I Thought A Think:

I think it can be universally accepted that teaching requires a certain skill set to transmit information to the students and get them to retain it. There's a science to teaching, and there's an art to teaching. I don't think it's out of line to suggest that if you haven't practiced the craft then you don't really have an authentic understanding of what's involved.

And then we have a comment from one of those teachers, Haley:

I agree with Ryan’s thinking in that it does take one to teach to fully understand the detailed process. Most people do not realize all of the details that are put into teaching and how every one of those details play a critical role in the learning process. For instance, in order to cater to all students’ needs, it is essential to incorporate different teaching styles to meet the varying learning styles students possess. It is imperative that teachers create lessons that are developed to meet the needs and interest levels of all students. I bet a lot of people did not even realize that there are different teaching and learning styles.

Moving right along.

*Playing the teacher card means claiming that teachers have some special knowledge with respect to education policy and should be deferred to.

Science Leadership Academy

I made it through about ten minutes of Chris Lehmann's keynote speech at this year's NECC entitled Progressive Pedagogy and the 21st Century. Nothing you haven't seen before, but here's an embedded link for you diehards.



Lehmann is the principal of the Science Leadership Academy (SLA). SLA is a Philadelphia-based magnet high school which claims to be, not unsurprisingly, an "inquiry-driven, project-based 21st Century school with a 1:1 laptop program." SLA also makes available its 2007 Family Night Book (PDF) for prospective students and their families. Caveat Emptor.

The Family Handbook and Lehman's speech paint a rosy picture of progressive education and technology. You're supposed to get the impression that progressive education is simply super and adding technology takes it one step higher -- to the super-duper level.

Normally, at this point I'd criticize the inquiry-driven, project-based brand of progressive-education practiced at SLA, but SLA has already done the heavy-lifting for me with its Family Handbook.

The Family handbook helpfully gives us actual examples of its student projects making it clear that Lehmann's lofty rhetoric doesn't survive its contact with reality.

According to the Family Handbook:

Although student projects, in general, generate more student interest they also often have involved multiple steps and drafts and can, quite often, require a great deal more effort than just studying for a test. One of the first things students realized when they first come to SLA is that projects here are different. The key to this difference lies in a concept that the faculty employs to create curriculum called “Backward Design.” Teachers in each course ask the question – “What are the enduring understandings students should have when they leave this class?” Teachers then create projects that can only be completed by showing both the skills and knowledge that are deemed to be critical to master the subject and demonstrate that deep level of understanding.

Furthermore, "the primary assessment of student learning is through their projects" (p. 9).

So let's take a look at one of these projects from the Family Handbook. I'm going to assume that this is a project that SLA is proud of and that the student example demonstrates mastery of both the skills and knowledge that are critical to master the subject and demonstrate a deep level of understanding.

The project comes from page 10 of the Family Handbook and pertains to African-American History. The project is:

To write a letter as the president of an abolitionist society to the group’s members upon hearing the decision in the Dred Scott case. Your letter should explain the decision, your perspective as a leader of the abolitionist cause, and what you want your membership to do about the decision. The content of the letter is restricted to information available in 1857.

It appears that the student is expected to consult the Supreme Court's 1857 Scott v Sanford decision (54 glorious pages of mid 19th century prose), and perhaps other secondary sources, and construct a sensible interpretation of the decision in light of pre-Civil war historical events like an expert historian might do. That's a tall order for a novice student lacking the deep-structured well of domain knowledge possessed by the historian. Lacking this domain knowledge, we would expect a superficial analysis of the decision focusing on surface features of the decision rather than the more important abstract, functional features needed for an expert analysis of the problem.

A decent high-school level analysis of Dred Scott might go something like this:

Briefly, the Dred Scott decision is important because it upset the political compromise at the federal level (the Missouri Compromise and the Kansas-Nebraska Act) which served to limit the spread of slavery. Each new state would now be able to determine for itself whether slavery would be permitted within its borders under the doctrine of popular sovereignty. At this time in U.S. history, the population of the western territories was growing rapidly and the U.S. was growing rapidly as territories petitioned for statehood. This rapid growth threatened to upset the existing balance between slave states and free states which maintained a gridlocked Senate. As long as the Senate was gridlocked, the North would not be able to pass a constitutional amendment banning slavery in all the states. The North was content with this arrangement as long as slavery was contained in the existing slave states and, thus, a political compromise had been reached which permitted the admission of free states and slave states in equal proportion. The Dred Scott decision threw out this political compromise in favor of popular sovereignty. Thus, the Dred Scott decision threatened to upset the existing political balance, would permit the growth of slavery, and would make it impossible to amend the Constitution to make slavery illegal in the U.S.

(This is my synthesis of about a half dozen relevant secondary sources)

A superficial analysis of Dred Scott, in contrast, would focus on the plight of the slave, Dred Scott, who was seeking his freedom in federal court. A superficial analysis would focus on Dred Scott and not the larger issue--the spread of slavery. With this in mind, let's take a look at the example student project that SLA chose to include in its handbook:

Greetings Members:

These past few months have been busy, with major changes being implemented in government decisions. In the Supreme Court, a case has recently been heard that affects our mission to help reverse the plight of African-Americans in this country. This case goes against our values as Pennsylvanians, Philadelphians, and abolitionists, and our beliefs in human rights.

In the case of Dred Scott v Sanford, the courts unfortunately favored Mrs. Emerson, Mr Scott’s “owner”. While under the rule of his master, Mr. Scott was taken to live in Illinois state and Wisconsin territory, respectively, both of which are free. Mr. Scott pleaded that since he had lived in those areas, he was entitled to his freedom, because it was illegal for him to be living there and be enslaved at the same time. The court eventually decided that Mr. Scott was not a citizen, being of the African persuasion, and therefore had no right to sue in a federal court. They went further as well, stating that our U.S. Congress has no right to declare certain states free and 0thers slave-owning.

This decision promotes the message that African Americans have no rights. In the majority write-up, Chief Justice Roger Taney writes: “They had for more than a century before been regarded as beings of an inferior order; … so far inferior that they had no rights which the white man was bound to respect; and that the negro might justly and lawfully be reduced to slavery for his benefit.” That statement is pure fiction. Negroes have had rights in our commonwealth for decades. One of our African American members, Mr. Forten, is a sail maker, and makes his living through his own business. He enjoys the right to own property and to make a profit. Mr. Forten would not benefit from being forced into backbreaking labor daily with insufficient food. No one would. The idea that slavery is beneficial to the slave is preposterous. No human would think it in his favor to be sold like a table and to work from sunrise to sunset, and then not get to reap a single seed he has sown.

Family Handbook, p. 10

This is a superficial analysis. The analysis focuses on the specific facts of the case, rather than the larger issue--the spread of slavery--presented by the decision. The student seems unaware that the (federal) citizenship issue was only important because Dred Scott was trying to gain redress in federal court based on diversity jurisdiction which required the plaintiff Scott to be a citizen of a different state than the defendant Sanford. The student fixates on the inflammatory language in the decision regarding plaintiff's citizenship which was merely of procedural importance. The analysis fails to cover any of the important issues presented by the decision, the historical context of the case, and why the case is historically important.

We shouldn't be surprised by the student's analysis. The student is a novice and is being asked to create a new interpretive narrative of facts stemming from the Dred Scott decision. The student, unlike the historian, is not an expert in pre-civil war history and how our republican form of government operates. The student lacks the background knowledge of the expert. Lacking this background knowledge, the student is going to have a difficult time separating the important facts from the unimportant facts, accessing what little relevant knowledge he possesses from his memory with the speed and accuracy needed to perform the analysis. The result is that the novice student is likely to focus on the surface features of the problem, rather than the abstract functional features of the problem that are needed to analyzes the problem. This is exactly what happened.

More importantly, the student's "inquiry" failed to teach the student what was supposed to be learned about the Missouri compromise, the Nebraska-Kansas act, bleeding Kansas, and all the other issues presented by slavery before the civil war. This wasn't supposed to be some silly busy-work project. This "project" was supposed to be "the primary assessment of student learning" showing "both the skills and knowledge that are deemed to be critical to master the subject and demonstrate that deep level of understanding." I think the project shows just the opposite. The student failed to acquire a deep understanding of the issues which was the entire object of the project.

The project continues with the student's recommendation for what the abolitionist society should do in light of the decision.

We must protest this unethical decision. Our society wishes its members to petition to lawmakers on the fairness of this ruling. The Chair of the Committee on the Judiciary in the Senate is one James Bayard Jr. of Delaware, a Democrat. His counterpart in the House of Representatives is George S. Houston, a Democrat hailing from Alabama. We beseech you to write to these men and protest this injustice! Help our brother in bonds who should rightfully be free. Every letter sent raises our chances to help Mr. Scott.

We thank you for your dutiful support of our efforts to help improve the condition of African Americans in our country and this Commonwealth. Many voices blend together to make a choir! With all of your voices singing, we know we make beautiful music. Please continue in your support so that one day we might all be free.

Petitioning the legislature's judiciary committee isn't going to accomplish much, especially for a 7-2 Supreme Court decision. And nevermind that the pre-civil war Democrats were mostly for the expansion of slavery, especially the southern Democrats, like the Senate member the student recommends petitioning. Again, the project shows that the student did not gain a deep understanding of the issues. A better answer would have been suggesting that its members persuade people to flood the new territories to affect the vote on whether slavery would be permitted under popular sovereignty which was the result of the decision.

Treating the student like a junior historian isn't going to make the student think like an historian. The student lacks the requisiste domain knowledge and the understanding of the abstract functional relationships inherent therein needed to perform the kind of analysis that an historian would be able to perform. This project isn't going to aid the student in acquiring that domain knowledge or give the student any practice thinking about the abstract relationahips which might have been gained by conducting a real analysis of the historical facts and the decision. Rather, the student wasted time conducting a superficial analysis which ignored all the relevant historical facts. The student missed a valuable opportunity to think about the important functional relationships presented by the exercise because the student was asked to create new content which is outsude his ability as a novice. This student didn't even gain practice in analyzing like an expert because he didn't analyze like an expert would have. It's a charade.

The fact that SLA included this project in the Family Handbook as an exemplary project is telling. This example supposedly shows the superior understanding supposedly conferred by the project-based inquiry pedagogy favored by SLA. Instead, what the example shows is that the "pedagogical" tail is wagging the "student learning" dog. That the student hasn't demonstrated a deep understanding isn't important. What is important is that SLA gets to teach according to its ideological bias.

It is somewhat embarrassing that SLA doesn't understand the difference between deep understanding and superficial understanding. Anyone can google "dred scott" and read the hits. But, it takes someone with some domain knowledge to separate the unimportant facts from the important ones and to synthesise the important facts into a good analysis. Sadly, this SLA student is no closer to that goal than he was before the project.

Update: Chris Lehmann stops by in the comments to provide a lengthy rebuttal of my arguments re SLA and this particular project. Chris actually advances the discussion and doesn't just give us talking points, whcih is a good thing. I'm tied up for the rest of the day and won't be able to respond until later. Play nice until then.

Update II: Here is my response.

Not Even Wrong

The New York Times reports on soon-to-be AFT presidente-for-life, Randi Weingarten's "new" vision for American education:

Can you imagine a federal law that promoted community schools — schools that serve the neediest children by bringing together under one roof all the services and activities they and their families need?

...

Imagine schools that are open all day and offer after-school and evening recreational activities, child care and preschool, tutoring and homework assistance.

...

Schools that include dental, medical and counseling clinics.

This is a very slick way of selling the proposed expansion of the public sector (and the expansion of the unionized jobs controlled by Weingarten that expansion entails).

Telemarketers do the same thing: Let me tell you about this great new service x we're offering ... Doesn't that sound great? let me verify your address ... can I go ahead and sign you up?

Telemarketer Weingarten wants you to focus on, i.e., imagine, all these great services provided for "the neediest children" instead of focusing on whether these services will boost student achievement and whether schools are the best vehicle for providing these non-educational services.

Weingarten just wants us to assume she is right. But, she's not right. She's not even wrong.

The problem is that she has no real basis for concluding that having schools take over the provisioning of these services will lead to increased student achievement.

She doesn't have evidence that providing more of these services will lead to increased student achievement.

That's because the research doesn't exist. There are no real world examples of schools providing these services and student achievement having risen. We have nothing, but for Weingarten's pretty words.

Call me a cynic, but I don't have much faith that public sector schools, especially those with unionized labor, are capable of providing these non-educational services any better than they currently provide educational services.

As long as a student (1) learns things easily from technically unsound teacher presentations, (2) readily retains what has been learned with technically unsound practice afterwards, and (3) has a strong familial support structure in place to keep the student on track and make sure the student's learning is progressing, then the student stands a good chance of becoming educated in public schools. This is because this student can be educated in virtually any school. We don't need a public sector monopoly to provide educational services to these students; any fool is up to the task.

Whether we need a public sector monopoly for providing educational services for the remaining kids is debatable. There certainly is no evidence that the public sector provides these services any better than the private sector. And, they've had an awfully long time trying to get their game together. Too long, maybe. It appears that they've given up. But for a few reformers, the present consensus, reflected in Weingarten's speech, is that students need to change for there to be improvement in student achievement. So Weingarten's idea is for schools to take over all the social services that they believe affect student achievement.

This is what we're supposed to be imagining -- allowing a dysfunctional monopoly to take over responsibilities outside of its core function. That makes little sense.

I find it easier to imagine just the opposite -- taking away the monopoly powers for providing educational services we've given to the public sector.

I'll print out this blog post and eat if, if anyone can provide a good reason why we shouldn't. The comments are open.

Update: Commenter Oldtimer informs us that the Cincinnati Public School stem has a school that provides all the social services Weingarten is advocating and has yet to show any improvement.

An Amusing read

Go check out this lengthy thread over at educationwonkette.

Academic guest posts a somewhat controversial, albeit politically-correct, view on race and education, gets challenged on it by commenters, and runs away without responding to the challenge.

It must be nice to have this kind of insulation from criticism in certain academic fields, but it doesn't help the reputation when you fail or are unable to defend your position in the real world.

New DI Program: Differentiated Reading

A new DI fluency building program, Differentiated Reading (PDF), was released today and is intended to be used with low performers who are reading at low fluency rates (less than 45 words a minute). The program is directed to children with whom simple fluency procedures have not proven to be effective. The program will be published commercially before the end of the year by SRA.

The article I linked to entitled Improving the Reading Rate of
Low Performers and is loaded with insights regarding the teaching of lower performers.

The frustration of slow readers

Trying to improve the reading rate of very low performers can be a frustrating experience for both learner and teacher. The learner typically knows that the goal is to read faster, without making a flurry of mistakes, and the learner tries, but the added effort most frequently leads to word guessing, word skipping, word stuttering, and to greatly increased physical signs of high energy, such as clenching their fists, taking deep breaths, and even sweating. The student knows how to try hard physically and thatʼs what he does. But it doesnʼt work for reading faster.

What Teachers Observe

The teacher may also notice that the studentʼs performance is not predictable from one day to the next. The typical pattern is for the learner to perform “better” on one day, and be very happy with his performance and the praise the teacher issues, but almost certainly, he reverts to his old habits on the next day and does poorly.

The teacher often concludes from observations that whatever it is that causes improvement is there one day and gone the next. The bottom-line conclusion is that something is wrong with the learnerʼs learning mechanisms.

This conclusion is thoughtful and comes after the teacher has tried
different approaches for improving rate-accuracy.

The Basic Rule

Teachers need an approach that permits students to show them through their reading behavior how much and how fast they can improve. The basic rule is that if students are properly motivated to read faster and donʼt, the reason is they canʼt.

Students Respond Logically

We donʼt want the task of learning to read a little faster to become an effort like Sisyphus trying to roll the rock out of the pit but never succeeding. This step is built around the fact that students respond to data. They are realistic. They know when they are failing and when they are progressing. If they receive good evidence they are doing well, and meeting reasonable expectations, they will keep trying and persist when they regress or when the material they read becomes a little more difficult.

If they canʼt see evidence of progress, they will tend to draw a conclusion we donʼt want them to draw—“I am a failure; I canʼt do it.”

The article describes the new program in detail and can be used by classroom teachers now, so there's no need to wait to use it.

Mystery Ingredient X

Tyler Cowen asks:

I don't think we have a recipe that says, "Take a child of two non-college educated parents, add primary education ingredient X, bake, and out comes a college-capable high school graduate." The mystery ingredient X has yet to be discovered.

Predictably, lots of commenters showed up and said the missing ingredient X is IQ. What they mean is that students continue to need a high IQ to succeed in today's education environment because, by and large, the prevalent educational techniques fail to simplify the complex concepts students need to learn to be "college-capable high school students." The result is that education remains largely inaccessible to those having low IQs.

I'd say mystery ingredient X will most likely involve something that simplifies the complex concepts that need to be learned, but currently aren't being learned by low-IQ students. (This also extends to low-SES students which largely overlap the low-IQ group.) DI provides a large portion of this mystery ingedient X at the elementary school level. In fact, this is speifically the goal of DI's design:

The net result of meeting these criteria is that DI materials appear to be easy. Possibly the most difficult concept for observers of DI programs to understand is that although the programs seem simple, they meet multiple design criteria that make them simple. The superficial impression of a program done right is that the authors may not understand some of the complexities of the content. The complexities, however, have been addressed and have been reduced to non-complexities that do not sacrifice the integrity of what is taught earlier or what is to be taught later. If the criteria are met, the prediction is that the student will generalize to a specified set of examples including those that have not been taught.

Rubric for Identifying Authentic DI Programs, pp. 18-19.

Today's Quotes

For most minority groups, then, and most particularly the Negro, schools provide no opportunity at all for them to overcome this initial deficiency; in fact, they fall farther behind the white majority in the development of several skills which are critical to making a living and participating fully in modern society. Whatever may be the combination of nonschool factors--poverty, community attitudes, low educational level of parents-which put minority children at a disadvantage in verbal and nonverbal skills when they enter the first grade, the fact is the schools have not overcome it.

Coleman Report, p. 20

The conclusion can then be drawn that improving the school of a minority pupil will increase his achievement more than will improving the school of a white child increase his. Similarly, the average minority pupil’s achievement will suffer more in a school of low quality than will the average white pupil’s. In short, whites, and to a lesser extent Oriental Americans, are less affected one way or the other by the quality of their schools than are minority pupils. This indicates that it is for the most disadvantaged children that improvements in school quality will make the most difference in achievement.

Coleman Report, p. 21

Relying on the Coleman Report

I'm embroiled in a bit of a debate over at Russo's with frequent commenter (though not here) John Thompson over the conclusions I drew for the Baltimore first grade reading achievement study posted below.

The argument revolves around the popular premise that the effects of low socio-economic status cause low student achievement. I argue that large-scale empirical research like the Baltimore study and Project Follow Through refute such causal interpretations which are merely based on the correlational data we have on SES and student achievement, like that found in the Coleman report.

John, however, raises the following point regarding the Coleman Report:

You should realize you wouldn't be digging yourself into such a deep hole if you would back off from gratuitous attacks on others, like the people who issued the Bolder Broader challenge, and didn't try to refute the Coleman Report with sweeping comments.

I am not trying to refute the Coleman report. I am refuting the causal implications John and his Broader, Bolder allies draw from the Coleman Report.

The Coleman Report of 1966 was based on achievement data on over a half million students. The Report noted the disparity that existed between at-risk students and those not at-risk. The report compared schools of equal physical characteristics serving both groups. Coleman's finding was that money spent on smaller classes, laboratories, counseling, higher teacher salaries, and higher teacher qualifications were not correlated with academic achievement.

If the physical characteristics of the schools and all the other factors made no difference in student performance, there still remains the possibility that the instruction at-risk students received was inadequate. This possibility, which has been shown to exist in such large scale controlled research studies as Project Follow through and the Baltimore study serves as clear indictment of the educational system and a clear premise that the instruction provided to at-risk kids needed to improve.

The premise goes like this: if the instruction is inadequate, school factors won't matter and SES effects will predominate. Conversely, if the instruction is adequate, school factors will predominate rather than SES factors. This premise is consistent with the Coleman Report findings which examined an educational system in which inadequate instruction predominated, before we discovered that adequate instruction did, in fact, exist.

This data has not been refuted by the Broader, Bolder proponents. They simply ignore it and hopes that no one will notice. This is precisely the reason I always bring it up--to watch them go through rhetorical gymnastics trying to explain it away.

Developmentally Appropriate Practice is Not Developmentally Appropriate

The Summer edition of American Educator is out and has another must-read article by cognitive scientist, Daniel Willingham, on the inappropriateness of developmentally appropriate practice in education.

I view "developmentally appropriate practice" as one of the many excuses educators use to avoid teaching at-risk kids, i.e. kids who are developmentally behind their peers, what they need to know to achieve academically. As with many of our most pernicious educational practices, this one has roots with Piaget. Here's a shocker:

let’s review Jean Piaget’s theory. Although development psychologists no longer believe that his theory is right, it is a good starting place

It is unfortunate that so many teachers and ed school professors haven't gotten this memo yet.

Willingham makes the point that there is considerable variability in children's cognition. If a child fails to understand a concept, for example, it does not mean that the task was somehow developmentally inappropriate. It often means that the task presented was flawed or otherwise, deficient.

For example, suppose you read Make Way for Ducklings to a preschool class. Midway through the story you ask, “What do you think will happen next?” and you are met with blank stares. You might think to yourself, “That question was developmentally inappropriate. It was too abstract to ask them to think about the future.” Maybe. But maybe no one has ever asked them to make a prediction about a story, and so they were just unsure of what to do, and would have answered readily if you had said, “Do you think the ducks will go back to the park or stay where they are?” Or maybe they hadn’t understood the story very well to that point, so they knew what you were asking, but they just didn’t know what might happen next. Or maybe they just don’t know that much about ducks.

If a child, or even the whole class, does not understand something, you should not assume that the task you posed was not developmentally appropriate. Maybe the students are missing the necessary background knowledge. Or maybe a different presentation of the same material would make it easier to understand.

This, I think, is the largest flaw of many of the common education memes like developmentally inappropriate practice. It provides a convenient excuse to focus the student's failure to learn on the student himself rather than on how the material was presented to the student. If the student has failed to understand, there is often an underlying reason which needs to be discovered, analyzed, and the presentation remedied to avoid the confusion preventing the student from learning. Labelling the task as developmentally inappropriate allows the teacher to avoid this difficult task. Being able to remedy the presentation to avoid students' failures of understand is one of the reasons why we pay for highly-educated professionals, and not merely trained technicians. Yet, oddly, educators are often reluctant to engage in this difficult activity, preferring the recourse of a myriad of labels that shift the blame for failing to learn to the students.

Many teachers don't even like having their curriculum scripted for them. Scripting the curriculum merely shifts the burden of creating teacher presentations that have been field tested and are known to be understandable by students to someone else. Being that teachers don't like doing this work on their own, you'd think they wouldn't mind if someone else did it for them. Yet, they don't, and I'm not quite sure why.

It's as if they want to be professionals and be treated like professionals, yet don't want to engage in the the hard work required of professionals to assure their services are being rendered properly. Willingham concludes:

If we accept that students’ failure to understand is not a matter of content, but either of presentation or a lack of background knowledge, then the natural extension is that no content should be off limits for school-age children.

Yet, developmentally appropriate practice frequently places content off-limits for many at-risk students because if is more difficult to present the material to them in a way that they understand.

As they say, go read the whole thing.

Apparently the poor can be taught to read if they're taught properly

Here's a large scale study that answers two important questions regarding the educability of "poor" students.

1. Poor kids often come to school far behind their middle class peers in reading ability, can this initial achievement gap be eliminated in a timely manner?
   
2. Can this initial achievement gap in reading ability be eliminated by a school-based instructional intervention?

The study is Academic Acceleration in First Grade Reading Using the Direct Instruction Model by Michael Rebar answers both questions in the affirmative.

I alluded to this study in an earlier post on Douglass High. As I indicated in that post, in the mid 90s Baltimore decided it would mandate the use of SRA's Open Court reading program in all its public elementary schools. At the same time, The Baltimore Curriculum Project contracted with the National Institute for Direct Instruction (NIFDI) to provide training, coaching, and support for reading instruction in eleven Baltimore schools.

This set up the conditions for a nice experiment. Eleven Baltimore City schools were found with similar demographics and achievement levels to the 11 NIFDI schools to serve as controls. Thus, the study involved three groups: the 11 NIFDI schools, the 11 matched control schools, and the remaining Baltimore schools. In total, between 1998 and 2003, 41,223 kindergartners and first graders in Baltimore participated in the study.

Here is the demographic break-down for all three groups:




The above table shows the average percentage of poor students (based on free and reduced lunch participation) in the NIFDI schools, the control schools, and the remaining Baltimore schools for each year of the experiment (1998-2003). As you can see, the poverty rate in Baltimore is high (about 72%) but the poverty rate at the NIFDI and control schools was even higher -- these were some of the poorest schools in the district. These schools were also the likely feeder schools for Douglass High.

The experiment was phased in over three years -- 1997-1999. Five additional schools initially chose to implement the NIFDI model, but dropped out; they are included in the Other Baltimore group. The number of schools in the Other Baltimore group varied between 103-122 schools.

In the spring of 1997, the Peabody Picture Vocabulary Test (PPVT) was administered to kindergartners in the NIFDI and Matched Control conditions. The PPVT is a widely used norm-referenced picture identification test that is highly predictive of future reading ability. The results serve as the best estimate of initial achievement reading level (i.e., prior to intervention).

The NIFDI schools received two 30 minute periods of reading instruction per day in K and 1. The curriculum was Reading Mastery Classic. The NIFDI schools also received an additional 30 minute period of language instruction using the DI program Language for Learning and Language for Thinking which focus on oral language development.

Prior to 1998, the Matched Control and Baltimore condition schools were free to use any curriculum program desired. There was no district-wide structured reading program and schools used a variety of instructional programs. In the fall of 1998, the district adopted Open Court Reading in kindergarten through second grade.

Let me translate that into English. Prior to the experiment, Baltimore schools taught Reading in whatever manner they desired with predictably bad results. In 1997 Baltimore schools were performing at the 27th percentile on the PPVT. In 1998, all but 11 of Baltimore's schools switched to the research-validated Open Court Program. The remaining 11 schools switched to DI under the guidance of NIFDI. Thus, the experiment allows us to make three comparisons: 1. between DI and Open Court, 2. between DI and the pre-experiment reading curricula, and 3. between Open Court and the pre-experiment reading curricula.

In the spring of 1998 and 1999 the Comprehensive Test of Basic Skills, Fourth Edition (CTBS/4) (CTB,1991) was administered to all Baltimore first graders. The Comprehensive Test of Basic Skills, Fifth Edition (CTBS/5–TerraNova) was administered to all first graders in 2000 through 2003. Here are the results., including the PPVT baseline results.



The NIFDI schools went from the 15th percentile in 1998 to the 75th percentile in 2003, a significant increase in student achievement. In contrast, the Matched Controlled schools (using Open Court) went from the 14th percentile in 1998 to the 40th percentile in 2003, also a significant increase, but not as much as the NIFDI schools. The Other Baltimore schools (using Open Court) went from the 24th percentile in 1998 to the 59th percentile in 2003, also a significant gain. Here are the effect sizes for each group.



As you can see, the NFDI intervention had a very large effect size of 1.87 standard deviations. Let's put this in context. The NFDI intervention was 750% more effective than lowering class sizes to 13-17 students. In fact, since most non-instructional interventions perform about as well or worse than lowering class size, the NIFDI DI intervention was at least 750% better than those as well.

Even the Matched Control Open Court schools had a large effect size of 0.90 standard deviations or 360% more effective than lowering class size. That's not too shabby at all.

Below is a graph of the NIFDI schools showing performance gains in normal curve equivalents.



Here's the analysis from the study:

The average student performance level in the NIFDI schools moved from the lowest quintile to well above the national mean. All NIFDI schools showed substantial achievement gains and several became beacon schools for the district. The three highest achieving schools in the district in 2003 were NIFDI schools. The Matched Control schools never achieved a mean score above the national average.

In all, 49 of the 103 Baltimore condition schools (48%) achieved a mean Total Reading NCE score of 55 or greater in the 2003 school year. In the NIFDI condition, ten of eleven schools (or 91% of these schools) reached this level. Only one of the eleven Matched Control condition schools reached an NCE score of 55; this school ranked 49th in the Baltimore condition.

The Fisher’s Exact Test indicates that schools in the NIFDI condition were significantly more likely to exceed the national mean than either the Baltimore or Matched Condition schools. The three highest achieving elementary schools in the district were NIFDI schools, achieving mean Total Reading NCE scores of 92, 90, and 83. The highest three schools in the Baltimore condition achieved scores of 78, 76, and 75. It is perhaps ironic that the highest achieving Baltimore school (with a mean NCE score of 78) is one of the DI schools that did not continue as a NIFDI school but continued to use the Direct Instruction reading program.

There you go. The top four schools in Baltimore were all schools using DI. Of course, the Baltimore schools, despite a long history of academic failure, still though they knew the best way to teach:

Despite the gains demonstrated with the NIFDI model in Baltimore in 1997 through 2003, the district did not systematically expand the model to other schools. In fact, during the course of this study the district mandated procedures that were at odds with model provisions. Teachers were required to attend district inservice trainings that advised them to do things differently than what the NIFDI model specifies. The central administration required schools to provide daily test-preparation periods for the entire school year. NIFDI advised the principals in their model not to do this because it believed greater gains would be possible by implementing the model rather than providing test preparation (which would also tend to artificially inflate achievement scores). In fact, the six principals who implemented the model most faithfully and ignored some of the district mandates achieved better performance than any of those who complied with district regulations.

So what do you think Baltimore did when faced with these improved scores?

That's right, they stopped the program and went back to what they were doing before the intervention. Unfortunately, I've been unable to find CTBS scores from 2004-2008.

So the answer to our initial questions is: Yes, we can eliminate the achievement gap between low-SES students and their middle-class peers in reading by the end of first grade by means of a school-based instructional reform. In fact, we can get low-SES kids to significantly outperform their middle-class peers. These achievement gains can be accomplished without any SES-based reforms or by reducing any of the symptoms associated with growing up in a low-SES environment.

So, how do all you Broader, Bolder types explain these results?

Douglass High: Pro NCLB?

I caught a partial view of HBO's Douglass High Documentary and I have a question.

Was the Documentary supposed to be anti-NCLB or pro-NCLB?

To me it seems pro-NCLB, though that was probably not intentional. I can't see how anyone could view the documentary and come away with the impression that the administrators of Douglass don't need independent oversight. Their idea of education seems to be seat time. If the student is capable of sitting in his seat during class, regardless of what he's learning, he'll be passed along for four years and graduate. What he's actually learned, if anything, is irrelevant.

I think the average person would be horrified at what passes for education at Douglass. It is probably best that they don't know because I think support for the notion of public education would be greatly diminished.

Sixty percent of Douglass' student drop out between ninth and twelfth grade. Granted, most of the student came to Douglass way behind where they should have been, but the performance of the 40% that didn't drop out was appalling. Only a tiny percentage of students tested as proficient in math and reading. These were the good students, with parental support. They all seemed well fed and well dressed (fashion sense notwithstanding). The teachers admitted that nearly none were at grade level, yet miraculously almost every student qualified for graduation in the end.

I don't think this is what the public has in mind when they signed on for public education.

More on Douglass High

The debate continues to rage over who or what is responsible for Douglass High's poor academic performance.

Douglass High's students' fate was determined long before they entered high school. First Grade performance is a good indicator of subsequent academic performance. If anything, first grade performance tends to be a high water mark.

When the students shown in the Douglass High documentary (shot in 2004-05) were in first grade, Baltimore's academic performance was pitiful--so pitiful that Baltimore decided to take drastic action in 1998. That year Baltimore mandated that almost all Baltimore schools adopt the Open Court reading program as their core reading curriculum. That year they also decided to test all first graders using CTBS. Here are the results.



In 1998, Baltimore students were performing at about the 24th percentile on CTBS/4. And, you can bet the Douglass was drawing students from schools falling below this average. The Douglass students shown in the Documentary were past first grade by 1998, but the low scores of the 1998 cohort are indicative of the quality of education they received in K and 1. Academically, you could write off 75% of Baltimore students. There is a very good chance that a struggling read in first grade will be a struggling reader in third grade and last I checked it's difficult to motivate a child to learn if they can't read very well.

After adopting Open Court, Baltimore's first grade reading performance shot up to the 59th percentile. These 2003 first graders are not yet in high school.

I think it's safe to assume that the incidence of toothaches, bellyaches, student hunger, racial discrimination, bad home life, uncaring parents, and all the other excuses teachers like to bandy about to excuse their performance remained about the same. The only thing that changed was the reading curriculum to something that has a validated research base. Seemingly by magic, reading performance improved by about a standard deviation. This, supposedly, is impossible.

The study from which I stole this table actually indicates that student reading performance in Baltimore could have (and actually was) improved by twice this amount in 11 high-poverty schools that adopted an even better reading curricula. You'll have to wait for the next post to find out more.

Nonetheless, it's safe to say that a large part of the problems present at Douglass High could have been prevented if the Baltimore schools actually did a better job doing what we pay them to do -- educating children.

Douglass High

Though my DVR has yet to record HBO's documentary, Hard Times at Douglass High, that's not going to stop me from opining anyway.

Douglass High's primary problem is that most of its students do not possess the skills needed to access a high school curriculum. Most of the students don't belong at Douglass High or any high school for that matter. Most of the them belong in remedial classes teaching elementary school material. This, of course, assumes that the students are still willing to sit in a classroom attempting to learn after experiencing nine years of academic failure. I doubt many of them are. I suspect many of them are causing serious disruptions to whatever classrooms they happen to be in.

Douglass High's problem is social promotion. The elementary and middle schools that feed into Douglass High failed to properly educate their students and passed the buck right on to Douglass High. Douglass was handed a thankless job.

Douglass High is the end point for a very broken educational system. Even if Douglass High were capable of remediating its students at a high rate, the results would still likely not show up in any state test.

Douglass High's primary problems lie elsewhere.

News from the trenches

My son, who begins third grade in the fall, just finished Connecting Math Concepts level C (CMC C). CMC C is billed as a 2nd or 3rd grade curriculum.

We went through the 120 lesson sequence sporadically on weekday nights after school and weekends. It took about nine months. Had we been more diligent, we would have gotten through it in five months.

In his day job, i.e., second grade, he covered most of the 2nd grade Everyday Math (EM) curriculum. Generally, we were ahead of what his school was teaching in EM, so EM mostly served as additional practice for what he was learning in CMC. Both curricula covered roughly the same skills, if anything CMC covered more topics than EM.

CMC provided significantly more distributed practice than EM which should not be surprising. The benefit of all this distributed practice was that at the end of the sequence he was able to take and pass a cumulative test of all the material covered in the sequence. In other words, he had retained material that had been covered up to nine months ago. We will jump right into level D because the more we do this summer, the less we will have to do during the school year, what with regular homework, piano lssons, various sports, and the like. I'm confident that he could go the entire summer without forgetting what he was taught, at least nothing that couldn't be refreshed with a few lessons of review. CMC Level D provides that review, but we'll skip those parts.

In CMC, lessons are easy to teach because they are scripted for an entire classroom. The difficult part is shortening and modifying the script to suit teaching a single student. I will present the script until he has learned what was intended and learn, which usually takes one example. Then I monitor his working of the remaining examples. Two thirds of each lesson is usually independent work practicing skills he's already learned. He does those exercises on his own, though I often let him skip parts of exercises if he balks or seems bored by the problems. Surprisingly, he will often work problems he already knows how to solve well without complaint.

We usually get through a lesson in about half an hour, fifteen minutes of which is independent work. I suspect presenting to an entire classroom would take longer. I also suspect that I'd have to do more teaching for average and lower performing students.

EM is known for its steep spiral. Topics come and go quickly. Nothing is taught to mastery. And, quite frankly, I don't see any of the supposed conceptual learning advantages I see touted. CMC does not teaching using a spiral and everything is taught to mastery. Needless to say I was surprised that CMC stayed ahead of the EM spiral the entire school year. To put it simply, CMC teaches much more efficiently than EM.

Perhaps too efficiently. My son has little tolerance for the hokey manipulative exercises that clutter EM, such as make a grid of dots to conceptualize multiplication. He knows how to do most addition, subtraction, multiplication, and some division problems mentally, so that is how he wants to solve problems -- mentally. I suspect there will be a clash next year when he writes down his "magic" solutions to problems. We shall see.

He knows the math part of math well. That's a good thing. he doesn't know some other important parts of math so well, however. he doesn't know the importance of neatness yet. he doesn't consistently provide the units for word problem answers. He doesn't consistently indicate the answers by, say, boxing them.

These problems are all my fault. I should have held him to a stricter standard, but I didn't for various reasons, mostly time considerations and to minimize crankiness when we did this extra work before bedtime. These problems are easy to remedy. Remedying deficient math knowledge not so much.

Next year he'll run through the EM spiral again reviewing, and for many students relearning, all the partially taught topics from second grade. In the meantime, he'll continue to overlearn the topics material covered in CMC D. This will lead to automaticity in elementary math skills which will be needed for learning algebra. That's the ultimate payoff -- learning algebra. If you don't know algebra, you're pretty much foreclosed from pursuing study in the hard sciences and engineering.

Facebook

Although I am way behind the curve, I just joined Facebook. Looks like fun. I sent out a bunch of invites from my yahoo address book. If I missed you, feel free to add me.

Question 6: Why not to use manipulatives for learning math

Ari-free asks:

in an interview, Zig Engelmann said that he doesn't believe in using manipulatives for teaching math. I thought it was universally accepted that kids start with counting blocks, fingers, etc

Here is what Engelmann and Carnine have written on the topic or the need for making mathematical connections:

Connecting Math Concepts presents skills in applications in a way that permits children to connect concepts.

...

If the program did an effective job of articulating connections between various concepts, the child who completes the first level would have a strong conceptual foundation on which to build...

How does Connecting Math Concepts forge these basic connections?

The principal vehicle is the number line. The number line is potentially dangerous for teaching number facts because it provides children with a method of figuring out the answer without attending to relationship. To figure out 4 plus 3, the children simply go to 4 on the number line and then move three spaces further. Although children learn the right answer, they don't easily learn the relationship between 4+2 and 4+3 because they have never been required to do anything that focuses on the relationship. The just follow the number line as a "rote" procedure.

A similar problem exists with any method that permits children to count and figure out the answer to problems like 4 plus 2. Finger operations are logical. (Hold up 4 fingers; hold up 2 more; count all.) However, these operations don't prompt systematic learning of relationships because children are not required to attend to relationships. Line-making operations suffer from the same problem. They provide children with a reliable method for figuring out answers to problems; however, they do not induce fact knowledge or knowledge of relationships. Furthermore, they often militate against systematic learning of facts because so long as the child is able to use finger-counting operation or a number line, what motivation is there for learning facts?

Connecting Math Concepts, Level A, Teacher's Guide, pp. 6-7.

Question 5: Various Questions about DI

Robert asks:

What is the total number of people/budget for R&D in DI? What portion of that is outside of Eugene Oregon?

I'm not privy to the inner workings of the various DI organizations like ADI and NIFDI and the Becker/Engelmann Corp, but I do know they are small organizations.

Why are not more of the materials made targeted at the home school market? 100 math lessons, 100 writing lessons..etc.

This is probably a licensing issue with SRA. I agree that it would be a good idea to adapt the DI materials for the homeschooling crowd which is a big untapped market.

It seems to me that DI is good at instilling basic skills, probably to about the 5th grade level. But that there must be a transition at some point to a more conceptual, self directed learning style after that. Such that an properly prepared 5th grader (or 6th or 7th etc) should be able to be given a algebra book and expected to learn form it and ask questions every few days if they need help. Given that I know this happens with some kids, I think it good to figure out if this could happen with more kids. Part of this also has to deal with instilling an internal drive, because an unmotivated child being given an algebra book isn't going to get real far.

Drawing back a bit, DI can be used to transmit known systematized information, but at some point the learner needs to get beyond that into creating knowledge of their own...it seems like there should be some more transitions between DI and Graduate School, but I am not sure what those transitions look like.

Any ideas?

I am somewhat familiar with most of the DI classroom materials. Most of the scaffolding is systematically removed between level 1 and level 6 with the intention that students completing level 6 should be ready for instruction in a more traditional classroom setting or independent work.

I don't know how successful this transition is with lower performers; the data is scant.

Does DI have any long term affect on measured IQ?

I doubt it. I doubt it even has an effect in the short term. The Follow Through students all took Ravens Progressive Matrix tests (which is highly g-loaded) which did not show significant improvement in IQ. However, IQ is often estimated from achievement tests and on achievement tests the DI students showed substantial improvement in achievement and IQ in the short term. But, the achievement tests require further achievement gains to maintain gains, so its difficult to attribute this achievement to DI once the students are out of DI.

What can be done to improve long term IQ?

Presently, it appears that we don't know of anything that is capable of improving IQ past adolescence.

What can be done to maximize a persons ability to capitalize on the IQ that they have? (outside if DI)

Clear teaching and sufficient practice for retention.