John Dewey is No Mr. Miyagi

File this under unclear on the concept. (Hattip: Hun-blog)

I must have stumbled into an alternate universe. A place where everything is not quite right -- like the episode of Star Trek where Spock had a goatee.

Nothing else could possibly explain this article's ham-fisted attempt to justify the progressive education lunacy of John Dewey using the cognitive science behind "effortful study." The lede is buried near the end:

The connection between Toyota, John Dewey and the Algebra Project lies in the research of K. Anders Ericsson, a psychology professor at Florida State University, who has shown that doing leads to learning, and learning by doing leads to doing better.

As you hopefully recall from this recent series of posts, Ericsson's research is the foundation for our understanding that effortful study, i.e., practice until perfect, is necessary in becoming an expert in nearly every field of human endeavor, including academics. To think that this in anyway validates John Dewey's discredited claptrap is a stretch to say the least.

John Dewey, the pragmatist philosopher and originator of progressive education, harnessed the power of learning by doing three decades before Toyota. In 1895, he founded the Laboratory School at the University of Chicago. His mission was to "reinstate experience into education"; as a result, Laboratory students spent most of their day outside the classroom, engaging in activities such as sewing, carpentry and cooking. But these activities weren't simply exercises in manual labor. Rather, they were demonstrations of "active learning." "If a child realizes the motive for acquiring a skill," Dewey argued, "he is helped in large measure to secure the skill."

The purpose was to make education seem indivisible from action. "Absolutely no separation is made between the 'social' side of the work, its concern with people's activities, and the 'science,' regard for physical facts and forces," Dewey wrote in 1899, in his best-selling pamphlet "The School and Society."

To Dewey, students learned by experiencing not by being taught. In Experience and Education (1938), Dewey juxtaposed traditional and progressive perspectives:

To imposition from above is opposed expression and cultivation of individuality; to external discipline is opposed free activity; to learning from texts and teachers, learning through experience; to acquisition of isolated techniques by drill, is opposed acquisition of them as means to attaining ends which make direct vital appeal; to preparation for a more or less remote future is opposed making the most of the opportunities of present life; to static aims and materials is opposed acquaintance with a changing world.

According to Dewey, students learn by experiencing. That's why he had them doing carpentry, sewing, and cooking instead of being taught the three Rs by teachers in a classroom. Dewey's mistake was that he had his children practicing carpentry, sewing, and cooking instead of practicing reading and math. They probably become good carpenters, cooks, and seamstresses instead of good readers and math nerds.

To understand why Dewey was so wrong, it helps to compare him to his modern day analogue, Mr. Miyagi of Karate Kid fame who taught Daniel-san karate by making him paint his fence, sand his floor, and wash his car. At first this "training" appears to be an elaborate ploy to skirt the child labor laws, but it turns out that our hero was actually practicing karate moves while he performed those menial task. In the movies, the karate kid eventually becomes a karate expert and kicks the asses of the Cobra Kai kids. In real life, John Dewey's students failed to learn academics by building pretty birdhouses and got their asses kicked by life.

Yet, Dewey's progressivism thrived. Child centered education is all the vogue. As the article points out, today's progressive's "teach" algebra by experiencing algebra. No, really:

Moses spent the next five years developing a completely new curriculum. He called it the Algebra Project. Instead of confronting students with abstract equations, Moses took them out into the real world, traveling around Boston in search of experiences that could demonstrate the practical uses of math. A ride on the T became a lesson in coordinate graphing and negative numbers. Neighborhood landmarks stood in for integers. When Moses taught students about displacement, he had them measure the dimensions of their own bodies. The first rule of Moses' math class was that students always had to "participate in a physical event."

His unconventional methods changed the way the students felt about math. "When you take a trip on the subway and learn about algebra," Maisha says, "what you're really doing is developing a new way of thinking. Instead of just trying to memorize these strange equations, you're busy relating the math to your own experiences. All of a sudden, you find math spilling over into other areas of your life."

Riding the subway to learn about algebra seems about the furthest thing away from actually "doing" alegebra that I can think of. And, I don't see any Mr. Miyagi subterfuge in this "training" either. Learning algebra like this, would be like Tiger Woods taking a golf cart tour of golf courses in order to experience golf instead of practicing golf from before he could walk.

Thanks to an encouraging father who happened to be a golf fanatic, Tiger took his first golf swing before he took his first steps. When he was 18 months old, his dad started taking him to the driving range. By the age of three, Tiger was better than most weekend amateurs.

This allowed Woods to get a head start on his current competitors, but what really made him great is how he practices. For starters, his routine is merciless. Rain or shine, Woods sets out. More importantly, he always makes sure his practice sessions revolve around learning by doing. He analyzes sequential snapshots of himself playing, relentlessly scrutinizes the elements of his swing, then drills these subtle alterations into his nervous system through thousands of repetitions. Of course, more practice leads to more new ideas, which leads to more practice. "Other golfers may outplay me from time to time," Woods wrote in his book, "but they'll never outwork me."

The right way to learn algebra is by "doing" algebra. Doing algebra means sitting down and solving algebra problems not engaging in hokey "real world" activities that purport to enable the student to relate the abstract concepts of algebra to things they've experienced. This isn't the kind of practice that makes experts. This is the kind of practice that just wastes time. Oddly enough, the article actually refutes its own premise, though the author appears to be oblivious to that.

[Ericsson] began looking in detail at the way people practice. He noticed that the best performers had a unique training style. They tended to downplay mindless drills and rote repetition. Instead, their practice sessions were deliberate, creative and thoughtful, like the outings of the Algebra Project or the progression of a rat through a maze. They set specific goals for themselves, continuously analyzed their progress and focused on process. "A crucial part of practicing well is that you are always learning while practicing," Ericsson says.

Solving algebra problems is neither rote repetition nor mindless drill. When will the dopey progressives get this through their thick skulls. Solving algebra problems you've just learned how to solve is exactly the kind of practice that is needed to gain expertise. Sufficient practice will lead to automaticity. Continued practice beyond automaticity will likely become mindless repetition. But any decent teacher will have moved on to the next challenge by that point.

Reading challenging texts and solving challenging math problems is not mindless rote memorization. Characterizing this kind of practice as "mindless rote" doesn't in any way justify loopy progressive theory. Is it mindless when Tiger Woods practices his swing over and over? Is it mindless when a concert violinist practices the same piece over and over? The only mindless activity I see in this entire article is taking a subway ride to learn about the coordinate system.

Finally, we have the obligatory swipe at standardized testing.

This is the paradoxical flaw of standardized testing--—in the rush to quantify learning, it discourages the sort of teaching that actually gets results. Instead of learning by doing, children are forced to memorize a random-seeming body of knowledge. Even if students pass the test, they never learn what to do with all their new information. As a result, they quickly forget the lesson plan—probably while dreaming at night.

Paradoxically, this paragraph is almost accurate. If students did in fact learn by doing and engage in sufficient practice to overlearn the material, they might have not fallen prey to the ravages of forgetfulness. However, learning through the progressive techniques of Dewey which wastes time which could have been better spent engaging in real practice, virtually ensures that the material won't be mastered and soon forgotten.

This is Dewey's legacy.

Effortful Study Requires Mastery Learning

In my last post on making experts, we discussed how effortful study entails continually tackling challenges that lie just beyond the student's competence.

Let's see how effortful study might play out in an academic setting. One such program that I know of that employs effortful study techniques is the program described in Engelmann's Student-Program Alignment and Teaching to Mastery. This is not a DI program per se, but DI is certainly based on its principles. Engelmann calls his effortful study program mastery learning and describes it as such:

A program design that supports mastery does not present great amounts of new information and skill training in each lesson. Rather, work is distributed so new parts in a lesson account for only 10–15 percent of the total lesson. The rest of the lesson firms and reviews material and skills presented earlier in the program. The program assumes that nothing is taught in one lesson. Instead, new concepts and skills are presented in two or three consecutive lessons to provide students with enough exposure to new material that they are able to use it in applications. So a lesson presents material that is new today; material that is being firmed, having been presented in the last two or three lessons;and material that was presented even earlier in the sequence and is assumed to be thoroughly mastered. This material often takes the form of problems or applications that require earlier-taught knowledge.

Each new lesson contains a small amount (10-15%) of new material and lots of practice with previously presented material. Here's Engelmann's rationale.

The amount of new material is relatively small because most students are not capable of assimilating more. This design provides for some “overlearning,” but having the program err in the direction of providing too much practice is better than providing too little practice. Work on material presented in the preceding few lessons is needed to ensure that students are "automatic" with information or operations that were previously taught.

There goes the key word automaticity. Students are to learn the material so well that they are automatic with it. When knowledge is automatic it requires little cognitive toll to recall and use.

Student performance is judged on their first time correct responses, i.e., how well they respond to the material the first time it is predsented in a lesson. Engelmann has four criteria:

Criterion 1. Students should be at least 70% correct on anything that is being introduced for the first time. This percentage is based on the understanding that even the new skills or procedures that are being introduced are not composed entirely of material that is new. Much of it will be familiar. Therefore, the initial rate of correct responses should not drop below 70 percent. If students are at mastery on the preceding lessons, this outcome will occur in almost all cases. If students perform much below 70 percent, they are not learning the material. If they are only 50 percent correct, they may be at a chance level—guessing at the answers or the steps in the operation. Their responses are not generated by an overall understanding of what they are learning.

Criterion 2. Students should be at least 90% correct on the parts of the lesson that deal with skills and information introduced earlier in the program sequence. Criterion 2 is based on the fact that students must be completely at mastery on earlier-taught material. When earlier-taught material occurs in later lessons, no reteaching should be required. If substantial reteaching is needed, the amount of new learning that students must achieve to master the lesson becomes too great.

Criterion 3. At the end of the lesson, all students should be virtually 100% firm on all tasks and activities.

Criterion 4. The rate of student errors should be low enough that the teacher is able to complete the lesson in the allotted time. If students enter the lesson with skills that permit them to attain 70 percent correct on new material and 90 percent correct on material taught earlier, students should be able to achieve virtually 100 percent on all exercises presented in the lesson.

To meet these four criteria, Engelmann devised a system having the following seven requirements.

1. All students must be appropriately placed in each instructional program. All placements are based on first-time-correct performance. Mastery is not possible unless students are placed according to the criteria for first-time-correct performance.

2. All groups must be homogeneous with respect to the performance level of all students in the group. This requirement is an extension of the first-time-correct requirements. Unless all students in the group are appropriately placed, the teacher will not be able to bring the group to mastery in a reasonable amount of time. The teacher will have to spend time providing additional practice to students who should not be in the group. This additional practice tends not to serve students who need it nor the other students, who waste time while the teacher works on firming skills that they have already mastered.

3. There are actually three critical scheduling issues. The first is that adequate time must be scheduled on a daily basis for teaching each group each subject. The second is that the schedules must be coordinated to permit relatively easy movement of students from one instructional group to another, based on their performance. The third issue is that movement of students from one instructional group to another should occur frequently throughout the year. All schedules must be coordinated across classrooms and grades so that cross-class grouping and regrouping is possible.

4. Schedules must provide adequate time for each subject and each instructional group, and teachers must faithfully follow schedules. The schedules must include time for workchecks, so that students receive timely feedback on any mistakes they made, and so teachers receive information about any skills or items that need additional firming.

5. A group’s progress in mastering new material must be continuous throughout the year. If the group completes level 3 reading in the middle of February, students must begin level 4 within no more than two or three school days. Level 4 should not be delayed until the beginning of the next school year.

6. All teachers must enforce the same set of schoolwide management rules and practices for celebrating academic achievements. There should be rules for how students are to behave in the class, so that if students misbehave, they understand both the rule that they broke and the consequence. The system of rules should be designed so students receive reinforcement for complying with rules.

7. The performance of students must be regularly monitored. The school must have systems for regularly monitoring students’ progress. The monitoring information may consist of weekly summaries of progress in each subject, summaries of student performance on in-program tests, and reports on daily independent work. The purpose of the monitoring is to guarantee that no students fall through the cracks and that all receive the best instruction that the school is able to deliver.

Finally, Engelmann has devised four rules for teaching to mastery in his system.

Rule 1: Hold the same standard for high performers and low performers. This rule is based on the fact that students of all performance levels exhibit the same learning patterns if they have the same foundation in information and skills.

Rule 2: At the beginning of the school year, place continuing students who have been taught to mastery no more than 5 lessons from their last lesson of the preceding year. If something is thoroughly learned and applied, it will be retained by lower performers as well as by higher performers.

Rule 3: Always place students appropriately for more rapid mastery progress. This fact contradicts the belief that students are placed appropriately in a sequence if they have to struggle--scratch their head, make false starts, sigh, frown, gut it out. According to one version of this belief, if there are no signs of hard work there is no evidence of learning. This belief does not place emphasis on the program and the teacher to make learning manageable but on the grit of the student to meet the “challenge.” In the traditional interpretation, much of the “homework” assigned to students (and their families) is motivated by this belief. The assumption seems to be that students will be strengthened if they are “challenged.”

This belief is flatly wrong. If students are placed appropriately, the work is relatively easy. Students tend to learn it without as much "struggle." They tend to retain it better and they tend to apply it better, if they learn it with fewer mistakes.

Rule 4: Move students as quickly and as reinforcingly as their performance permits. This rule opposes the notion that teaching to mastery is somehow synonymous with having picky or punishing standards.

According to Engelmann, mastery learning will confer the following benefits to students, teachers, and the school system.

Students benefit by becoming much more competent and by gaining options for their futures they otherwise would not have. Teachers benefit because students who are taught to mastery tend to succeed; therefore, teaching becomes easier. Schools benefit because students are much easier to teach in the upper grades if they have a solid mastery foundation starting in kindergarten. In the upper grades, students are able to learn new material at a good rate, and the bottom end of the student population performs more like traditionally taught students.

Two types of performance change occur in students. The most obvious is that students learn more material during a specified time period. The second change is in their ability to learn new material. There is a simple relationship between the amount of material they master and their overall facility to learn new material: The more success students have with a particular type of material, the better they become at it.

Teaching to mastery also instills self-confidence in students because they learn they are capable of learning whatever new skills or material the teacher presents. Their positive attitude is firmly grounded in experience. Because students have learned everything the teacher has taught, students understandably have confidence that it will happen the same way for future instruction.

Engelmann's mastery learning program works because it conquers the student's motivation problem. In Engelmann's program students are not only challenged every day with new material, but they also practice previously persented material until it is automatic. Students are placed, grouped and proceed at a pace in which they will have learned 100% of the material in each lesson by the end of the day. Student performance is continually monitored to ensure that students are in fact learning the material. The scope and sequence of the program is such that new material is based on previously presented material. Mastery learning ensures that students are successful in learning the material; therefore, students will be motivated to continue their success in learning new material. In short, motivation is not a problem in a properly designed and implemented mastery learning program.

Clearly, Engelmann's mastery learning program is based on sound cognitive science principles, rather than made-up faddish nonsense. We also know that it works because Engelmann's Direct Instruction program is based on these principles and it has been successfully shown to increase student achievement in numerous research studies involving thousands of children.

Of course, the instruction taht takes palce in most K-12 doesn't even come close to meeting the requirements of a mastery learning system as Engelmann has laid out. Predictably, motivational problems are rampant and all but a small faction of students are learning at the rigorous pace needed to succeed in college. Students don't retain information from one year to the next and the efficiency of the entire system is miserably low.

Who succeeds in our current system? The cognitive elite. Smart kids and not-as-smart kids with exceptional teachers and/or supportive parents. These kids learn quickly or have the support needed to learn at an acceptable pace. These kids don't require as much practice or get extra practice from their teacher and/or parents sufficient to master the material as it is presnted to them. these kids are successful learners and are motivated by their success and might be further encouraged by their teacher and/or parents.

Everyone else falls further and further behind as the years go on. this trend shows up clearly in NAEP and state exams. These kids are not mastering the material as it is presented. At best the material is partially mastered. Partial mastery is just another way of saying partial success which means partial failure. Failure kills motivation. Lack of motivation gets in the way of further learning. Eventually learning grinds to a halt or to a trickle. And, then its day care until diplomas are handed out.

So why don't more schools use mastery learning to increase student achievement? You'll find out in my next, and hopefully last, post on making experts.

Real World Physics Problems

Inspired by Rightwingprof's post on real world business problems that aspiring business majors need to know how to do, I'm going to give you a simple example of a real world phsyics problem that an aspiring science/engineering major will be expected to do. Actually, it's not real world, it's a greatly simplified real world problem -- real world problems are much more difficult:

The positions of a particle and a thin (treat it as being as thin as a line) rocket of length 0.280 m are specified by means of Cartesian coordinates. At time 0 the particle is at the origin and is moving on a horizontal surface at 23.0 m/s at 51.0°. It has a constant acceleration of 2.43 m/s2 in the +y direction. At time 0 the rocket is at rest and it extends from (−.280 m, 50.0 m) to (0, 50.0 m), but, it has a constant acceleration in the +x direction. What must the acceleration of the rocket be in order for the particle to hit the rocket?

You only need one physics equation to solve this problem: The distance formula.

The formula to determine the distance traveled by a moving object during a given time is:

distance = initial distance + initial velocity x time + 0.5 x acceleration x time²

Can you solve the problem?

All you need to solve the problem is the above formula and the math you should have learned in high school.

Let me give you a hint: The particle will collide with the nose of the rocket when it reaches the height (50.0 m) of the rocket since the rocket is flying level. How far along the x axis is the rocket at that point and how long did it take to get there? What is the acceleration at this time. What if the rocket hit the tail of teh rocket instead. What is the rocket's acceleration now.

Now you know the steps to solve the problem. Now get to it.

Can't do it? Let me make it easier for you by telling you every step of the solution.

1. Decompose the velocity of the particle into its x and y components. You need to know simple trigonometry and algebra to do this. Here's another hint: The sine of an angle equals the opposite side divided by the hypoteneuse. The cosine of an angle equals the adjacent side divided by the hypoteneuse.

2. Now use the distance formula and determine how long it will take for the particle to reach the height of the rocket. You'll need to know how to solve a quadratic equation to do this.

3. Now determine how far along the ground the particle will be when it reaches the height of the rocket.

4. Using the distance formula again determine the acceleration of the rocket if the particle were to strike the nose. This step requires simple algebra to solve.

5. Using the distance formula again determine the acceleration of the rocket if the particle were to strike the tail. This step requires simple algebra to solve.

6. The answer is that the acceleration of the rocket must be between the aceleration of the nose and the acceleration of the tail when the particle strikes the rocket or else it misses.

Ok, I've set-up the problem for you. I've conceptualized the problem for you. I've given you the physics. All you need to do is to use your high school math to solve.

Go ahead give it a whirl. It's still not an easy problem to solve is it? Especially if your algebra and/or trig skills are rusty. Try guessing and checking your way to the solution. I dare you to try.

If you're uncertain how to calculate an arcsine, how distracted do you think you'll be having to shift gears and ponder that while the physics problem is waiting to be solved. Or, maybe you are a real smartie and know how to derive an arcsine from first principles, go ahead work it out. The physics problem will still be waiting when you're done. Gee, I bet that was distracting.

Now we get to the quadratic equation. Remember how to solve one of those off the top of your head? That means you have to derive it yourself or go look it up. The physics problem is getting impatient now. Do you even remember where you were in the solution. What's the next step? What are you trying to solve anyway using the quadratic formula? What do you do with the extra solution you get when you finally solve the quadratic equation? Now all you have to do is use the distance formula to solve for three different variables. To do that you'll need to know how to manipulate algebraic expressions with some skill. Are you confused. Does your brain hurt yet?

This is a simple college level physics problem the likes of which is going to confront every aspiring science and engineering student in their first year of undergrad. It gets a lot tougher than this real quick.

This is one of the first gatekeepers students will meet. Most will not get through the gate. Most will not get through the gate because they did not have the domain knowledge in math to enable them to quickly solve the hundreds of physics problems they'll need to solve in order to acquire the domain knowledge in phsyics in order to pass their Physics course.

Once students have the physics domain knowledge, solving problems like this is simple and requires little cognitive ability. What seems like an insurmountable problem to the novice, requiring enormous cognitive effort, turns out to be a simple problem for the expert. One that can be easily broken down into simple steps that can be solved using one formula and some high school math.

Here's another real world question. Can you determine whether a student scoring at the advanced level of the 11th grade NAEP or any NCLB state exam knows the math necessary to do my real-world physics problem.

Answer: No. There is nothing on either the NAEP or any state exam even remotely approaching the difficulty level of math needed to do the problem. Less than 10% of students perform at the advanced NAEP level.

Becoming an Expert: -- Effortful Study

In Part I we learned how important it is to acquire vast storehouse of intricately-structured previously-acquired relevant knowledge to becoming an expert.

In Part II we learned why learning all that stuff was so important.

In Part III we'll focus on how to learn all that stuff.

Why is it in most areas of human endeavor we are continually improving in performance, yet in academics improvement has ground to a halt?

Meanwhile the standards denoting expertise grow ever more challenging. High school runners manage the four-minute mile; conservatory students play pieces once attempted only by virtuosi. Yet it is chess, again, that offers the most convincing comparison over time. John Nunn, a British mathematician who is also a grandmaster, recently used a computer to help him compare the errors committed in all the games in two international tournaments, one held in 1911, the other in 1993. The modern players played far more accurately. Nunn then examined all the games of one player in 1911 who scored in the middle of the pack and concluded that his rating today would be no better than 2100, hundreds of points below the grandmaster level--"and that was on a good day and with a following wind." The very best old-time masters were considerably stronger but still well below the level of today's leaders.

While watching swimming in the Olympics earlier this year and the Tour de France, I noticed that they've developed very sophisticated training techniques that, among other things, are designed to control the production of lactic acid in the athletes' muscles.

We've developed sophisticated chess programs and game databases that have greatly improved the level of game play. What about music?

There is usually no way to tell, from a recital alone, whether a young violinist's extraordinary performance stems from innate ability or from years of Suzuki-style training.

The "secret" to becoming an expert is effortful study which entails continually tackling challenges that lie just beyond one's competence. Two interacting factors go into effortful study: innate ability and motivation.

Think of expertise in an endeavor as the mastery of many related skills. Innate ability determines how quickly a student will learn one of those skills, how much effort is required to learn that skill, and how successful the student will be learning that skill. Motivation determines how long the student will keep at learning skills. Motivationis increased and strengthened by success. Success builds on success. So when a student is successful at learning a skill (success is a function effort which is a function of innate ability) they are motivated to keep on learning. Keeping up motivation is critical which means continual success is critical.

Any teaching/training program that is designed to build expertise must achieve student success and mastery of the material in order to build motivation so that the students keeps up the effortful study. During effortful study, the student is repeatedly challenged to learn new material, the student must successfully conquer each challenge presented so that he will be motivated and capable of moving onto he next challenge.

Thus, motivation appears to be a more important factor than innate ability in the development of expertise. It is no accident that in music, chess and sports--all domains in which expertise is defined by competitive performance rather than academic credentialing--professionalism has been emerging at ever younger ages, under the ministrations of increasingly dedicated parents and even extended families.

This is a bit of an overstatement, because innate ability will determine how successfully and easily the student will be learning each new challenge. Thus, innate ability will determine motivation. Innate ability will also determine how quickly a student will learn. Most importantly, innate ability determines how good the teaching/training program has to be.

This is what Willingham has to say about the need for effortful study or practice:

Some evidence that a great deal of practice, and not just talent, is a prerequisite for expertise is the "ten year rule," which states that individuals must practice intensively for at least 10 years before they are ready to make a substantive contribution to their field. What about prodigies like Mozart, who began composing at the age of six? Prodigies are very advanced for their age, but their contributions to their respective fields as children are widely considered to be ordinary. It is not until they are older (and have practiced more) that they achieve the works for which they are known.

How are such studies relevant to the average student? Few students will become a Mozart, Shakespeare, or Einstein, but if we want children to understand and appreciate excellence, we would do well to send the message that excellence requires sustained practice. The athletes and artists revered by many students excel not solely by virtue of their talent, but because of their hard work. Edison remarked that "genius is one percent inspiration and ninety-nine percent perspiration." The relative percentages of talent and practice are unclear, but the necessity of long periods of focused practice to exploit inborn talent is not.

Turning our attention to academics, Willingham, gives us the general areas in which students need to focus their practice on:

[S]ustained practice over time is especially useful for developing automaticity in specific skills (which enables higher-level thinking) and in ensuring that a memory lasts as long as needed. Thus, the following types of material are worthy of practice:

1. The core skills and knowledge that will be used again and again. In this case, we give practice in order to ensure automaticity. The student who struggles to remember the rules of punctuation and usage (or must stop to look them up in a reference book) cannot devote sufficient working memory resources to building a compelling argument in his or her writing. The student who does not have simple math facts at his or her disposal will struggle with higher math.

2. The type of knowledge that students need to know well in the short term to enable long-term retention of key concepts. In this case, short-term overlearning is merited. For example, as noted earlier, a science teacher may want students to know a set of facts about certain species so that she can introduce an important abstract concept concerning evolution that depends on these facts. Or, a high school history teacher may want students to master the facts of several Supreme Court cases in order to build long-term understanding of a particular constitutional principle.

3. The type of knowledge we believe is important enough that students should remember it later in life. In this case, one might consider certain material so vital to an education that it is worthy of sustained practice over many years to assure that students remember it all of their life. A science teacher might spend the better part of a year emphasizing basic principles of evolution in the belief that the material is essential to consider oneself conversant in biology. Further, the curriculum might address and require practice in evolution in multiple years to assure that such knowledge will last a lifetime. Do we expect that a 40-year-old will have retained everything learned through the 12th grade? No, but do we expect that she will retain anything? Should she be able to grasp the basics of evolution or describe the different responsibilities of the three branches of the federal government or calculate the area of a circle? Exactly what sorts of knowledge merit the focus required to create long-lasting memory will be controversial, but that practice is required to create such memories is not.

It is this effortful study or sustained practice that is so critical to student's creation of the vast storehouse of knowledge that will be used to chunk critical concepts and automatically recall critical facts quickly, thus, getting around the limitations of working memory and enabling one to acquire those critical higher-order thinking skills which are so important.

Educators think these higher-order skills can be taught directly. Sorry, they can't. One way or another the student will have to acquire all the basic skills along the way to enlightenment. There is no golden road to learning. The only way to real learning is paved with a lot of hard work.

In Part IV we'll discuss why some instructional programs are better than others. Hint: they focus on effortful practice, build motivation in, and are structured to guarantee student success.

The First Battle Has Finally Been Won but the War is Far from Over

After seventeen years or crippling children's math skills with nonsensical "standards" that had absolutely no research base, the National Council of Teacher's of Mathematics (NCTM) has finally seen the light and issued currciulum guidelines for PreK to grade 8: Curriculum Focal Points : A Quest for Coherence.

The subtitle says it all, don't you think?

The WSJ is already on the case. Some excerpts.

The nation's math teachers, on the front lines of a 17-year curriculum war, are getting some new marching orders: Make sure students learn the basics.

That's because learning the basics is the only reliable way to teach children how to do math. Whether you like it or not, before kids can do more advanced math they need to know how to do basic math. No, strike that. They need to know basic math so well it is automatic for them. That's one of the many things that the NCTM's previous standards so badly lacked.

In a report to be released today, the National Council of Teachers of Mathematics, which represents 100,000 educators from prekindergarten through college, will give ammunition to traditionalists who believe schools should focus heavily and early on teaching such fundamentals as multiplication tables and long division.

Shame on you WSJ. It's not just a "belief" at this point. It's been proven time and time again in the research base. It's not a debate between opposing viewpoints at this point. It's a debate between crackpots who have nothing but a legacy of failure and no research base and those that do. Pardon the ad hominems.

Here's a good example of that failure in the very next paragraph:

The council's advice is striking because in 1989 it touched off the so-called math wars by promoting open-ended problem solving over drilling. Back then, it recommended that students as young as those in kindergarten use calculators in class.

Downplaying practice and advocating the use of calculators in K before student's have mastered the basics would have been stupid. Nonetheless, our faddish no-nothing educators and textbook writers jumped feet first onto the NCTM bandwagon. You'd be hard pressed to find a decent textbook in widespread use nowadays that is based on learning math basics to mastery.

Those recommendations horrified many educators, especially college math professors alarmed by a rising tide of freshmen needing remediation. The council's 1989 report influenced textbooks and led to what are commonly called "reform math" programs, which are used in school systems across the country.

Did you think I was lying?

Problem was, it didn't horrify enough math educators below the college level. Go to almost any school's website and you will see how they've been fawning over the NCTM's standards for years. Let's see how many admit they were wrong after today's NCTM reversal.

Infuriated parents dubbed it "fuzzy math" and launched a countermovement. The council says its earlier views had been widely misunderstood and were never intended to excuse students from learning multiplication tables and other fundamentals.

Mostly parents who rely on math to make a living, no doubt. But, these parents didn't know anything according to sneering educators because the almighty NCTM, who've never successfully educated any kid, said they didn't know how to teach math.

If states adopt the new standards and teachers adjust their methods, "we'll be more competitive," says Prof. Fennell, who teaches at McDaniel College in Westminster, Md.

And, there goes the big IF. The big IF that'll likely sabotage the whole thing. Between the children and a math education stands a large ominous obstacle. That obstacle is our math educators who don't know how to teach math effectively today and need to "adjust their methods" to conform to the new standards. There will be much gnashing of teeth, pulling of hair, and other forms of adult tantrums along the way I assure you. Witness their reaction to NCLB and the changes it requires.

According to their report, "Curriculum Focal Points," which is subtitled "A Quest for Coherence," students, by second grade, should "develop quick recall of basic addition facts and related subtraction facts." By fourth grade, the report says, students should be fluent with "multiplication and division facts" and should start working with decimals and fractions. By fifth, they should know the "standard algorithm" for division -- in other words, long division -- and should start adding and subtracting decimals and fractions. By sixth grade, students should be moving on to multiplication and division of fractions and decimals. By seventh and eighth grades, they should use algebra to solve linear equations.

DUH! To think that this is in any way controversial boggles the mind.

A significant problem remains. Setting coherent standards is one thing-- a necessary first step if you will. Actually, getting that information into the heads of children, novices in math, is quite a different story.

Here's the dirty little secret in math instruction: constructivist pedagogy may have been patently silly, but traditional math pedagogy wasn't much more successful. Sure, using a rigorous traditional math program will help the top 25% of the class. But, what about the bottom 75%, the kids that didn't seem to learn math all that well in traditional programs either? It's one thing to mandate that kids learn "multiplication and division of fractions and decimals" it's another to actually teach it so that 99% of the kids can do it. There are few math programs that are capable of doing that.

Supporters of the council's previous views worry that the new report may lead to a return to the kind of rote learning they say left many children without any understanding of concepts. They say few adults spend much time doing long division, and students are better served getting a grounding in real-life problem solving.

Jackasses. I can't begin to count the ways that those two sentence are wrong on so many levels. First, they don't know the difference between rote learning and inflexible knowledge. Second, no one teaches by rote in any program --they don't even understand what rote learning really is. Third, for all their blather they did an horrendous job teaching "understanding of concepts" since most of their students didn't understand even basic math as demonstrated by their inability to solve basic math problems. Fourth, the ability to perform the long division algorithm accurately demonstrates that the child has become automatic in many basic math skills that are critical to performing well in algebra and engaging in "real-life problem solving."

Those two sentences are a good litmus test. If the math educator teaching your child math agrees with them, your child is being mistaught math. Don't wait to take action.

"The risk is that we end up with students who have no idea what math is all about or how to use it," says Joseph Rosenstein, a math professor at Rutgers University in New Jersey who reviewed the new guidelines.

As it turns out, that risk is the reality today under the NCTM's old standards. So pot kettle black, Prof Rothstein. It's easy to determine if kids know what math is all about or know how to use it: See how well they can solve math problems. The ones who can solve the most math problems, by definition, know what math is all about and know how to use math.

The war is far from over. Look out for "balanced math" to come to a classroom near you; it'll be the latest way our math educators will attempt to salvage their desire to keep doing things as they've been doing them. Mark my words.

And thus we begin

My son, who just started first grade, received his first homework assignment last night. This was day five.

The first part of the homework assignment consisted of a handwriting worksheet for practicing writing the letter "a." Based on a completed worksheet he had brought home, I surmised that he had already been instructed how to write the letter "a" in class, so this homework assignment was just additional practice. This is what homework should be about.

But then we get to the second part of the homework assignment. He was to "read for twenty minutes." He does know how to read because we've been teaching him at home. Many kids in his class, however, do not yet know how to read because -- wait for it -- the school has not yet begun teaching them how to read.

Talk about that first step being a doozy. The first step assumes you already know how to read.

So, for most kids, this homework assignment represented 20 minutes worth of wasted time. (Unless, of course, you are working under the mistaken belief that kids just pick-up reading "naturally" by being around books.)

How many nights of being forced to "read" before you've been taught how to read do you think it takes before a kid decides he doesn't want to read any more? How many days in class being humiliated in front of your peers do you think it takes before kids shut down when it turns out they aren't picking-up reading naturally?

And, thus, we take our first step down the road to disengagement from school and loss of motivation.

Just remember it's the kids' fault. Their parents' too. Their bad environment too. It's not the school's fault though. It's never the school's fault.

Update: My bad. He was only supposed to read for ten minutes.

My 100th Post

It's official.

100 posts
1277 paragraphs
72,814 words
Countless typos

This is truly Horrifying

Blogger TMAO tells us abouthis new seventh grade class. The lack of basic skills possessed by these kids is truly horrifying.

20% of my students earned a zero percent on my parts of speech diagnostic (Ex: write the definition of a noun; which of the following is a verb, etc.) 100% failed.

35% of students earned below ten percent on a writing diagnostic assessing specific writing skills that, according to a Board of Trustees presentation I recently attended, are being taught in classrooms across the District. 100% earned a D or below.

The average independent reading level is 2.5 -- that's second grade, fifth month, for those scoring at home. (If only we used Open Court and Reading First...)

The average fluency score is 83 words per minute, nearly 100 words below benchmark.

33% of students failed the alphabet quiz, which asks students to print each letter in capital and lowercase letters, and then circle the vowels.

Then there's this, from a student who's been in the U.S. four years, responding to the persuasive essay diagnostic about lengthening the school day: "I tenk the es net god to go in the dark. I cot get hor en yu ren gen the en mi opor non of os to get or. vi car cut hat os en the dark en mor es uy pas the estuits."

My understanding is that many of these kids have been in the public school system since K.

Let's see six years of elementary school--years when these kids were easily controlled and could be motivated to learn if properly taught.

In 2006, there is absolutely no reason for kids to know so little after seven years of schooling.

All of you pro-public education people need to reconcile this abysmal performance with your rosy rhetoric before I can take you seriously.

Let's Get Ready to RUMBLE

The AFTies call out Kevin Carey for making the recent claim that:

"...Rothstein's frequent assertions to the contrary--NCLB is not based on the premise that good schools can erase the achievement gap. A school can make AYP under NCLB and still have huge achievement gaps, as long it gets all students over that minimum standard."

The AFTies respond by quoting from the NCLB statute:

Where could Rothstein have gotten the idea that NCLB was an attempt to close the achievement gap? Maybe from the law itself. Here's what's at the top of the first page of The No Child Left Behind Act, as it was passed by Congress and signed by the President:

Public Law 107–110
107th Congress

An Act

To close the achievement gap with accountability, flexibility, and choice, so that

no child is left behind.

Carey makes a good point about the difference between closing the achievement gap and equalizing proficiency levels, but, given the actual text of the law, it's hard to dismiss Rothstein's claim that lawmakers created NCLB to close the achievement gap.

The AFTies delivered some good body blows there and almost smelled victory for a moment when they went and found the phrase "close the achievement gap" in the text of the statute, but then they went and snatched defeat from the jaws of victory by stating that "but, given the actual text of the law, it's hard to dismiss Rothstein's claim that lawmakers created NCLB to close the achievement gap."

Actually, it is easy to dismiss Rothstein's claim depending upon how "achievement gap" is defined in the statute. Kevin realized this and counterpunched:

AFTie John can't figure out the difference between the title of a law and what the law actually says.

[T]he phrase "close the achievement gap" can mean equally legitimate but very different things. It can mean "erase all academic performance differences between poor and non-poor students," or it can mean "make sure that both poor and non-poor students reach a defined (and in most states, not particularly high) level of achievement."

Since Congress chose the latter definition when the wrote the actual provisions of NCLB, it seems safe to assume that they also had that definition in mind when they referred to closing the achievement gap in the title.

Kevin managed to damage the AFTies badly, but isn't able to deliver the knockout blow. First, the term "achievement gap" is used no less than a dozen times in the text of the statute, not just the title. Second, Kevin should have provided textual support for his definition of "close the achievement gap" because he left himself open to this withering attack from Sherman "Crusher" Dorn:

In a word, dear readers, this is baloney (technical educational policy term). The mechanisms of AYP notwithstanding, rhetoric about NCLB has consistently been about the achievement gap. I'm not sure if Carey is echoing Fordham's Michael Petrilli (see my earlier entry on that), but when anyone starts to lowball expectations, it's, well, it's, well, ... soft bigotry? I'll avoid the purple prose and just note that defenders of the AYP mechanism and high-stakes testing are engaging in this rhetorical dance because NCLB and most accountability frameworks avoid concrete discussions of standards. We must have them, proficiency must be defined, but your everyday Joe wouldn't know what that means. Heck, I don't.

I'm not sure if we're headed towards the worst possible outcome of oversold education reforms (regression towards deterministic views of human capacity), but when defenders of high-stakes accountability start backpedaling as fast as they can from the rhetorical framework that's been the political underpinnings of NCLB, it's not good news.

That's the equivalent of a rhetorical roundhouse from Sherman who was clearly going for the knockout punch. Unfortunately Sherman misses the mark for the same reason Kevin did--failing to determine how the statute defines "achievement gap."

So why don't we go ahead and do that so we can declare a winner.

Right out of the box, we run into trouble because we quickly discover that Congress did not explicitly define "achievement gap." All we have to go on is references like "narrowing achievement gaps in accordance with section 1111(b)" throughout the text. So, clearly, we need to go to section 1111 to see if it sheds some light.

Section 1111 deals with "State Plans." Each State that desires NCLB funding must submit a plan (1111(a)) that will adopted challenging academic content standards (defined in 1111(b)(1)(D)(i)) and challenging student academic achievement standards (defined in 1111(b)(1)(D)(ii))to be used by the state to carry out the law.

The challenging academic content standards must be "in academic subjects that--(I) specify what children are expected to know and be able to do; (II) contain coherent and rigorous content; and (III) encourage the teaching of advanced skills."

The challenging student academic achievement standards must be (I) [] aligned with the State’s academic content standards; (II) describe two levels of high achievement (proficient and advanced) that determine how well children are mastering the material in the State academic content standards; and (III) describe a third level of achievement (basic) to provide complete information about the progress of the lower-achieving children toward mastering the proficient and advanced levels of achievement.

Note that the states must set both a proficient and an advanced standard to determine if students are mastering the academic content standards in addition to a basic standard to provide information about students who haven't yet met the proficient or advanced standards.

Now we can turn our attention to the Accountability system (defined in 1111(b)(2)) which must be enacted by each State to ensure that the state is making "adequate yearly progress." Now we're getting to the good stuff.

AYP is descibed in section 1111(b)(2)(B):

Each State plan shall demonstrate, based on academic assessments described in paragraph (3), and in accordance with this paragraph, what constitutes adequate yearly progress of the State ... toward enabling all public elementary school and secondary school students to meet the State’s student academic achievement standards, while working toward the goal of narrowing the achievement gaps in the State ...

So the states are supposed to be making adequate yearly progress (AYP) toward meeting the academic achievement standards and working toward the goal of narrowing the achievement gaps. So, as Sherman has already identified (but curtly dismissed), the notion of "narrowing the achievement gaps" is tied up in the notion of AYP. There are no separate standards for determine whether the achievement gaps have been narrowed. Presumably, if States are making AYP, they should also be narrowing the achievement gap. Let's see if this is in fact the case. To do that we need to take a look at AYP.

AYP is defined in 1111(b)(2)(C) and is intended to measure the "progress of public elementary schools, secondary schools and local educational agencies and the State based primarily on the academic assessments" (section 1111(b)(2)(C)(iv)). To effect the AYP, States are required to establish a starting point (section 1111(b)(2)(E)), a timeline (section 1111(b)(2)(F)), and separate measurable annual objectives (section 1111(b)(G)) for continuous and substantial improvement.

Starting Point. Section 1111(b)(2)(E):

Each State, using data for the 2001–2002 school year, shall establish the starting point for measuring, under subparagraphs (G) and (H), the percentage of students meeting or exceeding the State’s proficient level of academic achievement on the State assessments under paragraph (3) and pursuant to the timeline described in subparagraph (F).

Timeline. Section 1111(b)(2)(F):

Each State shall establish a timeline for adequate yearly progress. The timeline shall ensure that not later than 12 years after the end of the 2001–2002 school year, all students ... will meet or exceed the State’s proficient level of academic achievement on the State assessments under paragraph (3).

Measurable Objectives. Section 1111(b)(2)(G):

Each State shall establish statewide annual measurable objectives ... for meeting the requirements of this paragraph, and which ... (iii) shall identify a single minimum percentage of students who are required to meet or exceed the proficient level on the academic assessments ... and (iv) shall ensure that all students will meet or exceed the State’s proficient level of academic achievement on the State assessments within the State’s timeline under subparagraph (F)

So what does all this mean?

The primary criterion for determining if States are making AYP and satisfying the requirements of NCLB is determining whether students are meeting or exceeding the proficient level that they've set for themselves. Students are only required to meet or exceed a single cut score. (The advanced level never enters into the equation.) If the necessary number of students meet this criterion, then they've complied with NCLB. This mechanism is the political comprise that Congress settled on.

All that's left to do is to determine if the achievement gap will necessarily be narrowed as more and more students become proficient. The answer is yes it will. Let's take a look at a few graphs I blatantly stole borrowed from La Griffe.
The top distribution represents white student academic achievement. The bottom distribution represents black student academic achievement. Presently, blacks perform about a standard deviation below whites. The result is an achievement gap between the two groups. But the achievement gap itself is a statistical artifact which varies depending upon where the proficiency cut-score is located.

For example, if we put the cut-score at passing level 1 as shown in the above graph then the white failure rate will be 16% and the black failure rate is 50%, leaving us with a 34 point achievement gap. Now let's change the cut-score at passing level 2 , the white failure rate becomes 2% and the black pass rate becomes 16%, leaving us with only a 14 point achievement gap. Got that? Now take a look at this graph of how the achievement gap varies depending upon white pass rate.


So as we start to approach a point in which either all white students are either passing or failing the assessments, the achievement gap narrows until it practically disappears. Of course blacks will still be performing at about a standard deviation below whites, but the differential will be masked as long as either most students are passing or failing the assessment instruments.

Another way of saying this is that a combination of fiddling with cut scores and raising student achievement across the board will increase the number of students who are "proficient," thus narrowing the achievement gap. That's all that NCLB requires.

So it would appear that Kevin Carey holds on and wins the fight.

I disagree with Sherman that Kevin has backpedaled or otherwise lowballed expectations. NCLB says what it says, overheated politcal bloviating notwithstanding. In adition, as I've pointed out, increasing student achievement across the board will in fact narrow the achievement gap, assuming that States set the cut-scores sufficiently low.

No Truer Words have Ever Been Written

In response to this inane remark made by the principal of High Tech High:

"It's not about memorizing certain algebraic equations and then regurgitating them in a test," Grover said. "It's about thinking how math might be used to solve a quality-of-water problem or how it might be used to determine whether or not we are safe in Philadelphia from the avian flu."

Rightwingprof responds:

Of course, what this brain-dead principal misses — what all educrats who spout the "higher level thinking" line miss — is that you can't do those things until you've memorized those algebraic equations and regurgitated them on a test. What part of that is so hard for these idiots to understand?

Here's the part I don't understand. If the kids were really developing super higher-order thinking brains with these new-fangled progressive teaching techniques why are they unable to use those super brains to solve algebraic equations on a simplistic multiple choice exam? Why are they unable to higher-order think their way to the correct answers on lower-order basic skills exams?

And, while you're answering those questions, please point me to one credible scrap of evidence that indicates that the kids who've supposedly gotten their higher-order skills honed, but who don't do well on basic skills test, actually have acquired the higher-order skills and can demonstrate proficiency on a higher-order skills test.

It'll All End in Tears

The Philly School System has opened up High Tech High reports the Philly Inquirer, calling it the "School of the Future." I hope not.

Let's run through the list of things that High Tech High will bring to the table of the future:

• A laptop for every child. (They will be traded in for computerized tablets soon.)
• Smart cards that track student movement throughout the school.
• Virtual teaching assistants.
• Software that will allow parents to track students' progress from home.
• Lockers that open with the swipe of a smart card.
• A fully wireless building.
• Virtually no textbooks.
• Plasma screens & Plasma Boards.
• Ceiling projectors.
• Interactive white boards.
• Classroom furniture is on wheels to allow for group work in varying configurations.
• Students will be required to apply to at least one college. (huh?)
• A later start. School will begin at 9:15 a.m., acting on research that says teens think better a little later in the day.
• Photovoltaic panels in the windows and roof will convert sunlight into electricity.
• The building also will catch rainwater and convert it for non-potable uses, such as toilet and boiler water.
• The school also has established partnerships with local universities.
• Teachers were hand-picked by the principal.
• "They have those sinks that you just put your hands like that and the water comes out."
• "Mirrors for girls."
• "Toilets flush by themselves. It's all just so nice."
  

Yes, it does sound all so nice. Certainly with the $13k the Philly school district squanders on each student each and every year, every school should have such amenities.

There's only one little thing they forgot to upgrade:

The curriculum is traditional academic with a focus on projects and interdisciplinary learning.

So all those improvements will amount to little more than rearranging the deck chairs on the Titanic because they're still going to be teaching the same old crap. And, last I checked, the traditional high school curriculum hasn't worked all that well with the typical skill set that this population of kids will have. And, the "focus on projects" and "interdiscliplinary learning" are, at best, neutral when it comes to academic success. If anything, they tend to drive academic performance down because they complicate and throw unecessary variables into the learning process.

In case you were wondering here are the projected demographics of the school.

Racial breakdown of student body: 95 percent African American.

Gender: 54 percent female, 46 percent male

Economic background: 85 percent come from low-income families.

Special-education population: 12 percent

Entrance requirements: 75 percent of students are from the neighborhood and 25 percent from the rest of the city. Students currently in eighth grade must apply by Nov. 17 to the district for entrance in the 2007-08 school year. Enrollment is determined by lottery.

Mark my words, High Tech High will turn out to be an expensive failure that peforms no better than other similarly situated high schools. It'll all end in tears.

Update: Rightwingprof comments on High Tech High and deconstructs the principal's silly comments in this Reuters article. That principal will certainly be an albatross around the neck of the students. It'll take a lot more than plasma white boards to compsensate for the edunonsense being peddled at High Tech High. Bill gates really needs to learn a thing or two about education before pissing his and Microsoft's money away on this crap.

Mayor Street has already all but given High Tech High the kiss of death:

"You won't be able to say, 'I didn't have the computers. I didn't have the technology. I didn't have the teachers. I didn't have mentors,' because the young people who go to this school will be in the premier educational environment in the entire country, maybe even in the entire world," Street said. "So the bar for you is raised."

These poor kids. High Tech High is going to be one of the most embarrassing education experiments we've ever seen. You can take that to the bank.

This Scares Me

The story so far...

Jay Matthews argues in favor of national standards using the wide disparity between students labeled as proficient in the NAEP and in various state tests:

Maryland recently reported that 82 percent of fourth-graders scored proficient or better in reading on the state's test. The latest data from the National Assessment of Educational Progress, known as "the nation's report card," show 32 percent of Maryland fourth-graders at or above proficiency in reading.

Virginia announced last week that 86 percent of fourth-graders reached that level on its reading test, but the NAEP data show 37 percent at or above proficiency.

Sherman Dorn helpfully reminds Matthews that cut-score setting is not the same thing as standards setting. And further adds:

[A]ny attempt to use a test to "set standards" is getting things backwards. Don't we first decide what we want students to do?

and

So if there were a national test every child takes, I predict that there would be a yawning gap between the test and any sense of real standards or expectations.

This is, of course, a sucker's bet.

The current NAEP already reflects a yawning gap between the test and any sense of real standards or expectations.

Tom Loveless of the Brookings Institute recently took a look at the math portion of the NAEP. He coded each released arithmetic item according to Singapore's (who leads the world in math achievement) scope and sequence for its math program and determined how well kids answered the problems in the 8th grade NAEP. Here's the results:

Grade Level	% of Questions	% Answered Correctly
1st	16.3%	54.0%
2nd	23.3%	45.4%
3rd	18.6%	41.4%
4th	18.6%	32.6%
5th	13.9%	38.5%
6th	2.3%	NA
7th	7.0%	27.7%

So, for example, in the 8th grade NAEP 16.3% of the test questions were at a 1st grade level and yet only 54% of the 8th graders taking the exam answered the question correctly.

This is what Loveless concludes based on his analysis of both the 4th and 8th grade NAEP tests:

A couple of things stand out in the fourth grade portion of Table 1-3. First, the problem solving items on NAEP are not very challenging—at least not in the arithmetic required to answer them. Content taught in first and second grades is at least two years below grade level for fourth graders. But that is the level of difficulty of more than four out of ten (43.6%) problem solving items on NAEP. The second surprising finding is that even though the NAEP items are so easy,
fourth graders do not do very well on them. The first and second grade items demand nothing more than being able to add and subtract whole numbers and knowing basic multiplication facts. Yet a majority of the nationÂ’s fourth graders miss the average item pitched at this level.

Even more dramatic findings are evident at eighth grade. The eighth grade items are only slightly more difficult than those on the fourth grade test (3.4 mean grade level). Almost four out of ten items (39.6%) address arithmetic skills taught in first and second grade—six years below the grade level of eighth graders taking the test. Indeed, more than three-fourths of the items (33/43) are at least four years below grade level, taught in the fourth grade or lower. Yet the percentage of eighth graders answering problem solving items correctly is an unimpressive 41.4%. Problem solving items on the eighth grade NAEP only require knowledge of very simple arithmetic. Despite this, eighth graders have trouble getting them right.

Loveless then analyzes the "algebra" strand of problems and finds them also lacking.

Despite the simpler arithmetic on algebra items, fewer students answer the algebra questions correctly than the number sense questions, suggesting that some of what has been discovered here may be because of test design. It is clear that the algebra items are assessing something other than arithmetic. One assumes that the something else is algebraic—indeed it seems to be quite challenging to most eighth graders. Nevertheless, really knowing algebra means being able to solve equations that contain more sophisticated forms of numbers than whole numbers. Anything less challenging is appropriating the term “algebra” to convey a false sense of rigor to a pool of test items.

It would appear that Matthews is barking up the wrong tree if he thinks that the NAEP is the gold standard or that national testing is the answer to anything.

What scares me though is not where the Board of Governors has set the cut scores for NAEP, but where the states are setting their cut scores (some 50 points higher). At least the NAEP cut scores accurately show that most students can't still can't add their way out of a paper bag.

It also makes me wonder why teachers and public education apologists raise such a big fuss over standardized testing. Just look how low the standards really are. Imagine if we raised the standards to accurately reflect what students really need to know to succeed academically.

Knowledge is Good

In the last post, Schools Don't Create Experts, we discussed how experts rely on a vast storehouse of intricately-structured previously-acquired relevant knowledge more than their analytic abilities. The reason is that having considerable domain knowledge allows the expert to chunk information, thus getting around the limitations of working memory.

Think of chunking as the ability to mentally package information using pre-existing background knowledge for easy recall. Chunking is useful because the human mind can only keep and manipulate about seven things in working memory. Chunking allows you to get around this biological limitation.

Here's an example from Willingham of chunking in action.

[R]ead through this list one time, then look away and see how many of the letters you can recall.

CN

NFB

ICB

SCI

ANC

AA

There were 16 letters on the list, and most people can recall around seven—there is not sufficient space in working memory to maintain more than that. Now try the same task again with this list.

CNN

FBI

CBS

CIA

NCAA

Much easier, right? If you compare the two lists, you will see that they actually contain the same letters. The second list has been reorganized in a way that encourages you to treat C, N, and N as a single unit, rather than as three separate letters.

Chunking is a useful tool, but, as you can see, chunking requires you to have background knowledge in order to activate. In the example above, you need to know what the FBI, CNN, CBS, CIA and NCAA are in order to chunk the letters.

Another critical use for background knowledge is for making inferences and connections while reading text. Here's an example from Hirsch:

Cognitive psychologists have determined that when a text is being understood, the reader (or listener) is filling in a lot of the unstated connections between the words to create an imagined "situation model" based on domain-specific knowledge. This situation model constitutes the understood meaning of the text. Take, for example, this passage from my book What Your Second-Grader Needs to Know:

In 1861, the Civil War started. It lasted until 1865. It was American against American, North against South. The Southerners called Northerners “Yankees.” Northerners called Southerners “Rebels,” or “Rebs” for short. General Robert E. Lee was in charge of the Southern army. General Ulysses S. Grant was in charge of the Northern army.

Potentially, this passage is usefully informative to a second-grader learning about the Civil War—but only if he or she already understands much of what’s addressed in it. Take the phrase "North against South." A wealth of preexisting background information is needed to understand that simple phrase--going far beyond the root idea of compass directions, which is simply the necessary first step. The child needs a general idea of the geography of the U.S. and needs to infer that the named compass directions stand for geographical regions. Then a further inference or construction is needed: The child has to understand that the names of geographical regions stand for the populations of those regions and that those populations have been organized into some sort of collectivity so they can raise armies. That’s just an initial stab at unpacking what the child must infer to understand the phrase “North against South.” A full, explicit account of the taken-for-granted knowledge that someone would need to construct a situation model for this passage would take many pages of analysis.

Of course, there's much more to it than that and I suggest you , if you haven't done so yet, read all the articles I've linked to. Let me try to oversimplify. You need to know a lot of stuff in order to quicly learn more stuff, quickly process and analyze the new stuff, remember the new stuff more easily, helps you solve problems, and allows you to think lesss (by not having to derive everything).

The trick is, of course, getting all that knowledge into your head in the first place. This'll be the topic of my next post, but I'll leave you with this teaser. Schools do a miserable job at teaching facts, if anything they deride the entire process and consciously avoid teaching facts in favor of teaching students how to "learn how to learn," which you know, if you've read the articles I linked to, is a crock.

Schools Don't Create Experts

There is a great article, The Expert Mind, over at Scientific American. Go read the whole thing right now. Then we'll discuss.

Our schools have many problems, but their main problem-- the root problem if you will, is their inability to teach a large percentage of students. And, of the students that they do manage to teach, they teach incorrectly.

This article explains why in layman's' terms. So let's get to it.

But how do the experts in these various subjects acquire their extraordinary skills? How much can be credited to innate talent and how much to intensive training? Psychologists have sought answers in studies of chess masters. The collected results of a century of such research have led to new theories explaining how the mind organizes and retrieves information. What is more, this research may have important implications for educators. Perhaps the same techniques used by chess players to hone their skills could be applied in the classroom to teach reading, writing and arithmetic.

...

Without a demonstrably immense superiority in skill over the novice, there can be no true experts, only laypeople with imposing credentials. Such, alas, are all too common. Rigorous studies in the past two decades have shown that professional stock pickers invest no more successfully than amateurs, that noted connoisseurs distinguish wines hardly better than yokels, and that highly credentialed psychiatric therapists help patients no more than colleagues with less advanced degrees. And even when expertise undoubtedly exists--as in, say, teaching or business management--it is often hard to measure, let alone explain.

I don't think I need to point out that our schools are more concerned with cranking out credentialled students rather than expert students. To create a credentialled student all you need to do is keep him in his seat for 13 years and have him pass some minimal skills test periodically, at the end of the day the student might have the skills of a novice. To create an expert student you need to teach him so that he can demonstrate superior skills by the end of school.

What's the difference between an expert and a novice? Experts have a vast storehouse of intricately structured knowledge at their disposal, novices do not. More importantly, the expert's analytic abilities are not necessarily superior to those of the novice. This is not to say that the expert doesn't use his analytic abilities to develop his structured knowledge (as I'll explain later). This is a critical distinction and one completely lost on our educators who focus on developing analytic skills (which are largely immutable) and neglect developing knowledge which they deride as mere facts.

In a famous experiment Dutch psychologist Adriaan de Groot studied the ability of chess players to recreate chess piece positions from memory.

De Groot also had his subjects examine a position for a limited period and then try to reconstruct it from memory. Performance at this task tracked game-playing strength all the way from novice to grandmaster. Beginners could not recall more than a very few details of the position, even after having examined it for 30 seconds, whereas grandmasters could usually get it perfectly, even if they had perused it for only a few seconds. This difference tracks a particular form of memory, specific to the kind of chess positions that commonly occur in play. The specific memory must be the result of training, because grandmasters do no better than others in general tests of memory.

Similar results have been demonstrated in bridge players (who can remember cards played in many games), computer programmers (who can reconstruct masses of computer code) and musicians (who can recall long snatches of music). Indeed, such a memory for the subject matter of a particular field is a standard test for the existence of expertise.

Guess what happened when De Groot set-up the chess boards randomly instead of from actual game play? The experts performed no better than novices.

But what about the analytic ability of the chess players?

Recent research has shown that de Groot's findings reflected in part the nature of his chosen test positions. A position in which extensive, accurate calculation is critical will allow the grandmasters to show their stuff, as it were, and they will then search more deeply along the branching tree of possible moves than the amateur can hope to do. So, too, experienced physicists may on occasion examine more possibilities than physics students do. Yet in both cases, the expert relies not so much on an intrinsically stronger power of analysis as on a store of structured knowledge. When confronted with a difficult position, a weaker player may calculate for half an hour, often looking many moves ahead, yet miss the right continuation, whereas a grandmaster sees the move immediately, without consciously analyzing anything at all.

And in case you aren't convinced yet.

The conclusion that experts rely more on structured knowledge than on analysis is supported by a rare case study of an initially weak chess player, identified only by the initials D.H., who over the course of nine years rose to become one of Canada's leading masters by 1987. Neil Charness, professor of psychology at Florida State University, showed that despite the increase in the player's strength, he analyzed chess positions no more extensively than he had earlier, relying instead on a vastly improved knowledge of chess positions and associated strategies.

E.D. Hirsch, the developer of Core Knowledge, has been making this same point for years now:

You Can Always Look It Up -- Or Can You?

Not So Grand a Strategy

So has cognitive scientist Daniel Willingham:

Practice Makes Perfect--But Only If You Practice Beyond the Point of Perfection

How Knowledge Helps

I suggest reading all four articles to learn about chunking theory, short term working memory, the magic number seven (+/- two), and long term working memory. These are all critical elements that will help you understand how experts think.

We'll meet back up in part two and learn how experts are made.

Why I Blog

I am a simple man who enjoys simple pleasures.

Having one of your aptly-titled posts show up on the first page of google definitely qualifies as one of them -- especially when the search terms are the author's name and the title of the new book he's currently hawking.

Search: Alfie Kohn Homework

Rothstein Responds

Rothstein responds to the flood of criticism he received when the NY Times parroted his silly views. He backpedals away from his statement that schools can't do "much better" without complementary reform and then sets up a series of strawmen to be knocked down.

In short, given that, as Mr. Finn asserts, children's time influenced by families and communities exceeds the time they are influenced by schools "by a multiple of four or five," I am puzzled that he fails to agree that serious and successful efforts to substantially narrow the achievement gap must include social and economic policies to improve the circumstances of family and community life, as well as policies to improve the quality of schooling.

I suspect the reason he fails to agree is because it remains un unproven theory that "serious and successful efforts to substantially narrow the achievement gap must include social and economic policies to improve the circumstances of family and community life." There's no evidence that supports Rothstein's assertion. And, it's not for lack of research either.

Mr. Finn asserts that good schools are "powerful enough instruments to boost poor kids' achievement to an appreciably higher academic plane." Nobody - not I, nor anyone with whom I am familiar - disagrees with this assertion. But what is commonly argued (and the notion that I dispute) is not that good schools can boost the achievement of disadvantaged children to "an appreciably higher plane" but rather that such schools can "close the achievement gap;" i.e., produce achievement from lower class children that is approximately equal to the achievement of middle class children.

Using Rothstein own definition of "close the achievement gap; as "produc[ing] achievement from lower class children that is approximately equal to the achievement of middle class children," it is easy to show that Rothstein is wrong. There are educational interventions that have been proven to produce achievement from lower class children that is approximately equal to the achievement of middle class children.

Project Follow Through, the largest educational experiment in U.S. history, showed that we can raise the performance of your typical Title I school (performing at the 20th percentile) to the performance level of middle class children (about the 50th percentile).

Take a look at this graph of overall performance of Title I kids in the experiment. That looks like about 50th percentile performance to me. And, all it took was school reform.

Here's another tidbit we learned from Project Follow Through: "The increase[d] amounts of money, people, materials, health and dental care, and hot lunches did not cause gains in achievement. Becker (1978) observed that most Follow Through classrooms had two aides and an additional $350 per student, but most models did not show significant achievement gains."

So much for throwing money at the problem which is exactly what Rothstein wants to do.

Another way of thinking about the claim that good schools can "close the achievement gap" is that if all disadvantaged children attended good schools, and graduated, on average, with average middle class levels of achievement, the vast social inequalities that now pervade American society would disappear.

That's a strawman argument. We don't know of anything that is capable of eliminating the " vast social inequalities" that Rothstein has in mind. Unless, you count the social equality some unfortunate countries got under communist rule. I don't exactly see anyone making a mad dash toward that, especially the people who once lived under that system.

As I pointed out in this post, the "vast social inequalities" we have in the U.S. is not exactly the grinding poverty that Rothstein's trying to portray. We have too many fat poor people to take that argument seriously nowadays.

At present, for example, the average achievement of black and white children in America differs by about a full standard deviation, or about 30 percentile points in a distribution, on most standardized tests

Coincidentally, that also happens to be the magnitude of the IQ gap. That's the big elephant standing in the room now isn't it? IQ correlates highly with academic performance and to a lesser extent SES. All our efforts to raise IQ and through educational and/or SES interventions have been failures. Similarly, our efforts to raise academic performance through SES interventions have also been fruitless. You do the math.

Social scientists generally consider an intervention to be extraordinarily successful if it has an effect size of 0.5, or more than 15 percentile points. Such an impact of good schools would truly be extraordinary – my guess (without evidence) is that the best school reform, even including the extended school time that Mr. Finn advocates, might aspire to an effect size of 0.3, or about 10 percentile points.

That would have been a bad guess. In project Follow through the effect size was close to a full standard deviation. Subsequent research has validated those results.

Mr. Finn'’s claim makes sense only if we focus on average "poor kids' achievement." Any particular school, whether it is a typical or a "good" school, may have a larger than usual share of children who are above, or below average for all poor children. As the previous paragraph suggested, the variation in poor children's achievement is wide, as is the variation in middle class children's achievement, as is the variation in most human characteristics. Indeed, once you have controlled for major demographic factors, like race and poverty, there is more variation in within-school achievement than in average achievement between schools.

So what? An effective instructional intervention will be able to raise the tide of all boats. See here. At the end of the day we'll still see a bell curve with respect to student achievement. We'll also still see an achievement gap between certain groups. But since we will have shifted student achievement to the right by about a standard deviation, the sheer number of academic failures will be greatly diminished. And, that's about all we can ask of any school.

It would be useful and important to know whether such detailed comparisons of student performance can identify schools that substantially "beat the odds" and beat them to an extent that lifts students to middle class achievement levels. I know of no accounts of "beat the odds" schools that have attempted, or been able, to do this.

As long as effective schools remain islands in a sea of poor performers, the extent and number of schools that "beat the odds" will be scarce. This is due, at least in part, to the high mobility rate (upwards of 25%) we see in low SES neighborhoods. That means that most high performing schools are inundated with low-performing transfer students from low-performing schools who are in need of extensive remedation and who greatly contribute to the peer effect problems we see not to mention dragging down the performance of the entire school.

Nonetheless, just look at the performance of most any KIPP or DI school and you'll see what can be done.

The Rothstein article goes on and on from there, but its mostly a rehash of the same tired arguments. Project Follow Through demolishes almost all of his argument that isn't a strawman -- at least those parts of his argument that fall under the purvey of education.

You too can evaluate ed research

Rightwingprof has an excellent primer on understanding statistical analysis. If you add this article to the mix, you have about 90% of the tools you need to understand and evaluate ed research.

My handy rule of thumb: 90% of all ed research is crap.

Update: link fixed

Size Does Matter

Following up on my last post on reducing class size and why it is a load of bunk, I'll now explain why teaching small groups of students (at least at the elementary school level) is a necessary component in effective instructional programs.

The research tells us that by merely reducing class size, we can expect to see no more than about a 0.25 standard deviation increase in student performance. This is an educationally insignificant result. This unexpected outcome is the result, I believe, of ineffective teaching practices. If what is taking place in the classroom isn't working for the students, then it really doesn't matter whether there are 15, 25 or 50 students in the class. The result will be mostly the same -- students not learning.

But, what if we controlled for teaching effectiveness? What if we only look at highly effective instructional programs? Does class size make a difference in these programs?

The answer, unfortunately, is we don't know for sure. I don't know of any research on this exact point. But, I do know that in the realm of highly effective instructional programs, they all seem to rely on smaller class sizes as a component of the program.

For example, the two highest performing instructional programs, Success For All and Direct Instruction, both rely on small class sizes. I am more familiar with the Direct Instruction program, so I'll focus on that.

Let's first look at the research. Adams summarizes the DI research well:

[T]he meta-analysis shows that 17 studies lasted less than a year and 17 lasted over a year. The effect size can be calculated per comparison and per study but all of the results show large effect sizes: .95 for studies less than a year and .78 for studies more than a year.

[T]he age of the publications was analyzed (1972-–1980: 6 studies, 1981-–1990: 22 studies, 1991-–1996: 6 studies) and all of the effect sizes were large (.73, .87, 1.00, respectively).

I also analyzed the data 8 other ways: by type of student, age/grade of student, academic subjects, test,research design, teacher, fidelity checks, and country. No matter which way the data were analyzed, the results were consistent: implementing DI programs resulted in large effect size gains.

Fifteen of the studies were conducted by researchers who have been somehow connected with Direct Instruction. In contrast, the majority of the studies (18 studies) were conducted by non-DI-connected researchers. The effect size for studies by DI-connected researchers was .99--—a large effect size. The effect size for studies by non-DI-connected researchers was .76--—also a large effect size.

No matter how you slice and dice the data, the result is the same-- a large effect size of about 3/4 of a standard deviation to one full standard deviation. This is four times the effect size we get when merely reducing class size. To put this in perspective, if your typical title I school (performing at the 20th percentile) were to increase its performance by 0.83 standard deviations it would be performing at the 50th percentile (i.e., like an average classroom).

Small class sizes play an important role in DI classrooms. This is because it is necessary for the teacher to be able to identify student errors quickly and provide corrections immediately. Allow me to shift into teacher mode briefly and instead of explaining how this works, let me show you a movie instead. Go here, and click on the movie "How to Set Up a Reading Group Carefully."

Clearly a lot of attention is given to presenting the material to optimally sized groups of students. The rule of thumb is teach lower performers in groups of 8 or less. Higher performers can tolerate larger instructional groups.

Not only is size important, but so is the placement of the students in the class. The lowest performers are placed front and center where the teachers can keep a close eye on them. Higher performs are placed around the outer periphery since they need the least attention. Middle performers are placed in between these two groups.

In later grades more teaching is done to the entire class and the same type of class room set-up is maintained. The optimally sized classroom depends on the learning ability of the students and the teaching ability of the teacher. A good teacher teaching a bunch of high performing kids can tolerate a much bigger classroom than an inexperienced teacher teaching a bunch of low performers.

In both DI and SfA classrooms, close records are maintained by the schools and student progress is monitored constantly. If students are falling behind or a teacher is not performing up to standard, correction can be taken very quickly and a remedy provided. There is no waiting until the end of the school year to recognize that students weren't learning.

These observations only apply to the elementary school level. In later years students are supposed to take on increasing responsibility for their learning. This is made possible in the DI or SfA classroom because most kids will be performing at grade level if they've been through the program in elementary without having significant gaps in their knowledge base.

High schools generally do a decent enough job with students who have been prepared for high school level work. The problem we have to day is that few kids are really prepared for the rigors of high school.

So class size does make a difference in the earliest years of schooling, but teaching effectiveness is a necessary prerequisite for reductions in class size to make a difference in student performance.