Everyday Math is one of those constructivist "problem solving" based curricula that progressive educators like so much. It's currently in use in about 20% of all elementary schools.
It also doesn't work, as WWC has finally determined.
A preliminary note. The iron-clad D-ed Reckoning rule of education research has been verified yet again: ninety percent of all Ed research sucks.
In Everyday Math's case there were 61 "research" studies. 57 did not meet WWC's evidentiary standards. That means that 93% of the Everyday Math research sucked.
None of the research fully met the evidentiary standards. Only four of the studies met the evidentiary standards with reservations. These were quasi-experimental studies. Here are the results of those four quasi-experimental studies:
The Carroll (1998) study included 76 fifth-grade students in four classrooms from four school districts using Everyday Mathematics and a comparison group of 91 fifth-grade students in four classrooms from similar districts, matched on student demographics and geographical location. The intervention group had used Everyday Mathematics since kindergarten. The comparison group had used traditional basal mathematics texts at all previous grades.So the results from the first quasi-experimental study were not statistically significant. That's all you need to know. Let's move on.
The Carroll (1998) study reported a statistically significant positive effect of Everyday Mathematics on geometric knowledge. After accounting for pretest differences between Everyday Mathematics students and comparison students, the WWC determined that this finding was substantively important but not statistically significant. Based on this study finding, the WWC categorized the effect of Everyday Mathematics on geometric knowledge as being a substantively important positive effect
The Riordan and Noyce (2001) study included 3,781 fourth-grade students in 67 schools in Massachusetts using Everyday Mathematics and a comparison group of 5,102 fourth-grade students in 78 similar schools, matched on baseline mathematics achievement scores and student demographics. Forty-eight schools in the intervention group had implemented Everyday Mathematics for four or more years (early implementers), and 19 schools had implemented Everyday Mathematics for two or three years (later implementers). The comparison group used 15 different textbook programs representing the instructional norm in Massachusetts, with the most commonly used programs being those published by Addison-Wesley, Houghton-Mifflin, and Scott-Foresman.The Riordan and Noyce (2001) study reported a statistically significant positive effect of Everyday Mathematics on overall math achievement. Using school-level data provided by the authors, the WWC determined that this finding was statistically significant and substantively important for the 48 early-implementing schools. For the 19 later-implementing schools, however, the WWC determined the finding to be substantively important but not statistically significant. Based on this study finding, the WWC categorized Everyday Mathematics as having a statistically significant positive effect on overall math achievement for the 48 early-implementing schools and a substantively important positive effect for the 19 later-implementing schools.
This study was funded by the Noyce Foundation, one of the authors of the study, which also has a financial stake in Every Day math. So we have a quasi-experimental study that was conducted by a potentially biased researcher. David Klein also noted the following defects in the study:
One of several shortcomings of [the Riordan/Noyce study] is that the schools studied are not identified. That makes it impossible to verify the results independently, thereby raising the possibility of fraud. This is a realistic possibility as the Noyce Foundation (headed by one of the authors of the study) has invested a lot of money in CMP, one of the programs found successful by the study. Clearly, that author has an interest in good results for the schools using the program she endorses. The editors of the journal should have asked for an independent confirmation of the methodology used to select both sets of schools and how they were matched before publishing the article. Instead, we get another example of "advocacy research." The comparison schools are constructed in a questionable way. The authors mix up all kinds of textbooks in the comparison groups--for half of which the authors report no curriculum program at all in the published article. No follow-up studies have ever appeared showing whether these schools maintained improvement and continued to improve in subsequent years of MCAS (2000, 2001, 2002), which would be easy to do since there were only about 20 or so schools in the experimental group.
The most telling evidence against the Riordan/Noyce study, however, is the fact that despite growing use of NCTM endorsed math programs (financed by millions of dollars from the NSF), percentages of kids in the top two categories on grade 4 and grade 8 on MCAS have been stable since 1998. For 5 years, there has been no discernable increase in the percent of kids moving into the two top categories, based on a test that matches the NCTM reform agenda.
Let's move on.
The Waite (2000) study included 732 third-, fourth-, and fifth-grade students in six schools using Everyday Mathematics and a comparison group of 2,704 third-, fourth-, and fifth-grade students in 12 similar schools, matched on baseline math achievement scores, student demographics, and geographical location. The schools in the intervention group were in their first year of implementing Everyday Mathematics. The comparison group used a more traditional mathematics curriculum approved by the school district.
The Waite (2001) study reported a statistically significant positive effect of Everyday Mathematics on overall math achievement. After accounting for the misalignment between the school as the unit of assignment and the student as the unit of analysis, the WWC determined that this finding was substantively important but not statistically significant. Based on this study finding, the WWC categorized the effect of Everyday Mathematics on overall math achievement as being a substantively important positive effect. The Waite study reported subtest results (concepts, operations, and problem solving). After WWC calculations, these results were found to be positive but not statistically significant. The subtest analyses do not factor into the rating.
Another statistically insignificant result. Let's move on.
So this small study had indeterminate results with statistically insignificant subtest results.The Woodward and Baxter (1997) study included 104 third-grade students in five classrooms in two schools using Everyday Mathematics and a comparison group of 101 third-grade students in four classrooms in one similar school, matched on student demographics and geographical location. The comparison group used the Heath Mathematics curriculum, a more traditional mathematics program.
The Woodward and Baxter (1997) study reported no significant effect of Everyday Mathematics on overall math achievement. After accounting for pretest differences between Everyday Mathematics students and comparison students, the WWC confirmed this finding. Based on this study finding, the WWC categorized the effect of Everyday Mathematics on overall math achievement as indeterminate. The study also reported subtest results (computation, concepts, and problem solving) and found a statistically significant positive effect on the concepts subtest. WWC calculations revealed a substantively important, but not statistically significant, positive effect for the concepts subtest and a substantively important, but not statistically significant, negative effect for the computations subtest. The subtest analyses do not factor into the rating.
The WWC generously conclusion:
"The WWC found Everyday Mathematics to have potentially positive effects on mathematics achievement"The average effect size was (0.31) or +12 percentile points. This represents a small effect size which is barely educationally significant.
That's certainly the equivalent of putting lipstick on a pig since those "potentially positive effects" come from three studies with statistically insignificant results and/or indeterminate results and one , and only, "study" with statistically significant results, came from a biased researcher with the research contained serious methodological defects.
You can put lipstick on a pig, but a pig is still a pig.
Read some critical reviews on Everyday Math at NY-Hold.
Today, Everyday Math is being used in about 20% of all elementary schools despite having no credible evidence of success. If your school district is using Everyday Math I'd suggest forwarding this WWC report to your school board and insist that someone's head roll.
30 comments:
Unfortunately, the same thing will happen with math as it did with reading. In response to the NCTM report, textbook publishers working in collusion with teaching schools and other educrats will develop "balanced math."
Guided math, silent math and working collaboratively with your peers will become all the rage.
They will call it balanced, but as with balanced literacy, only 5% or less will be the basics.
Since these cycles usually last about 15 years, we can expect a true return to instructivist math sometime around the year 2021.
I reviewed a paper for publication in a journal that used an alpha of 80%, I assume in order to get "statistically significant" results.
It did not get published. I dug in my heels and did not relent.
Even if I did forward the study to my son's school, nobody would get beyond "Everyday Mathematics was found to have potentially positive effects on students’ mathematics achievement." Voila -- it works!
My son is in a first grade gifted & talented program. Everyday Math and its ridiculous activities stink from top to bottom. Guess my kids are lucky to have a math teacher for a father.
nyc math teacher, I feel for you. My son just started in the TAG program. Its a pull out model with an art major as its teacher. The course description didn't even mention the word acceleration... quite frankly I will be surprised if they even do math. I expect it will be a bunch of feel good projects... more homework... no learning.
ha! HA! Vindicated at last!!!
I love Everyday Math. My sons have used it all through their elementary years. They're currently both in advanced math, and have much better overall understanding of math than my daughter did, who is older and had a different math program.
Thanks for the amusing anecdote, Jude.
I wonder whether the NCTM folks knew the Everyday Math research was going to be turning up on What Works.
Wish to heck they'd get something up on Math Trailblazers.
Jude,
Perhaps your sons have some natural aptitude and/or involved parents -- enough to overcome the silly "real-life" examinations in Everyday Math. Others, for whom there is no emphasis on scholarship at home, suffer greatly. I see it every day in many of my 6th graders who cannot make it in middle school math.
Even if I did forward the study to my son's school, nobody would get beyond "Everyday Mathematics was found to have potentially positive effects on students’ mathematics achievement." Voila -- it works!
I hear this same argument when it comes to the class-size research. Sorry, a 0.25 effect size is just not an educationaly significant result no matter how you slice it.
I am in a fifth grade mainstream classroom in NYC and I detest Everyday Math. There is no other way to say it. Just like the TC Reading and Writing Workshop, there is no availability for creativity and differentiated instruction for ESL or children with Special needs. The EM book is very disconnected and the experiments have nothing to do with the lesson. One day you're learning time and then next day it's geometry. No theme=no understanding.
OK, so remind me, what program did the WWC report works? Oh, they did not say that any program has evidence.
So, we have one with potential (EM) and all the others - well they have no, none, zero evidence.
Your turn
Lots of anonymous commentors lately, little substantive comments.
First, the WWC doesn't determine "what works." They merely examine the research, throw out that which isn't scientific, and evaluate what remains.
Second, the WWC hasn't completed their review of the elementary math program research yet.
Third, so far none of the math programs reviewed has much evidence of effectiveness, including EM.
What exactly is your point?
Let me spell it out for you.
If you all hate EM for various reasons, then tell us what you are for. Saxon? No evidence.
Singapore? No evidence.
The negative rhetoric does not help solve the amazing lack of mathematically understanding in today's world.
So what is your solution?
I do NOT see one post that gives a solution.
You talk of little substantive comments - well all of the above rhetoric is even less substantive. Do some research and get informed people.
Connecting Math Concepts by SRA. Research:
http://www.sanjuan.edu/accountability/program-evaluations/connecting-math-2002.pdf
https://www.sraonline.com/download/DI/Research/Mathematics/research_base_for%20di_math.pdf
http://darkwing.uoregon.edu/~adiep/math.htm
Dickey45 beat me to it, but CMC has enough evidence to fall within the WWC's highest category. And, that doesn't include all the Project Follow Through data.
And, BTW, EM has no evidence that it improves mathematical understanding.
The big one question then becomes why ISN'T CMC an option for teachers if it has data behind it?
Anyone check this site? It seems to indicate CMC has moderate evidence for success (which does look like the best evidence available for any textbooks), but the "strong evidence" category is for program shifts. This makes a certain intuitive sense -- that you might get better results changing the way math is taught at a level more fundamental than the textbooks used.
Looking at the first grade math work my son brings home (TERC) would make me weep if he weren't learning at home. It's slightly bothersome even so, as he's pretty bored with it. This weekend, he solved his first problem using the Pythagorean theorem (he needs work to fully understand it, of course). He's very bright, but not a prodigy -- I think we're wasting most of the math talent in this country.
I looked at the bee site linked above from John's Hopkins. Interesting that they used a lot of Robert Slavin's research. He also works there. Also, am I to understand that they rated by how many positive research results and size of study were found for the curriculum? What if a curriculum had 10 studies showing enough gain but 20 other studies showing it was less effective?
"Let me spell it out for you.
If you all hate EM for various reasons, then tell us what you are for. Saxon? No evidence.
Singapore? No evidence."
No evedence, or no proper research? Without "proof", there is no way to compare curricula? Proof of what? Schools have been doing this forever. And, when the WWC comments on research or curricula, are they using an absolute or world standard, or are they using a relative standard?
Without proper research people can and will make judgments. The problem is that many matematicians, engineers, and scientists have made their own judgments that do not agree with the education community.
There is also the issue of curriculum coverage. Does a curriculum get the student from point A (counting in Kindergarten) to point B (algebra in 8th grade)? Just open the fifth grade Everyday Math books next to those by Singapore Math or Saxon Math. The content difference is obvious. You don't need research for this. I compare EM and Singapore Math with my 5th grade son on a daily basis. Perhaps you would like some examples of EM stupidity that I see - things that create confusion.
"The negative rhetoric does not help solve the amazing lack of mathematically understanding in today's world."
Sure it does. The negative comments try to get schools to realize that professionals in the subject area want change; that the current crop of "reform" math curricula are part of the problem.
"So what is your solution?
I do NOT see one post that gives a solution."
The California Green Dot Standards (for content) and their list of approved textbooks.
"You talk of little substantive comments - well all of the above rhetoric is even less substantive. Do some research and get informed people."
You don't use research to set standards. The California Standards were set by very informed people. And some of us are quite informed, thank you very much. You need to stick around and read a few more threads. You may not agree with our judgments, but then again, we aren't forcing bad math curricula on your kids.
The problem with WWC is that what works might not be anywhere near good enough and that "No discernible effects" should be "Not enough data". Unfortunately, my son's school is using incomplete WWC information in place of value judgments on content and expectations. They know what they want and they will find a way to justify it.
" Perhaps you would like some examples of EM stupidity that I see - things that create confusion."
Steve, I would be quite interested to see some examples as we are fighting EM here.
"I would be quite interested to see some examples as we are fighting EM here."
Well, the most recent one I ran into with my son (also posted on KTM) has to do with EM's (5th grade) slow introduction to fractions as compared to Singapore Math. EM wants kids to discover things, so rather than have a generalized introduction to finding a common denominator, they have students look for patterns in special cases.
For example:
1/2 + 1/3 = 5/6
What is the pattern?
The pattern is that to get the numerator of the answer, you add the two denominators, and to get the denominator of the answer, you multiply the two denominators. EM loves patern matching and thinks this somehow helps students "understand" adding fractions. The problem is that you can find patterns without understanding anything. And, in this case, the pattern will get you into big trouble when you add other fractions.
Then they had subtracting fractions:
1/3 - 1/4 = 1/12
The pattern is that the answer has a numerator of 1 and the denominator is found by multiplying the two denominators. Rote pattern recognition.
Over the long run, this is very harmful. When I taught, I saw too many students try to solve math problems using pattern recognition. A problem looks like another one, so they try to plug the numbers into the other solution.
For all of the talk of "understanding", EM runs away from true mathematical understanding based on definitions and identities. They stick with pictures and superficial understandings and then back away when the time comes to mathematically justify something like invert and multiply for dividing fractions.
Pictures might be nice when you are trying to visualize how many 1/8th pieces of pie can be made from 3/4 of a pie, but you need something more rigorous when you are required to divide
3/(x-5) by 15/X^2
The goal is a solid course of algebra in 8th grade. Most schools have some sort of algebra course in 8th grade and some students do well, but often it's a watered-down algebra course. Our schools talk about our kids "holding their own" in high school. However, you should hear parents talk about the shock when their 'A' math student finds themselves way behind in high school. Then you hear about all of the parents who supplement at home.
Algebra in 8th grade is the goal?
For whom?
For all students? Why?
Have you seen 8th grade students in this country?
Remember more than 75% of them do NOT use EM. So it can't be all EM's fault. I am sure the ones who use CMC are doing great in 8th grade algebra. No supplementing or tutors for those students.
"Pictures and superficial understanding" - check out Singapore's math texts.
"back away when the time comes to mathematically justify something like invert and multiply for dividing fractions" - how do they back away? - do you even have the text? - it is clear as day in the text.
Continual amazement with the pure nonsense that is said at blog after blog.
I read one positive point - you want Connecting Math Concepts by SRA. Well, I applaud you for a position. Congratulations. Now work to get the word out on it and in your schools.
The Green Dot standards, which by the way are very similar to the NCTM Focal points. Common ground?
Maybe someone could actually have a decent discussion of improving mathematics rather than bashing EM. Even Milgram is coming around.
http://www.centeroninstruction.org/files/Milgram1.pdf
Milgram presented (Common Ground session) at this year's NCTM annual meeting (attendance 20,000) in March. He wasn't heckled too much for a change - although he avoided many questions.
Also you could read: Fuzzy Policy, Not 'Fuzzy Math,' Is The Problem." The article lays out a seven-point plan for strengthening the instructional core of curriculum, pedagogy and formative assessment.
http://www.edweek.org/ew/articles/2006/11/01/10confrey.h26.html
Doubt you have access to Ed Week, though. You might actually be more informed if you did.
Off to another blog - long live EM, now the #1 selling math text on the market.
"Algebra in 8th grade is the goal?"
Yes. By ninth grade at the latest, and it has to be real algebra, not the fake stuff.
"Have you seen 8th grade students in this country?"
You don't have a high opinion of students, do you? Low expectations and bad curricula produces poor results.
" more than 75% of them do NOT use EM. So it can't be all EM's fault."
I didn't say that. There are many bad math curricula, lots of low expectations, and plenty of bad teaching.
"Pictures and superficial understanding" - "check out Singapore's math texts."
I do. Daily. It isn't superficial.
"how do they back away? - do you even have the text? - it is clear as day in the text."
I see the books daily. Most of the understanding they teach is superficial. I gave you a clear example using patterns for fractions.
Here's what EM's Teacher's Reference Manual (Grades 4 - 6) has to say about fraction division.
"Indeed, few adults ever need to divide fractions once they leave school. Therefore, the main goal of division of fractions in Everyday Math is not to
give students practical skills . . ."
EM guarantees that the kids will not need it when they get to be adults.
"Continual amazement with the pure nonsense that is said at blog after blog."
This is not an argument. You have to do better than this.
" you want Connecting Math Concepts by SRA."
You are getting me mixed up with someone else. You better read more carefully.
"Now work to get the word out on it and in your schools."
Schools don't care one bit what parents think regarding curricula. They do what they want.
"The Green Dot standards, which by the way are very similar to the NCTM Focal points. Common ground?"
Only if NCTM keeps moving in the right direction. And this is after how many years of resisting?
"Maybe someone could actually have a decent discussion of improving mathematics rather than bashing EM."
There are plenty of decent discussions. You just don't like them. Besides, this is just a vague, throw-away comment.
"Even Milgram is coming around."
No. It's NCTM that is coming around - pretending that there has been no real change. If true, however, then thre is no common ground.
"Confrey reminds the math-phobic journalists and policy agitators that the new NCTM Focal Points wasn't a dramatic shift anywhere and that the polarization within the mathematics education community doesn't get us anywhere."
The problem is that NCTM doesn't really want to change, so they change the vocabulary and reframe the problem. They are reduced to complaining that arguing "doesn't get us anywhere".
"Doubt you have access to Ed Week, though. You might actually be more informed if you did."
Now you are reduced to Ad hominem attacks.
"Off to another blog - long live EM, now the #1 selling math text on the market."
Google and run. It's not about math, it's about pushing EM. Perhaps a little disclosure would be appropriate here.
Anonymous - I was the one advocating for CMC. But that is because I think it would be great for many kids, not all. No way would I ask to have it used exclusively. I see a district in Oregon that requires only Bridges math be used. I don't know how that program would suit all students and all teachers. You end up only pissing off a bunch of teachers and leaving a bunch of kids behind.
Yes, I get 2 free articles from the obviously opinionated edweek. They sort of churn my stomach but I read it anyways.
Some lovely quotes:
"Consider, for example, the release in September of the National Council of Teachers of Mathematics’ “Curriculum Focal Points for Prekindergarten Through Grade 8 Mathematics."
I agree with the poster above. NCTM is like a political windsock - "oh my, we made a mistake, people are complaining so let's see how we can cover it up." Talk about losing credibility. What ever happened to organizations that look at good research to base their opinions on. Egads.
"The report [NCTM] addresses a major problem faced by our population: variation among standards in the 50 states. It provides teachers and schools with advice on the most significant concepts and skills at each grade level, identifying three “focal topics” per grade. While it’s true that five of the 24 topics target quick recall and fluency of basic operations, the other 19 stress the importance of deep conceptual understanding and relationships among key ideas."
OK. Keep it fuzzy but throw them a bone - learn math facts. Wahooo! That ought to get them [parents and professionals] off of our backs [NCTM, teachers, and ed schools].
"There are critical weaknesses in the testing system. Chief among them is the failure of most assessment systems to inform teachers of specific shortcomings in their students’ learning."
Yep, blame your crappy standards on the testing system. Although there is some truth to that, it doesn't explain all of the standards problems. NCTM has such fuzzy and constructivist standards they are hard to measure and teach.
"Identify explicit learning trajectories for key concepts around which to organize instruction. Learning trajectories are the next clear step from the curricular focal points. These are not simple developmental or cognitive pathways, but rather are the results of careful curricular design combined with the strategic use of technological tools and hands-on activities."
What the f*ck does that mean? Gads, I couldn't throw more edu-babble at something if I put it through a Dilbert statement generator. Scott Adams ought to make one for the ed people.
Although you won't disclose what you do or who you are, I will. I'm an Oregon PARENT of a child with autism. I'm a programmer but studying to be a k-5 teacher. And I do that realizing that I will likely never get hired because I lean towards instructivism far more than constructivism. There is no room in the field for people like me unless I create a charter, which is difficult to say the least, especially if you have a little mortgage and need a paying job.
Admit it, change in the system is difficult and will take a long, long time. But I have lots of patience and I realize that the changes I am looking for (so parents and teachers can, in the minimum, choose their methodology) will take place waaay after I'm gone.
The NCTM derived curricula have been notorious underperformers. They have failed to increases student performance in mathematics and have failed to develop the deep understanding like they claim.
See this study matching up CMC and one of the well regarded constuctivist curricula which, thankfully, is no longer with us.
I am a CA teacher and I find your blog fascinating. I am hunting for a math intervention for our school site because the CA standards are so ridiculously long (a mile wide and an inch deep).
I will check out the various programs you have highlighted (as well as EM). But I felt I wanted to add my 2 cents in because I feel that we keep running from the real issues as well. There needs to be policy change to pare down the standards and then teacher training to train teachers how to teach the core concepts for each grade level IN DEPTH. Right now, we have no time to teach in depth. It's sad and frustrating. Everything is reduced to memorizing algorithms. At least that is the way California wants it.
Imagine being in a public school just as I was in 1-4 grade where I was put in the lowest math group with dyslexic kids. I was earning really high B's with this Everyday Math Program, but the school would not let me move up into the middle class because my average of all my grades wasn't a low B. So Viola! Off to Catholic Private School I was sent. This time around I got a nun for a teacher who completely caught me up in 6 months of the class. By that time, I was getting 100's. Now sadly, throughout middle school, I missed my friends at public school and wanted to try it out. What a fail attempt. I don't even really understand Geometry now as a Sophomore who is a year behind everyone else. I actually feel stupid that I can still remember all that stuff about "Ballpark Estimates" and "Input/Output" boxes. Who cares?
I think you suck, especially if you don't realize the politics of WWC research (bunch of right wing a-holes with a privatization/charter agenda). Besides, Everyday Math is not for everyone; especially the narrow-minded idiots who try to teach it but only bastardize it. The only complaint I ever hear about EM is "lattice multiplication" -- one freakin' lesson out of hundreds. The program does require an intelligent teacher to use discretion by tempering it to the needs of one's students. And if there are teachers out there following the lessons like a script, then they should have their certificate revoked and find a different subject (not to mention, some creativity) to teach. Can't find it? It's on the "fuck off" aisle next to the clues.
Occasionally some comments refute themselves.
Post a Comment