May 7, 2006

A Few Steps Short of a Proof

J.D. of Math and Text responds to my Parental Support post below.

First J.D. cites the article that made the rounds recently in which a super teacher recognized that some of her students were hungry and did the reasonable thing and fed them. A heart-warming anecdote for sure, but an anecdote nonetheless, because when we've studied the link between feeding children and academic performance of American kids we don't find one:
Taras noted that studies of absenteeism rates among students who are offered breakfast at school found that when schools offer breakfast, children are not only more likely to attend school but also have low rates of tardiness. American-based research studies do not show any positive effects of a good breakfast on students'’ academic performance once they get to school, but he noted that the studies showed that grades do improve with a school breakfast in undeveloped countries where children are malnourished.
That's because in the U.S. our poor are more likely to be obese than malnourished. Nonetheless, I'm all for feeding any kid who isn't getting enough to eat. With the number of state, federal, and charitable programs that address this concern, there really is no excuse for any kid going to school hungry. But, let's not pretend that it's going to make more than a dent in student achievement failures outside of the Congo.

JD's next argument is a strawman:
If you're a teacher of a student who spends one third of her time in your class, one third of her time being verbally and/or physically abused by her parents, and, hopefully, one third of her time sleeping, that teacher is not responsible for jack-squat. The parents have determined the student's fate...

... The radical message implied by these ramblings is that, even if I were to have sent my daughter off to be raised by wolves in her first 4 years, the state is responsible for catching her up to everyone else.
The NAEP indicates that about 70% of students are not performing at grade level. JD would have us believe that this 70% comprises the "verbally and/or physically abused" and kids raised by wolves. JD forgot about the cognitively disabled and a whole host of other problems that prevent some kids from achieving. But I didn't:
Sure, they'll be a few hard cases that won't respond to effective classroom management or who lack sufficient cognitive ability.
When we add up all the abused kids, the emotionally and/or cognitively disabled kids, and JD's wolf boys we wind up with a tiny fraction of students, about 2%-10%, who won't be able to achieve academically given even the best schooling. I've already excused the performance of these kids. So what about the remaining 60%, JD? Many of these kids are getting at least some parental support.

JD, of all people, should know that the primary reason why these kids aren't achieving is overwhelmingly due to bad curricula and instruction. Maybe an example is in order.

My favorite is the City Springs School in Baltimore City. In 1998 it was the worst school in Baltimore City, which is quite an accomplishment considering the general wretchedness of the schools in Baltimore City. CTBS/TerraNova scores for the fifth grade were 14th and 9th percentile for reading and math respectively. Then they changed the school management and curriculum (one that does not require parental support, JD) and five years later the scores improved to 87th and 79th percentile respectively. This school has every disadvantage you can think of, and yet they've managed to succeed. Apparently, six hours a day, 180 days a year is sufficient time to compensate for all but the very worst of parenting. According to JD, this should be impossible. And yet here we are.

QED

16 comments:

Anonymous said...

Ed heard Jay Matthews and someone from Harvard on NPR.

They were saying that "parental involvement matters," but that parental involvement is an effect of a good school, not a cause of a good school.

When an inner city school is good, parents are drawn in.

I'm sure that's true, and I love tossing the Parental Involvement Dodge back in the lap of the school.

otoh, I'm sure that parental involvement is a net plus, so I think a good school should build "social capital" - i.e. trust - with parents.

Mike in Texas said...

"The NAEP indicates that about 70% of students are not performing at grade level."

It says no such thing. You obviously do not understand what the different achievement levels signify.

http://nces.ed.gov/nationsreportcard/reading/achieveall.asp

For 4th Grade:

"Fourth-grade students performing at the Basic level should demonstrate an understanding of the overall meaning of what they read. When reading text appropriate for fourth-graders, they should be able to make relatively obvious connections between the text and their own experiences and extend the ideas in the text by making simple inferences."

In short, a 4th grader at the basic level understands 4th grade writing.

http://nces.ed.gov/nationsreportcard/nrc/tuda_reading_mathematics_2005/t0005.asp?printver=Y

Indicates 62% are reading at or above reading level, as defined by the meaning of "basic" above that I took off the US govt. website.

Still an unacceptable figure, but well off the 70% you claim. If you examine the charts it is the large urban school districts dragging the scores down.

Anonymous said...

"Still an unacceptable figure, but well off the 70% you claim."

So, your just quibbling and not disagreeing with the main point.

Also, who ever said that the NAEP Basic level is anything to be proud about? Let's look at some examples from fourth grade math.

1. A club needs to sell 625 tickets. If it has already sold 184 tickets to adults and 80 tickets to children, how many more does it need to sell?

56 percent got this wrong. In 4th grade!

2. 4/6 - 1/6 = ?

46 percent got this wrong.

3. 972 - 46 = ?

24 percent got this wrong. I forgot to check whether a calculator was allowed for this one.

4. 24 / (6/2) = ?

42 percent got this wrong.

5. Jo's recipe says to bake a cake for 25 - 28 minutes. About how long is this?

A. A quarter of an hour
B. Half an hour
C. An hour
D. An hour and a half

49 percent got this wrong.


Even if you accept NAEP's calibration of "Basic Level", these results are astounding.

This has very little to do with parental support and everything to do with educational philosophy and competence. Once again I ask how a school knows that the problem is related to parental support rather than bad curriculum and teaching.

Mike in Texas said...

Question #1 is written at a 7th grade reading level, perhaps that's why so many kids had trouble with it. Copy and paste it into word and check it for yourself.


Question #2, is subtraction of fractions even taught in 4th grade in most states? I checked the California state math standards, which are generally accepted as being well written, and there is no mention of adding or subtracting fractions.

Anonymous said...

"Question #1 is written at a 7th grade reading level, "

Parsing. I've heard this excuse before. By the way, what percentage of the 56 percent wrong is due to reading?

Maybe it's a 7th grade reading level in Texas, but not where I'm from. OK, you stand up in front of all parents and explain to them that it's OK for 56 percent of 4th grade kids to get this wrong because they can't read.
You can also explain why the school is teaching this kind of math before 7th grade even though you don't think that these kids should be able to read it. Perhaps in Texas they don't teach math problems like #1 before 7th grade. That anyone would think it is OK for a 4th grader not to be able to read and understand this question is quite incredible. Many parents might not be able to discuss educational philosophy, but they sure know low expectations when they see them.


"I checked the California state math standards, which are generally accepted as being well written, and there is no mention of adding or subtracting fractions."

Even adding and subtracting fractions with like denominators is covered in the Everyday Math program my son uses in 4th grade. As for California's standard, here are some practice problems for 4th grade:

1. State whether this is true or false:

1/2 > 2.5

5/2 > 2.7

3/7 < 10/21

2. Change the following to a decimal:

11 2/100

3. Is the following equation true?

5 + 5/4 = 5 + (5-4)


You pull out and quibble about a couple of problems. What's your point? That everything is fine? That it's not a school, curricula, or teaching problem?

My premise is simple. The results of testing are so horrible by anyone's standard that you cannot blame it on kids, parents, society, lack of money, class size or any sort of other "if only".

Anonymous said...

Basic Instincts



"Basic" denoted "partial mastery of knowledge and skills." "Advanced" signified "superior performance beyond grade-level mastery." "Proficient," though, was the key. NAGB termed it "the central level," representing "solid academic performance for each grade tested" and "a consensus that students reaching this level have demonstrated competency over challenging subject matter and are well prepared for the next level of schooling." NAGB intended that "proficient" would represent the skills that every student ought to possess -- even if many were not there yet. On NAEP tests since 1990, this level of performance has usually been reached by about three kids in 10. Everyone knows that's unsatisfactory. But it's also reality, an accurate gauge of the gap between U.S. pupils' prowess and what they need to match world standards.

From the outset, some educators protested that NAGB's "proficient" was too ambitious, but the board stuck to its guns. For the past 15 years, both NAGB and the Department of Education, which manages NAEP, have resisted pressure from politicians and educators to back away from, or dumb down, the "proficient" standard. With NCLB, however, that's begun to change. More voices are demanding that NAEP focus attention on the much-lower "basic" standard. Explains a spokesman for the Tennessee Department of Education: "NAEP's basic is comparable to our proficient." Federal officials should push back, insisting on NAGB's "proficient" as the gold standard. They should continue to highlight -- and deplore -- any gaps between it and state test results. But the White House and Education Department now crave proof that NCLB is succeeding and seek to accommodate state pleas for "flexibility" and pacify governors threatening to withdraw from NCLB.

Hence they, too, are subtly substituting "basic" for "proficient" when they report NAEP results -- and downplaying standards altogether in favor of simple up-and-down trend lines. In releasing the 2005 scores, the Education Department for the first time published comparison tables showing state-specific progress only in relation to "basic." And even NAGB members now highlight "basic" rather than "proficient." In October, chairman Darvin M. Winick, a long-time Texas associate of Education Secretary Margaret Spellings and President Bush, spoke only of gains at the basic level. His "reporting and dissemination" committee acknowledged that "We're trying to draw attention to basic as an achievement level with some value."

KDeRosa said...

Mike,

I think you missed this page.

Basic = Partial mastery of prerequisite knowledge and skills that are fundamental for proficient work at each grade.

That's why I use proficient which indicates "Solid academic performance for each grade assessed."

Referring to the definition of proficiency lets see what kids with only a basic understanding can't do:

Fourth-grade students performing at the Proficient level should be able to demonstrate an overall understanding of the text, providing inferential as well as literal information. When reading text appropriate to fourth grade, they should be able to extend the ideas in the text by making [simple] inferences, drawing conclusions, and making [relatively obvious] connections to their own experiences. The connection between the text and what the student infers should be clear.

Proficiency should be the level we are striving for, not just basic. Although, most states have set their NCLB targets closer to the basic levels than the proficient level in any event.

KDeRosa said...

With respect to Steve's problem #2, the problem is subtraction of fractions with like denominators. According to my designing math curriculum book, this is a skill that is usually introduced in the second grade. Plain old addition, subtraction, and multiplication of fractions are skills introduced in the fourth grade usually. So depending on when in the school year the NAEP is adminstered, even these may be fair game.

Mike in Texas said...

SteveH,

The problem with the question is it is so poorly written it requires 7th grade reading ability to understand it. It was given to 4th graders. You don't have to be a genius to see it is poor choice for 4th graders. What it is really determining is that 56% of 4th graders have an inability to read a question written on the 7th grade level.

BTW, its called a Flesch_kinkcaid readability score. MS Word can do it for you. I don't know YOU are from but in the rest of the world its fairly simple to determine the reading level of a given selection. If you need me to explain it to you I can.

Mike in Texas said...

And following a link on the page KDeRosa linked to you'll find this gem:

"The most recent congressionally mandated evaluation conducted by the National Academy of Sciences (NAS) relied on prior studies of achievement levels, rather than carrying out new evaluations, on the grounds that the process has not changed substantially since the initial problems were identified. Instead, the NAS Panel studied the development of the 1996 science achievement levels. The NAS Panel basically concurred with earlier congressionally mandated studies. The Panel concluded that "NAEP's current achievement level setting procedures remain fundamentally flawed. The judgment tasks are difficult and confusing; raters’ judgments of different item types are internally inconsistent; appropriate validity evidence for the cut scores is lacking; and the process has produced unreasonable results."1


Now, do I have to explain the meaning of fundamentally flawed and unreasonable results?

Mike in Texas said...

"My premise is simple. "

Your premise is wrong. The assessment instrument is so poorly constructed even the US govt. says its a piece of garbage.

http://nces.ed.gov/nationsreportcard/achlevdev.asp

KDeRosa said...

Mike, you have your opinion study and I have mine:

A new report from the Brown Center on Education Policy at the Brookings Institution finds that math items on the National Assessment of Educational Progress (NAEP) math assessment lack challenging arithmetic, often requiring skills that are several years below grade level. The findings cast a disturbing light on recent highly-publicized math gains as measured by the NAEP assessment.

Anonymous said...

Mile wrote:
"Still an unacceptable figure, but well off the 70% you claim."

You can't have it both ways. You say that it is unacceptable, but then say:

"The assessment instrument is so poorly constructed even the US govt. says its a piece of garbage."

I'd like to see you get up in front of a group of parents, show these questions, and explain why it's OK that the results are so bad. Perhaps you have an "instrument" that shows that everything is fine? I've heard all of this from you before. Name a test that you think is reasonable.

KDeRosa said...

With respect to Steve's problem #1:

First, the wording for this problem is typical wording for math word problems. The readibility formula Mike is using is notoriously inaccurate for short passages less than 100 words in any event.

Second, the problem is a multistep problem with three larger numbers; the sum of two numbers is subtracted from the third number. This is a fourth grade problem according to my designing effective mathetatics Instruction book.

Ryan said...

City Springs schools has 20 percent of their kids not meeting standards, and yet you praise them?

KDeRosa said...

The Rain, City Springs is an inner city school not a typical middle class school. The average math score was 9th percentile. This means that half of this school used to score below the ninth percentile in math. This is scraping the bottom of the barrel. Now, the school scores at the 79th percentile. So, assuming a normal distribution, only about 20% of the kids scoring below the ninth percentile are below average, or about 2% of the general student population. Considering that the school has a 25% mobility rate and math is brutally cumulative, this is spectacular.