You can lead a horse to water ...

A presidential panel said yesterday that America's math education system was "broken," and it called on schools to ensure children from preschool to middle school master key skills.

...

F. Joseph Merlino, project director for the Math Science Partnership of Greater Philadelphia, which runs a research program involving 125 schools in 46 school districts, said that while he agreed with the finding that "you can't teach so many topics that you aren't able to get into depth," he disagreed with the report's focus on improving algebra instruction as central to better math education for all students.

He said he favored tailoring math instruction to the learning styles of students more than the report does. (emphasis added)

Philadelphia Inquirer, Panel: Math education is 'broken' The presidential panel called for ways to improve teaching and fight "math anxiety."

Bear in mind that in 2005, only 15.8% of black 11th graders in Philadelphia performed at the proficient level or above on the state math test. This placed them 2.61 standard deviations below the mean pass rate of 52.8% in Pennsylvania. This places these students below the first percentile.

See here.

I guess they haven't found the right learning style for these students yet.

Statistical Illiteracy

Here's a nice example of some dubious reasoning based on unwarranted statistical assumptions.

Iris C. Rotberg, a professor of education policy at George Washington University, said any comparisons based on international tests, such as PISA, would be more reflective of the poverty in a state—or country—than of the quality of its schools or teachers.

“Making more comparisons and having more tests won’t solve the basic problem: We have a lot of kids living in poverty,” she said. “Governors can probably predict what their test scores will look like.”

Comparing poverty effects and school/teacher quality effects cross-nationally is fraught with problems. Each country defines poverty levels differently and I'm not sure anyone has devised a methodology for rating school/teachers that is objective, comports with reality, and has predictive value.

The OECD botched-up their analysis of SES effects for the 2003 PISA. No doubt this report forms the basis of Professor Rotberg's dubious comparison.

Fig 4.8 (p. 176) of this 2004 PISA report is a pretty graph of the relationship between math performance and SES of OECD countries.



See the pretty trend line? Notice how the data is a poor fit to the trend line? Notice how the best of the low-SES students outperformed the worst of the high-SES students?

If you turn to Table 4.3b, col. 3, p. 398, Annex B1 you see that the OECD Avg R square is only 17.9%.

This means that only 17.9% of the variance in student performance is accounted for in the variance of SES. Notice how I was careful to avoid an implication of causality, something that the OECD and Professor Rotberg take a rather cavalier attitude toward.

This means that 82.1% of the variance in student performance is not accounted for by the variance in SES. Other factors, such as school quality, account for this remaining 82.1%

The OECD tried to quantify school effects on student performance. Somewhat unsuccessfully. (I challenge anyone to read Fig. 3.6 without giggling.)

It may be the case that school effects do account for less than poverty effects. Poverty effects account for less than 18%. It could be that school effects account for even less. Certainly, most schools are clueless when it comes to educating lower-SES students. But that is not to say that the few highly effective schools have a greater effect. The data is few and far between.

According to Rotberg, the basic problem is poverty which is more reflective of (comparisons involving) testing results. This implies causation when all we have is correlation data. And the correlation between poverty (SES) and testing results is weak (17.9%). Why focus on this small factor when 82% of the variance is attributable to other factors? It's not like anyone's been successful raising student achievement by increasing a student's SES.

Two final notes.

1. You can add the term "a lot" to the Downes Lexicon of Misrepresenting Statistics Through Language. Apparently, "a lot" means 22% -- the largest figure I could find for childhood poverty in the US (as long as you don't mind excluding non-cash benefits from "Income."). Remember, "often" and "many" are 16% and "a lot" is 22%. We're going to have to make-up some new words to represent truly high levels or frequencies since most of the existing words are quickly being used up for low incidence events. I want to recommend "super duper" for 80% and "shiiiit!" for 90%.

2. According to table 4.3b, the R² for SES effects in the U.S. is 23.8%. I calculated the R² to be 31.2% for parental education levels in Pennsylvania relative to student performance. I'm still waiting for Stephen Downes to explain to me why Pensylvania's scores don't generalize to the rest of the U.S.

(Hat tip to Charles Barone)

The State of Black Education in Pennsylvania

Is shocking.

Appalling.

In fact it's so bad, I think I mangled the data somehow. Feel free to perform a reality check.

I'm using school district level data from S&P's School Data Direct. Specifically, I'm using the data set from 2005 since it is the latest year containing financial and demographic data. I'm also using scores from the PA State test, the PSSA, for the 11^th grade since it represents the end-product of a public school education in Pennsylvania.

Pennsylvania has 501 school districts. 127 school districts reported test scores for black students. The remaining 374 school districts presumably had less than 10 black test takers which I believe is the reporting cut-off in Pennsylvania. About 13,206 black 11^th graders have scores included in the datas et.

Black students are highly concentrated in a handful of school districts. 70% of all black students in reporting school districts are concentrated in the following 10 school districts: Erie City, Harrisburg City, Norristown Area, Penn Hills, Philadelphia City, Pittsburgh, Pocono Mountain, Upper Darby, William Penn, and Woodland Hills. In fact, 2/3 of black students are enrolled in the Philadelphia School District alone.

With this in mind, let's look at the data.

First let's look at school district expenditures for these ten schools.

School District	Total Exp	z-score	Instr Exp	z-score
Erie City (180)	$11,310	-0.01	$5,491	-0.23
Harrisburg City (309)	$14,887	1.42	$7,635	2.13
Norristown Area (171)	$14,775	1.38	$8,008	2.54
Penn Hills (200)	$11,779	0.18	$6,270	0.63
Philadelphia City (6,617)	$13,498	0.87	$4,919	-0.86
Pittsburgh (980)	$18,065	2.69	$6,856	1.27
Pocono Mountain (185)	$13,441	0.84	$5,432	-0.29
Upper Darby (199)	$9,541	-0.71	$5,551	-0.16
William Penn (238)	$12,328	0.40	$6,826	1.24
Woodland Hills (176)	$12,611	0.51	$6,863	1.28
Mean	$13,224	0.76	$6,385	0.75
Weighted Mean	$13,739	0.96	$5,380	-0.35
State Mean	$11,322	--	$5,699	--

I've given the mean and weighted mean to show the effect that the Philadelphia School district, which contains a substantial majority of the tested black students, on the data. The numbers in parentheses are the number of tested black students in reading (math numbers were similar).

(A quick word on z-scores: the z-scores are given in standard deviation units. Here's a graph you can use to convert z-scores (white) to percentiles (yellow.)



So what does the data tell us?

These ten school districts are not starved for funding. All but two (Upper Darby and Erie City) have total expenditures above the State mean ($11,322). The weighted mean tells us that the mean black student is enrolled in a school with more expenditures than about 84% of PA school districts. So much for racist funding.

I've also given the data for instructional expenditures to give you an idea how much money actually makes its way into the classroom. Unfortunately, Philadelphia's wasteful ways, with instruction accounting for only 36% of total expenditures, brings the weighted mean down to the 36^th percentile. This is unfortunate, but as we know from previous analyses school expenditures are only weakly correlated with student performance (and to make Stephen Downes happy: at least in Pennsylvania for at least the year 2005 (feel free to generalize up to your zone of comfort)).

Now let's see how black students are performing in these ten school districts. You better sit down for these numbers.

School District	Math	z-score	Read	z-score
Erie City	34.4	-1.30	49.1	-1.67
Harrisburg City	6.5	-3.27	16.5	-4.39
Norristown Area	22.2	-2.16	29.9	-3.27
Penn Hills	18.5	-2.42	37.6	-2.63
Philadelphia City	15.8	-2.61	25.3	-3.66
Pittsburgh	17.4	-2.50	30.8	-3.20
Pocono Mountain	23.8	-2.05	45.1	-2.00
Upper Darby	20.6	-2.27	28.5	-3.39
William Penn	14.7	-2.69	19.9	-4.11
Woodland Hills	15.3	-2.65	33.1	-3.01
Mean	18.9	-2.39	31.6	-3.13
Weighted Mean	16.4	-2.57	26.7	-3.54
State Mean (All)	52.8	--	69.1	--

For State mean (All) I'm giving proficiency scores for all students in Pennsylvania, not just black students.

Talk about an achievement gap. For math, the achievement gap is -2.57 sd and for reading it is -3.54. This means that the mean black student in these ten school districts is performing at below the first percentile. 99 percent of students in Pennsylvania are outperforming them. (Like I said, I'm suspicious of these scores, but I can't seem to find the error.)

These results are fairly consistent across the ten schools. All ten schools perform poorly. The achievement gap tends to be in the neighborhood of a standard deviation, but the gap in most of these schools is double and triple that.

I'm not sure exactly what's going on in these schools, but it's a safe bet to say whatever it is it isn't working for these kids. And, bear in mind that about half of the black students have dropped out by this point, so these scores are likely for the top half of the curve.

The Ongoing Reading First Debacle

Last week Sol Stern penned an article for Fordham detailing the blunders committed in the 2006 Reading First investigations and the media's mis-reporting of same.

The article was well written, but didn't contain much new reportage, such as the Slavin/Obey connection. A year and a half ago, I reported much of what is found in Stern's article right here on this blog. In fact, many of Stern's points are identical to points I made.

Stern's article was critical of Bob Slavin, creator of Success for All a program that received little Reading First funding. Dean Milot, of EdBizzBuzz, has posted Slavin's somewhat tepid response. Slavin claims that Stern left out all the juicy parts. I disagree. Stern did include the juicy parts. Stern left out the gristle. But, apparently, Slavin likes gristle.

Here are Slavin's arguments:

1. Stern says nothing about the fact, prominently reported by the Inspector General (IG) and the press, that leaders of the Reading First Technical Assistance Centers, Edward Kame’enui, Deborah Simmons, and Sharon Vaughn, were also authors of the Scott Foresman basal text and authors of Voyager Passport, and yet were making key decisions from the outset that favored basal textbooks and Voyager Passport.

During the April 20, 2007 Hearing, IG Higgins confirmed that no one involved in the RF scandal had any actual financial ties or conflicts of interest with respect to any basal reading program. Furthermore, there was never evidence adduced in any IG report that Kame’enui, Simmons, or Vaughn made decisions "key" or otherwise that favored Voyager Passport or any other reading program. How could they have? Less than 10% of the states even specified the actual names of reading programs in their Reading First applications. Even if there was a conflict with the reviewers, they had no way of knowing which programs the states were actually selecting. Moreover, my understanding is that Kame’enui, Simmons, or Vaughn are authors of the DIBELS testing instrument which was subsequently included as a component of Voyager Passport. They were not actual authors of the reading program, nor did they derive profits from the reading program

In addition, there is no evidence that Voyager Passport failed to qualify for Reading First funding under the loose "based on SBRR" standards contained in the statute. In fact during the hearing IG Higgins expressly indicated that he had not even looked into this issue.

This is an exceedingly slim grounds for a finding of impropriety. To the extent that this is Slavin's first and most prominent point, you can imgaine the strength of his remaining points.

2. Stern says nothing about the fact, reported by the IG, that in the early Reading First Academies, when state leaders were learning how Reading First would operate, they were exposed to speakers representing only Direct Instruction and selected basal textbooks, and were given notebooks full of information on DIBELS and no other assessment.

This falls under the dubious "appearance of impropriety" standard since no actual harm or foul ever resulted from this activity. Nor was this activity prohibited under statute or any other regulation. As one of the only three reading programs with validated reading research, it's difficult to see how DoE would be prohibited from using Direct Instruction as an exemplary program. Nor is DoE obligated to demonstrate every potential exemplary program. Demonstrating a program does not amount to endorsing it. And there is no evidence adduced that any program was actually endorsed by DOE.

According to the IG, there was no actual infraction here, merely the appearance of an infraction. And, even that is a stretch.

3. Stern says nothing about the fact, reported by the IG and the press, that the Simmons & Kame’enui “Consumer’s Guide,” based in detail on elements of Direct Instruction, was given by Department officials as the de facto official criterion for “scientifically based research”. The Oregon review of reading programs, based on the guide and carried out in part by University of Oregon researchers who were authors of one of the programs, was frequently recommended by the Department as a list of programs to be used under Reading First.

Reading First is a program directed to at-risk students. DI has more validated reading research with respect to at-risk populations attributed to it than any other reading program, including Slavin's SfA. So, if the Oregon researcher did base their criteria on DI, that would seem to be appropriate. However, based on this summary report it would appear that the DI reading program (Reading Mastery Plus) was not always the highest scoring reading program evaluated by the Oregon Researchers. Take for example the first grade evaluation for discretionary items:



Reading Mastery comes in fifth. Success for All appears also to have done well in the evaluation, at least in the early grades. perhaps that accounts for the fact that SfA was on the RF approved list in 28 states.

Again, I'm not quite sure what the infraction is here even if what happened is exactly what Slavin claims. Basing the criteria on the most research validated reading program and including that program on a list of programs meeting the criteria does not seem to be prohibited. And, if this amounts to "recommending" or "endorsing" a program then those recommendations and/or endorsements weren't very effective. Less than 3% of states adopted the program despite all the alleged improper recommendations and/or endorsements.

4. Stern fails to mention how DIBELS became the de facto national assessment of Reading First in most states, enriching Roland Good and the University of Oregon, one of the Technical Assistance Centers. Under Department funding, Good and Oregon colleagues reviewed a variety of reading measures and gave positive ratings to DIBELS.

I'm not as familiar with the DIBELS part of the scandal, but it appears that Good actually donated all the royalties he received from DIBELS.

One official, Roland H. Good III, said his company made $1.3 million off a reading test, known as DIBELS, that was endorsed by a Reading First evaluation panel he sat on. Good, who owns half the company, Dynamic Measurement Group, told the committee that he donated royalties from the product to the University of Oregon, where he is an associate professor.

Moreover, no evidence was ever adduced in any IG report showing that any state was actually steered to adopt DIBELS.

5. Stern claims that schools avoided Success for All (SFA) just because it was too expensive. ... We have a file drawer full of anguished reports from Success for All schools and potential SFA schools all over the U.S. pressured by state or local Reading First officials to avoid SFA because it was inconsistent with Reading First. Many schools refused Reading First funding to adopt or keep Success for All. If the Department did not directly tell state officials to exclude SFA, they did not correct the national perception that SFA did not fit in Reading First.

I'm not sure about Stern's initial claim; however, there is no evidence of record that DoE steered any state away from SfA. Remember, just because SfA was eligible for RF funding does not mean that they were entitled to it. States had the discretion to exclude any program from RF funding they wanted. DoE could only preclude states from adopting reading programs that did not meet the statutory requirements. DoE could not force a state to include any reading program in RF. In my opinion, excluding any validated reading program, such as DI or SFA from Rf funding, is scandalous. But it's a scandal at the state level, not the federal level.

I'm going to skip the last two points since point six makes little sense and I have no way of disproving the assertion point seven.

Slavin has had a weak case from day one and nothing in this latest letter changes that fact.

Theory IV: Do Poor Students Perform Better in School Districts with Educated Parents

Let's take a look at what the data shows with respect to the performance of low-SES students attending school districts with highly educated parents.

The theory is that low-SES students will perform better in these school districts. Educated parents are supposed to value education more highly and care about the education their children are receiving. Apparently, these high-SES districts attract better teachers, have better students with better motivation, have more resources, can pay more attention to the small number of low-SES students, and the like. The children of these highly educated parents are supposed to value their education more highly and, as a result, will perform better. In theory, low-SES students should perform better when in classrooms with these children of the highly educated.

Let's see if the data supports this theory.



The graph compares student achievement (11^th grade combined math and reading) with the percentage of adults with bachelor degrees residing in the school district. There are two populations being compared. The pink squares represent all students. The blue diamonds represent economically disadvantaged students (those qualifying for free or reduced lunch). I added trend lines for both populations and also indicated the mean score of all students (the horizontal red line. Mean = 60.7) .

Here's the regression results for all students:

• R = 0.56 (there is a medium association between parental education (of the community) and the performance of all students)
  
• R² = 0.312 (there is a medium fit between the data)
  
• P = 3.77729 x 10^-42 (the results are statistically significant)

Here's the regression results for low-SES students:

• R = 0.22 (there is a small association between parental education (of the community) and the performance of low-SES students)
  
• R² = 0.048 (there is a low poor between the data)
  
• P = 0.000003 (the results are statistically significant)

Interpretation

(The correlation between high-school diplomas and student achievement was similar.)

We see that as parental education increases so does student (all) scores. There is a medium sized correlation and fit between the data. This correlation is well known.

Things start falling apart, however, when we look at the performance of low-SES students. We should expect to see a lower correlation because we've restricted the range of students. Both the correlation and data fit for low-SES students is low. The trend is not reliable given the low data fit. I've left it in for reference purposes. Whatever is causing the increase in student (all) scores seems to be missing from the low-SES student scores. The magic aura of the kids with highly educated parents does not seem to be rubbing off as well as some have predicted. Few of school districts have low-SES scores above the mean for all students.

To the extent that low-SES test scores are slightly higher in school districts with high percentages of highly educated parents, this is most likely attributable to rising SES status of the low-SES students. We know that SES is positively associated with student performance. We also know that the variance in family income for students eligible for free or reduced lunches is wide and includes about 40% of all students, i.e., most students to the left of the mean.

The take away is that just because the correlation between parental education and total student performance is midsized, does not mean that the correlation between low-SES student performance and parental education is also midsized. It's much lower than that. And the student achievement gains we should expect to see by placing low-SES students in higher-SES school districts (as a function of parental education) will likely be low to non-existent. In any event, we should not expect those gains to pull low-SES student achievement up to the mean performance of all children based on this data.

Kozloff on Reading First

Martin Kozloff just posted this on the DI Listserv in response to Sol Stern's article on Reading First. I think it deserves wider distribution.

There are some folks in our Great Nation who use statistics to make the case that our students are learning just fine (thank you very much), and that the criticisms of public education are therefore in error and (because of the alleged political leanings of the critics) politically motivated (a subtle form of ad hominen). [Ed: this is a not-so-subtle jab at Jerry Bracey who burdens the DI listserve from time to time.]

These same persons are often the harshest critics of the tests and the raw data (e.g., student achievement) that are the basis for the statistics.

In so doing, they betray what might be considered bad faith.

"Sure, the tests are bogus, and the 'standards' for passing are so low that nonreaders can pass, but I like what the stats say---'everything is fine in Edland.' So, I'll promote the stats as if THEY portray reality."

Personally, I couldn't care less what official statistics say---not when I can see and hear behavior.

Everything is NOT fine in Edland when I (all by my wittle self) have tested over a thousand kids in my county, and they are reading at below first grade level.

"Kite mad a bowat. She mad the bowat of thin. The noise of the bowat was real thin..."

Wow. She got 5 words right!

Oh, yeah, these kids----who have had FIVE years of school, 5 x 180 hours of reading "instruction"---can't read a simple sentence and therefore have no idea what it says---these kids are being WELL served.

And I know exactly who taught them and exactly where they learned to "teach" reading, and I know exactly HOW they taught reading....


Teacher. "So, what do you think THIS word is?"

Kid. "How the f$#@ should I know? I don't know how to read."

Teacher. "Well, what word do you think FITS here?"

Kid. "How the f%$# should I know? I can't read ANY part of the sentence. So how can I tell what FITS means, you stupid f%$#ing cow?"

Teacher. "Well, do you see pictures on the page that can help you?"

Kid. "Oh, Sweet Jesus! Is there a picture for EVERY word? Is that how I'm supposed to read? I use pictures to tell me what the WORDS say? Then why not just write books with PICTURES and forget the alphabet, you stupid f%$#ing cloth headed rump fed gas bag?"

Teacher. "Now, you don't need to cuss. Cussing is offensive."

Kid. "Right. I DON'T need to cuss. And apparently I don't NEED to read, either! You smarmy, arrogant, dumb-ass, RACIST piece of $#@*."

Teacher. [sharp intake of breath] "Racist!! How dare you accuse me of being a racist. I give annually to every minority charity there is. Okay, maybe two. AND I once heard a speech by Martin Luther King, Jr."

Kid. "You are a racist because---despite all your bull$#@! talk about serving the needs of DIVERSE learners---and I am pretty f%$@ing diverse---you INSIST on having me GUESS what the words say and you REFUSE to teach me the five main reading skills in a STRAIGHTFORWARD way, as the RESEARCH and 3000 YEARS of history shows. THAT'S why you are a skanky, airheaded, puss-bag racist piece of $*&^. And I will hate you forever for refusing to do what the simplest COMMON sense would tell anyone with a SOUL to do. But you care MORE for your precious PEDAGOGY and your PRIVILEGED social class of UNACCOUNTABLE educators, than you care for a real live human being, you duplicitous smug self-satisfied sack of crap."

Teacher. "Well, you are certainly a struggling reader. You need to be in Reading Recovery. Sadly, Reading Recovery is for FIRST grade and YOU, as you know, are in FOURTH grade. But don't take it personally. As one of my gurus used to say...

'Saying that we are determined to teach every child to read does not mean that we will teach every child to read....The best we can do ... is ... to ensure that, if not every child lives up to our hopes, there is a minimum of guilt and anguish on the part of teachers, students, and parents. (p.441) Smith, F. (1992). Learning to read: the never-ending debate. Phi Delta Kappan, 74, 432-441.

Kid. "That's what your guru used to say?"

Teacher. "Yup. He sure did. Said it often. Finally, we understood. NOT our fault. Maybe YOURS."

Kid. "Well, f%$# him, too. You people are always blabbing about social justice and revolution. If there ever WERE a revolution, you and your EXPLOITING class who, as Marx said, control the means of production----in this case, the technology for educating kids----would be hanging from the street lamps."


So, if anyone wants to know if pre-Reading First instruction worked just fine, visit a NONRF school (a school whose teachers were NOT REtrained with RF money) and see how well the DIVERSE learners are doing.

Then go across town to a RF school---where they are using Reading Mastery (not SFA)---and see.

This isn't about statistics or what works or who emailed what or who influenced whom or who feels left out or what research "really" says or...

This is nothing but plain old privilege---a class of deaducators that presumes it is some kind of aristocracy--The Best---simply because it is in a position of power and has been able to get generations of gullible teachers and administrators to dance to its progressive tunes.

And when it is challenged---its legitimacy ("You don't know as much as you think. Therefore, why are you in a position of power?"), its position, and therefore its privilege---it acts the way all ELITES act.

It tries to kill those who threaten it.

But of course it has to do the killing in a socially approved way.

So it stages a pageant. Makes it seem as if some folks in RF acted improperly.

Makes it seem as if RF didn't do much good, anyway.

The usual bull$^&* "narrative."

And this elite class has just enough useful idiots (an audience and bit players)---for ex in congress and in the ed establishment of pundits---to do its bidding and applauding.

It's all show. The entire ed establishment.

A gigantic production number almost as gaudy as a Busby Berkeley...

We need law suits.

Break their f%$#@ing backs with law suits.

Although hanging is both effective AND provides teachable moments relevant to physics.


And as for Success for All, have y'all ever compared it with Reading Mastery---which came out maybe 20 years before SFA.

Remarkable similarities.

Even wording!

And was SFA developed by persons with a long history of designing instruction and in reading?

I dunno.

Easy to find out.

Interesting that SFA hasn't claimed that ZIG ripped THEM off.

WaPo Colors outside the lines

The Washington Post colors a highly misleading article on NCLB's narrowing effects on non-reading and non-math subjects, such as art, in elementary schools.

It was all art, all morning at the Montgomery County school, casting a local spotlight on a national reality: that art is often squeezed out of the curriculum by the academic rigors of the No Child Left Behind law.

...

Her sentiments echoed a report released last month by the Washington-based Center on Education Policy, which found that many elementary schools across the country have allotted more time to reading and math by cutting time for social studies, science, art and physical education. The issue of "curriculum narrowing" has become a key part of the debate over reauthorizing the 2002 federal law, which is designed to improve reading and math proficiency. (emphasis mine)

According to WaPO, art is "often squeezed out of the curriculum" by NCLB.

Often! Often? What percentage of schools do you think WaPo considers to be "often"? 2/3rds? 3/4ths?

According to the CEP report the percentage of schools that have reduced instruction time Post-NCLB is a whopping -- wait for it -- 16%.

I've heard that 40 is the new 30, but I didn't realize that 16% was the new often.

And, the category is "art and music" not just art. These 16% of schools could have cut either music or art or both.

Now let me give you some actual data from the CEP report, which WaPo fails to do

84% of schools didn't cut any time from music and art.
2.9% cut less than 25 minutes per week.
3.8% cut between 25 and 29 minutes.
4.3% cut between 50 and 74 minutes.
4.5% cut between 75 and 149 minutes.
0.5% cut more than 150 minutes.

For those schools (16%) that did cut time, the pre-NCLB time spent on art and music instruction was 154 minutes a week. The post-NCLB time was 100 minutes. So, on average, about 54 minutes per week was cut from art and music instruction per week. The report doesn't tell us how much instructional time the 84% of schools who didn't cut instructional time for art and music. Maybe it was already less than 154 minutes.

I also like the way they picked as their poster child the North Chevy Chase Elementary School, in Montgomery County, Maryland where over 93% of the children are testing at the proficient level. Why not pick a DC school where over half of fourth graders tested at the "below basic" level on the 2007 NAEP test? Certainly, these illiterate and innumerate kids need to maximize their instructional time in art and music since there's little hope of them graduating with the ability to read above the fifth grade level. Assuming they graduate at all.

Then we have this gratuitous comment:

Fourth-grade teacher Jackie Moore considered it a protest against a decline in public school arts education attributable to budget cuts and a focus on standardized test scores spurred by the federal law.

Whenever some educator claims that there's been budget cuts you can rest assured they are either a) misinformed, b) lying, or c) both. According to school matters not only has Montgomery county not seen budget cuts, their total expenditures have been steadily rising from $11,262 in 2003 to $13,785 in 2005. I'd bet that they're spending close to $15,000 per year now based on that trend.

And then there's this:

Moore decided a half-day of drawing would highlight how little art instruction students usually receive. She thinks that art helps students learn while improving concentration and observation skills but that there's no longer time to have her classes sew colonial embroidery samplers or create Native American jewelry and pottery.

She thinks; she doesn't know. Surely there must be some evidence she can point to in which these artist-students use their superior "concentration and observation skills" to improve academic outcomes in other academic subjects. Otherwise, all they've learned is how to sew embroidery samples and create Native American Jewelry and pottery, two high-growth jobs in the DC area.

This article is bad, even for WaPo.

Update: Kevin Carey has similar thoughts and even stresses a point I made back when the curriculum narrowing debate first surfaced:

The 16% of districts that cut art in favor of reading and math didn't necessarily make a bad choice, unless you think that all districts had, pre-NCLB, miraculously arrived at the precise optimal mix of subjects and time. Reducing time for art in order to ensure that elementary school student can read might be exactly the kind of hard decision those students need.

It's hard to argue that pre-NCLB had the optimal mix considering the rampant amount of academic failure.

AFTie John OTOH apparently thinks that 16% qualifies as "often," though he doesn't provide much of a supporting argument.


Update II: AFTie John responds: "Well, a little extrapolating and back-of-the-envelope arithmetic suggests that 4 million students (16% of ~ 25 million public K-6 students) are missing more than 30 hours of art instruction per year. "

As long as we're pulling a Fermi, my back-of -the envelope calculation is that 8.25 million students (33% of ~ 25 million) are performing at the below-basic level in reading and 5.5 million in math. Are we really getting up in arms that these kids get an hour less finger-painting and macaroni collage time per week in favor of some additional time actually learning how to read and do basic math?

Theory III: Improve Household Incomes

Hot on the heels of the Minneapolis busing failure story, let's take a look at what the data shows with respect to the performance of low-SES attending high-SES school districts.

I ran three regressions for household income versus low-SES student performance. For trial one, I used the percentage of households with incomes greater than $75,000. For trial two, I used the percentage of households with incomes greater than $100,000. And, for trial three, I used the percentage of households with incomes greater than $150,000. All the regressions showed nearly identical results, so I'm only going to discuss the middle trial, trial two.

The theory is that low-SES students will perform better in high-SES school districts. Apparently, these high-SES districts attract better teachers, have better students with better motivation, have more resources, can pay more attention to the small number of low-SES students, and the like.

For this example I'm using community household income as a proxy for high-SES. Specifically, I used the percentage of households having incomes greater than $100,0000 as my measure of high-SES. The greater the percentage of higher income households, the higher the SES of the school district. Let's compare the performance of low-SES students to the percentage of high income households.



Here's the regression results:

• R = 0.13 (there is a weak association between high household income school districts and the performance of low-SES students)
  
• R² = 0.016 (there is a poor fit between the data)
  
• P = 0.007 (the results are statistically significant)

I added a trend line (black line) to the graph, but with a correlation this low, the trend is not reliable. I also added a horizontal line showing the mean of the performance for all school districts in Pennsylvania (mean = 60.7). As you can see, few school districts had low-SES student performance that exceeds the mean score. Low-SES students tend to be underperformers. RWP found that this underperformance was statistically significant.

Interpretation

The mean school district contains about 11% high-income households. This means that if we were to bus all the low-SES students from school districts with less than 11% high-income households to school districts with greater than 11% high-income households, most students would be sent to high-SES school districts having between 11% to about 25% high-income households. That's not much of an improvement and the regression suggests that low-SES student won't improve much anyway. Even if you were to believe the slightly positively sloping trend line, the expected gains in student performance would be minimal.

But there is good reason to question the trend line. Take a look at this graph:



This is a graph comparing high-income households to the fraction of moderately poor households in the school district. I've defined moderately poor households as households with between $15k and $29k of household income. The graph gives the percentage of moderately poor households relative to all poor households (household income less than $29k). As you can see from the graph as the percentage of wealthy households increases so does the fraction of moderately poor households. In other words, the higher-SES school districts have more low-SES students falling in the lower middle class as opposed to the lower or under class.

This is an important finding. The disaggregated data for low-SES (or "economically disadvantaged" under NCLB) students includes students receiving free or reduced lunches. A student can be from a family having a household income approaching $50k and still be included in the low-SES group. But, as the graph above indicates, not all low-SES kids are created equally. The low-SES kids in wealthier school districts tend to have higher household incomes. And, we already know that SES is positively correlated with student achievement.

Many books have been sold by the Kozol/Rothstein clique of SES hustlers who claim that our education woes can be cured if only we could send these kids to high-SES school districts. These claims are founded on the error of ascribing causation to a weak correlation (the top graph) and failing to account for the substantial intra-group differences (the bottom graph) of students defined as low-SES or economically disadvantaged.

It's all based on a lie.

Breaking News: Busing Still Doesn't Work

We learned in the 70s that busing low-SES kids to high-SES suburban districts didn't work.

Minneapolis just re-learned that fact.

For the second straight year, low-income students in the Minneapolis Public Schools fared better than the ones who were bused to suburban schools under the Choice is Yours, a voluntary desegregation program.

I guess they thought they had some new magic school buses. Or maybe the magic aura or the suburban kids grew stronger.

"We're just saying if the number one reason parents sent their kids to suburban schools was academics, they need to look closely and carefully at the results," said Dave Heistad, the district's research and testing director. "Just choice by itself doesn't seem to be the answer."

That's a false choice. The choice is between attending a city school where low-SES students fare poorly or a suburban school district where they also perform poorly.

"This illustrates what we have always suspected," said Minneapolis Superintendent Bill Green. "Whatever frustration people have felt about the Minneapolis schools is based on a sense that we have so much potential and we haven't been able to mine it.

"This data shows that kids don't have to go far away from home to get a quality education," Green said.

Quality education my ass. In the Minneapolis School District, proficiency levels of low-SES students are 56.3 and 39.1 in reading and math respectively.

RWP On the Ed Data

RWP is also analyzing the education data from School Data Direct.

If we look at the maximum and minimum proficiency scores for both variables, we see that although the max schools spent more than the mean for core spending per student, so did the schools with the minimum proficiency percentages. In fact, the school district that produced the lowest proficiencies total and SES spent more money per student than either of the schools that produced the best proficiencies.

I'm sure there'll be more to come.

Denver Goosing Gifted Classes

The racial balancers are at it again--this time in Denver.

More minority and poor students in Denver are being classified as highly gifted under a new system that gives extra credit to children who are economically disadvantaged or nonnative English speakers.

Denver Public Schools is trying to fix a disparity in the program that serves its smartest and most talented students — which up until now has drawn mostly white students in a district that is mostly Latino.

Here we go again.

The article gives us some data to work with.

• More than 1,800 students in Denver Public Schools — about 3 percent — fit the highly gifted classification
  
• DPS's student population is 57 percent Latino, 20 percent white and 19 percent black. But the highly gifted and talented program serves only 25 percent ethnic minorities, Howard said.

Using that data I was able to roughly determine the criteria for the previous DPS gifted program. I assumed normal distributions and that black and Hispanic IQ scores would be able a standard deviation below the IQ scores of white students. I fiddled with the numbers in excel and came up with this:

With an estimated IQ gap of 1.1 standard deviation, the cut-off point for DPS gifted students would be an IQ of about 118.5 for about 25% of the gifted students to be minorities.

This means that about 11% of white students are classified as gifted. But. only about 1% of black and Hispanic students are classified as gifted. This is the result of an IQ gap of 1.1 standard deviation.

IQ's of 118.5 are not exactly Einsteinian, but at least these kids get out of those awful heterogeneously grouped classrooms they are forced to endure.

The meddlers in the DPS don't like that racial balance; they want a racial balance of 33%.

One way to achieve this 33% is to simply lower the cut score until we start hitting a fatter part of the black and Hispanic IQ tail.

I did the calculation for you already. If the cut score were lowered by 6 IQ points (to 112.5) the racial balance would be increased to nearly 34%. Now, nearly 20% of white students would qualify and over 2.5% of black and Hispanic students would qualify.

There is an unfortunate side effect though. At these not so-gifted levels, the number of "gifted" students would double in size and probably break the DPS budget.

So, DPS gets around this little problem by cheating:

To make things more equitable, the district now relies on a sum of measures to determine eligibility into the highly gifted program — cognitive tests, annual assessments, reading tests and teacher nominations. Next year, the district will consider artwork and writings.

Translation: Anything goes.

For example, a student who scores as low as the 75th percentile on cognitive tests could be considered, Howard said. Previously, that child would not have been admitted.

Now, that is overkill. The 75th percentile equates to an IQ of 110. DPS only needs to allow in 170 additional minorities into the gifted program to achieve their 33% balance rate. By my calculations, they'd only have to lower the IQ cut score t0 116.5 to attract this number of students, assuming that all the new students were minorities.

No doubt they're paving the way for the future when they can up the balance even more. I'm wondering at what point they'll stop calling them gifted classes?

"We want to find the gifts that these children have, not exclude them," she said.

It appears those gifts aren't going to be related to cognitive ability.

Update: Alan Gottlieb of Head First Colorado also has a post on this story.

Theory II: Teacher's Salaries

Charles Barone stopped by in my last post and commented:

Spending varies greatly WITHIN districts, between schools. Money follows teacher salaries, and the highest paid teachers tend to be in the schools with the highest SES students.

With the possible exception of the well-funded Philadelphia and Pittsburgh school districts, most of Pennsylvania's school districts tend to be small, usually with one high school and a couple of elementary/middle schools. That's not much room for within district variation. I think we're going to lose much by looking at spending on a district basis, rather than a school basis.

As luck would have it, there is data for Teacher Compensation in the data set. Here's the definition given:

The total amount of money spent on salaries and benefits for instruction, support services, and other elementary/secondary programs. Compensation includes gross salaries and benefits for instruction, support services, and other elementary/secondary programs. Compensation excludes objects not directly related to salaries or benefits, such as purchased services, supplies, and debt expenses. At the district level, this expenditure includes only money spent on students taught in the district. It excludes money paid to other school systems for the education of students outside the district.

So, let's see what the data says about teacher salaries flowing to the high-SES districts. Let's compare teacher compensation to the percentage of low-SES students.

Again, we get a near random distribution. The correlation is weak between teacher compensation and the number of low-SES students in the district.

Now let's test the theory that high teacher compensation correlates with increased low-SES student performance:

Once again we see a very weak correlation. Seems that low-SES students with lower paid teachers perform about as well as those with higher-paid teachers.

I'm sure the causation here is complex. I'd guess that over-paying for bogus credentialism plays a role inflating salaries with little gain to show for it, overwhelming the higher performing teachers that leave the poor districts for greener pastures.

The Data Shows You Are Wrong

This weekend I discovered that S&P's School Data Direct makes all their school data available for download. This represents a cornucopia of school related data we can play with.

Of course, they make it as difficult as they possibly can to use that data, but where there's a will there's a way.

All this week I want to take a look at the performance of economically disadvantaged students and see what the data tells us. Let's see if the various bromides and theories we've been debating for the last few weeks find some support in the data.

I'm going to use the dataset for Pennsylvania 2005. It's not the most recent dataset (that would be 2007), but it is the latest dataset that includes financial and demographic data. I found data for 446 of Pennsylvania's 501 school districts.

Today let's focus on the relationship between school expenditures and low-SES student performance. Hardly a day goes by in which some edupundit will claim that we need more school funding to improve test scores. It's become a kneejerk reaction, especially by those slightly-left-of-center edupundits who still cling to the notion that more money can solve anything. Let's put this theory to the test.

Theory One: Low-SES students perform better in schools with lots of resources. Stated differently, low-SES student achievement can be improved by providing schools with more resources. To test this theory, let's compare the performance of low-SES students on PA's state test to the total expenditures in each school district.

I also calculated:

R - The coefficient of the correlation between the score/actual data and the score predicted by the regression. For R values close to 1, there is a strong association between the variables. For R values close to zero, there is a weak association between the variables. Generally, we don't get to excited until the R values are greater than about 0.5.

R² - the percentage of the variability in the predicted score explained by the variability in the actual data. In other words, how well does the actual data fit the predicted score. For R² values close to one, there is a good fit. For R² values, close to zero, there is a poor fit.

P value - If the P value is less than 0.05 we reject the null hypothesis, i.e., it is safe to assume tht the results are statistically significant.

For this regression the results are:

R = 0.06 (there is a very weak association between total school expenditures and the performance of low-SES students)

R² = 0.0036 (there is a very poor fit between the data)

P = 0.20 (the results are not statistically significant)

I put the trend line in the graph, but there is such a poor fit between the data and the predicted score (the trend line) that the slight downward slope of the trendline is misleading.

Interpretation: I calculated the average score for all PA students on a per school district basis to be 60.7 in 2005 with a standard deviation of
13.1. I also calculated the average score for low-SES students to be 43.0. this represents a large achievement gap of 1.36 standard deviation. As you can see from the graph, it didn't matter whether the school district was spending $8,000 or $19,000 per student, the performance of low-SES remained unchanged with a large amount of variation between high performing districts and low-performing districts across the board. At today's funding levels, I'd say that resources are not an issue. Especially, when you consider that, at least in PA, school districts with lots of (>50%) low-SES students are funded just as well as school districts with few (<10%)

School expenditures are weakly correlated with the number of low-SES students in a district.

In case you were wondering, I also ran the regression with instructional expenditures instead of total expenditures to see if perhaps these districts with lots of low-SES students were squandering their resources on non-instructional expenses. Apparently, they aren't. The correlation between instructional expenditures and low-SES student achievement was even weaker than that for total expenditures (R = 0.03, R² = 0.0012, and P = 0.47).

The data does not support the theory that spending more money on schools will led to improved low-SES student achievement at today's funding levels. the data suggests that even if we doubled the funding to the typical school district, we should not expect to see improved low-SES student achievement.