June 2, 2008

Does money matter in education

Over at the Quick and The Ed, Kevin Carey writes:

I'm not one of those people who believe that "money doesn't matter" in education. That's absurd; money matters a great deal, and there are plenty of schools that don't get their fair share.

While I tend to agree with Kevin Carey on education policy matters (at least the non-financial ones), I tend to disagree with all of his non-education policy posts over at The Quick and the Ed. And since I try to focus this blog on education policy, let's leave it at that. (I also don't mean to single Kevin out since this is a common meme in the edusphere.)

Nonetheless, the "money doesn't matter" meme is not as absurd as Kevin is suggesting, provided that it's qualified with "at today's funding levels."

At today's funding levels, money doesn't matter. The correlation between school expenditures and student performance is very low to non-existent.

The fact that some schools "don't get their fair share" is irrelevant. The important question is whether schools are getting sufficient funding to educate their students. No one knows the answer to that question. What I do know is that many schools with low funding outperform many schools with much higher funding.

Let me cherry-pick some data points like Kevin did in his School Funding’s Tragic Flaw study.

From the 2005 Pennsylvania dataset from SchoolDataDirect, I found ten high-performing/low-funded school districts and ten low-performing/high-funded school districts based on 11th grade test results and total expenditures. The districts had similar poverty rates and minority attendance. Here's a graph of those 20 schools.



None of the high performance/low expenditure schools got their fair share. On average the low-performance/high-expenditure schools had $6880 less in funding (44% less) than the high-expenditure schools. That's not fair, now is it? Of course, it didn't stop these schools from performing over a standard deviation better than these high-expenditure schools. Perhaps if the high-expenditure schools received less funding they would have performed better?

Not content to merely cherry pick schools to "prove" my point, I decided to take a closer look at the dataset and focus on the performance of the economically-disadvantaged students.

First I looked at the performance of these economically-disadvantaged students in the best (top decile) schools. The average proficiency was 66.7%, well above the state-wide mean of all students (59.1%). These schools had expenditures averaging $11,640.

Then I looked rate at the performance of economically-disadvantaged students in the worst (bottom decile) schools. The average proficiency in the worst schools was 20.1%, well below the state-wide mean of all students. These schools had expenditures averaging $11,926.

Similar expenditures, vastly different outcomes.

Then I looked at it the other way.

I looked at the performance of economically-disadvantaged students in the best funded school districts (top-decile). The average proficiency in these rich schools was 41.1%, well below the state-wide mean of all students. The average expenditures for these schools was a whopping $16,260.

Then I looked at the performance of the worst-funded schools. The average proficiency in these poor schools was 45.1%, well below the state-wide mean of all students. The average expenditures for these schools was only $8,604.

You read that right. The economically-disadvantaged performed about a quarter of a standard-deviation better in the poorly funded schools. To put that in perspective, that's about the differential you'd expect from reducing class-sizes or hiring more effective teachers. (See here.) But, in this case, it worked the opposite way.

You would think that these highly funded schools pissed most of that funding away on facilities and other window dressings, but surely some of it went to reducing class-sizes and hiring more experienced teachers -- two of the more favored educational panaceas. But, if it did, it still didn't work as the literature suggests it might. Which shouldn't be surprising if you are familiar with the way monopolies work, especially state-run monopolies.

You'd be hard-pressed to torture the data in a way that permits you to conclude that "money matters" at these funding levels.

18 comments:

ZZMike said...

The title is compelling, because we have an election tomorrow in California, in which yet another school bond is being voted on.

Even though it's a local bond, I still insist that giving money to school adminitration is like giving money to third-world dictators.

They keep it, the people see none of it. (As in Burma today, where one of the government spokesmen cried "we need money, not chocolate".)

The problem is where the money goes and what they do with it. Neither money nor computers by themselves can improve educational outcomes. But wise people, with a little money and a few computers, can make a huge difference.

But even there, youcan only work with what you've got. You need to have students with some interest in learning. Supportive parents would also be a plus.

ms-teacher said...

While I understand that more money doesn't necessarily improve education, my bigger issue is that students shouldn't have to learn & teachers shouldn't have to teach in unsafe/unsavory environments.

KDeRosa said...

I agree, but I don't think that this is a financial problem. It's a leadership/mismanagemnet problem.

educatorblog said...

You can't look at the impact of funding without doing sophisticated multivariate regressions over a large sample size. Here's why: funding probably matters more for schools with at-risk populations. Schools with low levels of students who need counseling, reading interventions, special education classes, second language help, etc. don't suffer as much when budgets are cut as schools and districts who have to provide these services to every student. A statistically sound study about the importance of funding would control for all of these variables (from class size to teacher preparation and poverty level). You also have to realize that correlation is different than causation.

You've inspired me to look through peer reviewed journals and compare research on the topic (or use my economics degree and state/federal datasets to do analysis of my own). Look for a blog post within the next week.

KDeRosa said...

educatorblog,

Let me save you some time. Back in March I ran a regression on the performance of low-SES students with school expenditures for a large dataset (N = 445). The R was 0.06. See here. The correlation was weak.

I actually ran a bunch of regressions on many of the variables you listed back in March and found weak correlations across the board.

So, right off the bat I am not convinced that many of these variables need to be or should be controlled.

That's why I like my ham-fisted counterexample analysis. The existence of high performing/low expenditure schools shows that most schools are already wasting resources.

The question remains whether these high performing schools can be improved by increasing their expenditures. A controlled experiment might tell us something here, a regression won't.

Tracy said...

You also have to realize that correlation is different than causation.

However, how worthwhile is it investigating causation when there's bugger-all correlation? For this reason, we can look at the impact of funding without doing sophisticated multivariate regressions over a large sample size.

Paul B said...

Nationally we're spending something like $153,000 per teacher on public education. So if this were a charity operation you could argue that we are delivering services at something like 33% efficiency (assume teacher pay at 50k).

Could anybody in their right mind argue that pouring money into this leaky bucket allows one to carry more water?

Anonymous said...

Regardless of school funding and socio-economic environment, parental/family involvement in a student's life directly affects academic outcome. When a parent makes school the priority (yes, even over sports) with strict rules and ramifications for poor work, the student focuses on a stronger outcome. We all have witnessed students overcome economic obstacles and excel academically. Ninety nine times out of a hundred, there is a strong adult presence in the child's life guiding him/her towards success.

A study performed by the ACT organization found that college professors regard "the fundamentals" as THE prerequisite for success in college. Yet, their high school counterparts believe a "well rounded education" is most important. State test scores are not based on jewelry making or graphic design. Reading comprehension, math, critical thinking and writing makes the student. When families make learning the priority we will begin to see growth. Until then, you can build a separate magnet school for every individual child in America and it won't do any good. Especially if the parents are more concerned about arriving for hockey practice on time rather than the child finishing (correctly) their schoolwork.

Tracy said...

Regardless of school funding and socio-economic environment, parental/family involvement in a student's life directly affects academic outcome.

This of course is the situation we would expect to see if schools did no teaching at all. For example, back before girls schools were started, the only way girls learnt reading/writing/etc was if their parents taught them or hired someone to teach them directly.

To put this another way - we observe and we see that parental involvement is vital for children's academic achievement. There are two possible explanations for this:
1. Parental involvement is fundamentally essential, no mattr how good the schools are.
2. Schools are incompetent (or non-existance) so parental involvement picks up the slack.

The results of the Direct Instruction curriculum in Project Followthrough indicate that children's achievements can be drastically improved without doing anything to increase parental involovement. And improving the performance of schools strikes me as an easier job than improving the performance of parents.

Mr. McNamar said...

I wish I had more understanding of how federal, state, and local money mixes and is ultimately distributed.
What I do know is that many districts waste a good deal of money on middle managers at the central office.
But at the same time, not all schools are funded equally, and that is a problem. It is irresponsible to suggest that money doesn't matter, especially when we say public education.

Kim said...

I read part of Kevin Carey's response http://www.quickanded.com/2008/06/money-matters-in-education.html

This jumped right out: "There is, for example, a long-established multi-billion dollar private market for K-12 education characterized by significant price variance. Here in DC you can spend a relatively modest amount of money to send your kids to Catholic school or several times that for tony private academies. Presumably the people forking over $31,428 per year (!) for St. Albans aren't idiots who are getting ripped off, but rather smart, well-educated consumers who are getting something for their money besides just peer effects."

That is the ultimate apples to oranges comparison. Public education is supremely fouled up because people cannot pick up and move their money when the schools don't perform, unlike in the private sector. When the public has control over their own funds, they can even decide if they think the $31K is worth it or if they think $2K--with whatever it may or may not include--is just right. I don't think we'll ever get the right incentive until there's a big enough punishment. One punishment is firing (impossible to do to teachers and not nearly as effective for principles) another is taking away funding--which puts me smack dab into the "privatizers and anti-government zealots (to be clear, I'm not putting Ken in this camp) who use it as an excuse to promote reckless tax cuts and disinvestment in public educaiton."

Tracy said...

Here in DC you can spend a relatively modest amount of money to send your kids to Catholic school or several times that for tony private academies. Presumably the people forking over $31,428 per year (!) for St. Albans aren't idiots who are getting ripped off, but rather smart, well-educated consumers who are getting something for their money besides just peer effects."

I suspect that what the people who are sending their kids to St Albans are getting for their $31,428 is the very useful signal that they can afford to spend $31,428 sending their kids to St Albans.

After all, $31,428/year is not an outrageous amount to spend on signalling. I know a woman who spent $14,000 on her wedding dress. Paintings regularly sell in the $50,000 to $100,000 range. You can buy sports cars for more than $100,000. How much does high-end jewellery cost?

When I was a kid, my Mum went into business with a friend who moved in very rich circles (by NZ standards) and so I was dragged along to a fair number of bbqs and what not, where in turn I encountered a fair number of kids who were definitely impressed by the amount of money their parents were spending on their schooling. And felt no hesistancy about letting others know about it.

Anonymous said...

". . .children's achievements can be drastically improved without doing anything to increase parental involovement."

As much as you think parental involvement has little to no bearing on a child's learning, parents' attitude toward education greatly affects academic achievement. Society places greater emphasis on sports and entertainment than education. Parental "bragging rights" greatly influences the household's focus. The basic statistic of 'children from families whose parents graduated college are more likely to attend college' proves the point. Only 27% of working age American adults has a college degree, which means 73% do not. With such a high percentage of parents having not attained a college degree it is no wonder why most students are not focused on academics.

Visit a ball field on weekends (or weekday evenings for that matter) or the hockey rink and you will find dedicated parents rooting on their child to inflict damage on another. Do we witness that same enthusiasm with education?

Owning a tutoring business, I see(and hear) a great deal regarding parental involvement in education. Students whose parents are committed to their childrens' academic success perform to a higher standard than those whose parents believe academics interferes with athletic interests (and parents' social life).

Affluent families' willingness to pay for private schooling and academic enrichment do so from the understanding that a strong education greatly increases the probability of economic self- sufficiency. That is not to say success is solely determined by level of education, but there is a strong correlation.

Tracy said...

Students whose parents are committed to their childrens' academic success perform to a higher standard than those whose parents believe academics interferes with athletic interests (and parents' social life).

Of course. If schools are ineffective, then the majority of kids who succeed will be the ones whose parents step in to do the school's job.

I'm sick of people responding to criticism of schools by saying that parental involvement is vital. Don't they realise that's exactly what we would see if the schools *are* incompetent, or indeed didn't exist at all? Before anyone can defend schools' poor outcomes by saying that parental involvement is vital, they first have to first show that the schools are competent. Otherwise the evidence is completely compatible with the hypothesis that schools are incompetent. It never apparently occurs to anyone who trots out that line though to trot out that evidence. This indicates that neither the defender's school nor their parents managed to teach them about hypothesis testing.

Society places greater emphasis on sports and entertainment than education.

What's the GDP spend on each of these things?

The basic statistic of 'children from families whose parents graduated college are more likely to attend college' proves the point.

Yes. If schools are incompetent, then the better the educational achievement of the parent, the more effectively they can teach their kids. If schools can't prepare kids for college, then if your parent who has been to college then you have a massive advantage over the kid whose parents hasn't been there.

Again, saying that kids whose parents went to college are more likely to go to college than kids who didn't is entirely compatible with the hypothesis that most schools are incompetent.

Visit a ball field on weekends (or weekday evenings for that matter) or the hockey rink and you will find dedicated parents rooting on their child to inflict damage on another. Do we witness that same enthusiasm with education?

Have you seen how much people pay for college? As for rooting on from the sidelines, how many teachers or tutors want that literally while teaching?


Affluent families' willingness to pay for private schooling and academic enrichment do so from the understanding that a strong education greatly increases the probability of economic self- sufficiency.

And also from the failure of many public schools to provide a strong education. (Though I certainly have my doubts about the quality of the tutoring and private schooling - I don't know of any good evidence that the average private school provides a better education than the average public school for students of a given socio-economic level. Of course you are not an average, this is not a criticism of your tutoring business.)

Anonymous said...

Tracy,

Never did I say that schools were perfect. In fact, we could all write dissertations regarding the ineffectiveness of the public school system. However, even when I was a child (I am 46), my parents were engaged in my schooling. There are not enough hours in the school day for a teacher to realistically reach all 20+ students and keep them all moving ahead at the same pace.

There are few teachers that don't want to see their students successfully progress through school. However, as long as you have parents defending their out of control child who is disrupting the entire class, or whose child consistently does not prepare for class, there will be issues in successfully completing the course curriculum. I have several students here that are frequently in detention or creating problems in class. Most are from divorced families where both parents are vying for the child's affections and therefore, refuse to discipline properly.

As for a GDP analysis - that simply does not make sense. I am not saying parents spend more money on sports and entertainment - only more time. (If you look into how many students seek additional academic assistance vs. how many play sports, etc. I am certain you will see the latter much more heavily weighted.) By the way, do you want to compare sports and entertainment salaries to teachers? There is no question where society places greater emphasis.

Today's teachers are greatly impacted by increasing state and local requirements. Parents demand from the boards of education additional programs - but none of the existing ones are ever taken away. It is a continuous cycle of growing responsibility with no additional time for instruction.

Third, no, I do not want parents cheering in the classrooms. However, most people I speak with agree that parents place greater emphasis and speak more frequently about their children's athletic and extracurricula successes than educational achievement. It sends a direct message to the child that mom and dad care more about sports than education.

Also, yes, I do see how much parents pay for college. I also see that there is a far smaller percentage of high school graduates attending college than those who do not go to college. So the enthusiasm for education (with regard to the outlay of money) is from a minority of parents. That is why, according to the U.S. Dept. of Ed. only 28% of working age adults has a college degree.

Lastly, you may want to research the percentage of incoming freshman at Ivy League colleges that graduated from prep schools. My understanding is that approximately half the Ivy League student body comes from prep schools. Yet, prep school enrollment is significantly lower than public schools, leading me to believe that although one may not believe that private schools provide a better education, the Ivy Leagues think otherwise.

Tracy said...

Never did I say that schools were perfect.

You did however attribute the failure of kids to a lack of parental involvement, without bothering to question whether failure of kids could be due to incompetent schools.That's the argument I was criticising.

There are not enough hours in the school day for a teacher to realistically reach all 20+ students and keep them all moving ahead at the same pace.

And indeed the most successful educational programme I know of, Direct Instruction, does not attempt to keep all the students moving ahead at the same pace. Instead it splits the students up into smaller groups and places them in the lesson curriculum according to how fast each individual kid can learn.

It is realistically possible for a *school* (note, not merely a teacher by themself, a teacher needs support from the school adminstration and from other teachrs) to reach about 95% of its students, perhaps more, and keep them all moving.

As for a GDP analysis - that simply does not make sense. I am not saying parents spend more money on sports and entertainment - only more time.

Your claim was that *society* values sports and entertainment more than education. Not parents. That's why I asked for the GDP figures.

(If you look into how many students seek additional academic assistance vs. how many play sports, etc. I am certain you will see the latter much more heavily weighted.)

Firstly, if you look into how many hours students spend at school versus how many hours they spend playing sports, I am certain you will see the former much more heavily weighted.

Secondly, it's not surprising to me that kids would spend more time *out* of school playing sports. School is for academic stuff, free-time is for running around exercising the body. The body should be taken care of as well. Plus, if a kid enjoys sports, why shouldn't they do that with their free time?

By the way, do you want to compare sports and entertainment salaries to teachers? There is no question where society places greater emphasis.

Of course there is a question as to where society places greater emphasis. I'm questioning right now. Do you want to compare total salaries paid to teachers compared to total salaries paid to sports and entertainers? The sports and entertainment sector is weird, there are a few superstars making massive fortunes, a larger number of people who manage to make a living, and a vast number of people who can't afford to give up the day-job. Meanwhile education absorbs a larger amount of money in total. From http://www.bea.gov/industry/xls/GDPbyInd_VA_NAICS_1998-2007.xls, Compensation of employees in the sector "Educational Services" totalled 109,068 million dollars, while compensation of employees in the "Performing arts, spectator sports, museums, and related activities" totalled 36,839 million dollars, 33% of the compensation to education. Society places a greater emphasis on education than on sport and entertainment.

However, as long as you have parents defending their out of control child who is disrupting the entire class, or whose child consistently does not prepare for class, there will be issues in successfully completing the course curriculum.

There will always be issues successfully completing the course curriculum. Teaching involves humans, so it's never going to be issueless. If teaching was issueless, we could turn it over to a bunch of computers.

On the topic of discipline and educating kids whose parents won't discipline them, there was a useful series of posts on this blog a while ago about discipline. Start with http://d-edreckoning.blogspot.com/2007/10/how-to-effectively-manage-classroom.html.

Parents demand from the boards of education additional programs - but none of the existing ones are ever taken away. It is a continuous cycle of growing responsibility with no additional time for instruction.

So perhaps parental involvement is not an unmitigated good?

However, most people I speak with agree that parents place greater emphasis and speak more frequently about their children's athletic and extracurricula successes than educational achievement.

And how often do schools talk to parents about their kids' educational achievements? A kid has a chance to win on the sports field every week - what chances for academic success do schools offer like that? And how many kids are achieving at school? Parents like to boast about their kids (my parents never boasted about my athletic successes for the simple reason that there weren't any:) ), if the kid is getting poor report cards, what is there to boast about when it comes to academics?

Also, yes, I do see how much parents pay for college. I also see that there is a far smaller percentage of high school graduates attending college than those who do not go to college. So the enthusiasm for education (with regard to the outlay of money) is from a minority of parents. That is why, according to the U.S. Dept. of Ed. only 28% of working age adults has a college degree.

I read that only 28% of working age adults have a college degree, and wonder if that is possibly because only 28% of school students have been getting a good enough education from their high schools to be able to attend the high school of their dreams. Before you can fairly blame the relatively small percentage of school graduates attending college on societal disinterest in education, you have to exclude the alternative hypothesis - which is that most schools are incompetent.

Lastly, you may want to research the percentage of incoming freshman at Ivy League colleges that graduated from prep schools. My understanding is that approximately half the Ivy League student body comes from prep schools.

*My* understanding is that prep schools are expensive. Therefore the kids going there are the ones who are from higher socio-economic backgrounds in the first place. In public schools, a consistent result is that kids from higher socio-economic backgrounds on average do better academically. Therefore, I would expect the kids whose parents can afford to send them to prep schools to be the ones who would be far more likely to get into the Ivy League schools even if they went to public schools.

Anonymous said...

"By the way, do you want to compare sports and entertainment salaries to teachers? There is no question where society places greater emphasis."

I'm not sure how you can draw many conclusions from this. As an example, the US has about 3M K-12 teachers. They make, on average, probably around $35K per year (plus benefits). We can compare these 3M teachers with the *best* 750 or so professional baseball players and find that the baseball players average a few million dollars per year. *BUT* ... below these 750, we also have the players in the minor leagues (who tend to make a few thousand dollars per month for six months a year).

The median K-12 teacher makes *much* more than the median professional baseball player.

The same holds true for pretty much all other sports as well as for music and acting.

What we find is that society is willing to pay the *best* entertainers *HUGE* sums of money and the rest very little.

We *could* do the same for teaching ... take the $500B or so per year and divvy it up so that the best 5000 teachers go 90-95% and the rest got around minimum wage. But so what?

-Mark Roulo

Anonymous said...

"I am not saying parents spend more money on sports and entertainment - only more time."

And yet the typical K-12 student spends about 1000 hours/year at school. Maybe more if we count homework. The parents will be paying thousands of dollars a year in taxes to support this schooling.

Maybe it is rational to then spend the next block of time and money on something else ... like sports.

"(If you look into how many students seek additional academic assistance vs. how many play sports, etc. I am certain you will see the latter much more heavily weighted.)"

Which proves that K-12 students will choose to do things that are fun instead of things that are 'good for them.' This is not news.

But we still have 1000+ hours/year spent in school. After this, I don't see how budgeting time on something else has to be a bad choice.

If it is, at what point does it become reasonable to not spend the next dollar and hour on education?

-Mark Roulo