January 30, 2009

Pennsylvania's High Remediation Rates

The Pennsylvania Department of Education released some data last Wednesday indicating that one in three Pennsylvania high school graduates who enrolls in a state-owned university or community college must enroll in remedial math and/or English courses before they are capable of taking college-level courses.

As usual, Pennsylvania has released its data in a way that makes it all but unusable for analysis purposes. basically, they just tell us the percentage of students in each district that took remedial courses, the number of courses taken, and the amount it cost. Not very helpful. But not to worry, fearless readers I did the heavy lifting of putting the data into my all-purpose Pennsylvania database of education statistics and managed run a few regressions.

It should be noted that there are selection bias issues out the wazoo because PA failed to disaggregate any data. So take that as a large warning in interpreting the data.

Anyway, off we go.

The first regression I ran compared the percentage of adults with bachelor degrees in the district and the percentage of students needing remediation. Adult education level is a proxy for student socio-economic status (SES) (and parental/student IQ). The thought is that students with higher SES levels should be better prepared for college level work. Let's see if that hypothesis holds up. (Note that I've indicated the Philadelphia School District as the alrge red triangle. And, note that Philadelphia generally appears above the regression line indicating that its actual remediation rate is generally higher than its predicted rate.)



No, the hypothesis doesn't hold up. Less than 1% of the variance in parental education level is associated with the percentage of students in need of remediation. We would probably have gotten a better result if we knew the actual parental education level of the students who were in need of remediation since the parental education levels in my database are for every adult in the district (not just parents and not just parents of remedial kids).

Now let's check if the percentage of students receiving free and reduced lunches (another proxy for poverty and low-SES) correlates with the percentage of remedial students.

Nope. Once again the variance is less than 1%. I was somewhat surprised at this outcome. Low-SES students are generally underperformers so I was expecting to see districts with high numbers of low-SES students have higher remediation rates. But that wasn't the case. Maybe the selection bias effects are showing up here in that the low-SES students may be applying to college at lower rates. This analysis would have benefited from the demographic data of the actual remedial students. I'm certain PA has his data. Release the data, PA!

Now let'see if school expenditures make a difference on remediation rates.

Not really. Again the variance is low -- 3.6%. If anything, school expenditures are negatively correlated with remediation rates. The more a district spends the higher the percentage of remedial students it gets. Instead of dreaming up various explanations for this outcome, let's just keep it simple and say that school expenditures don't seem to have much of an effect on remediation rates.

Now let's look at whether there is a correlation between the percentage of students failing PA's NCLB test (the easy minimum-skills PSSA test) and the remediation rate.

(Note: the x-axis caption is incorrect. It should read: % failing state test)

Again, the variance is somewhat low at 8.7%. Schools with higher PSSA pass rates produce slightly lower remediation rates. I guess that's not too surprising. Students who pass the PSSA probably know more and as a result are less likely to require remediation. This is another analysis that would have benefited greatly from knowing the percentage of remedial students that passed or failed the PSSA exam.

These results aren't terribly interesting since they basically are just showing low correlations. But the following regression is a bit of a doozie.

Let's take a look at the percentage of non-white students and the remediation rates. Note that in PA, the number of Asian and Native American students is very low, so you can read non-white as mostly black and Hispanic.

Hello! Finally a decent correlation. 25% of the variance in remediation rates is associated with race. The more black and Hispanic students in the district the higher the remediation rates.

I'm wondering why the PA Dept. of Ed didn't highlight this inconvenient piece of data. PA schools continue to do a poor job preparing black and Hispanic students for college. PA schools with high percentages of black and Hispanic students fared poorly with remediation rates. Yet, oddly, schools with high numbers of low-SES, regardless of race, didn't fare much worse than schools with low percentages. And, spending does not seem to affect the remediation rates, if anything the more schools spend the higher the remediation rates.

I'm not sure what to make of all this, but PA could have done a much better job analysing and presenting the data. But, i suppose it's not in its best interest to do so.

6 comments:

Anonymous said...

Ken says,"they just tell us the percentage of students in each district that took remedial courses, the number of courses taken, and the amount it cost. Not very helpful."

"Not very helpful" is the understatement of the day. But providing ANY info on this front puts PA "above proficiency."

Good Work badge for you though, Ken! You squeeze as much blood from the turnip as is possible. and the regression analyses yield useful info.

The thing is, this whole notion of "remedial courses" really sucks. And so does the horse that's now pulling the cart: "Response to Intervention." Mis-instruct kids until you can dump them with impunity. And who gets "remediated" the most? Look at your last analysis.

We're not in a "post racism era" yet, are we. But we didn't need the regression analyses, the PA data, or NCLB to know that. It's been known throughout the history of the US.

The question is how to eliminate the need for "remediation." Yes we can! But whether or not we will remains to be seen.

Anonymous said...

"It should be noted that there are selection bias issues out the wazoo because PA failed to disaggregate any data."

I think I would be more worried by the self-selection caused by the good students going to different universities than the poor ones.

I'm at a bit of a disadvantage here, because I don't know any of the *public* Pennsylvania colleges.

From www.passhe.edu, I get a list of 14. The only one I recognize is Slippery Rock (and for its sports teams, not for academics).

In California, we have public universities like Cal and UCLA that can and do attract a lot of very good students [Michigan has UMich at Ann Arbor]. Below our UC system we have the CalState system, and below that we have our community colleges.

If Pennsylvania doesn't have anything equivalent to California's UC system (and especially a few very good campuses), my fear is that many of the non-remediating students won't show up because they will go to out of state universities or to private universities.

Any idea how big an effect this is?

-Mark Roulo

KDeRosa said...

Dick, I'm not sure racism is implied here (not even sure if you are trying to imply this).

Mark, PA has Penn State which has the main campus which most students go to for their last two years and a bunch of satellite schools which feed into the main campus and in which most students attend their first two years of college. Only the most capable students go to the main campus for years oneand two. And, it is a well respected system.

Also, when I say selection bias I mean to include all the students who selected themselves out of the public college system and into private cooleges. So, I'd characterize the students who selected themselves into the public school and community school system as mostly not the best students (who selected a private school) and not the worst (the ones who didn't attempt college or who dropped out of high school).

Anonymous said...

Okay.

I've heard of Penn State ... I wonder why the link I found didn't mention it ...

-Mark Roulo

Anonymous said...

California (and NY, which has a similar system) is the exception, not the rule. Pennsylvania, too, but because Penn State has no large state school competition. There are 24 campuses, with 42,500 enrolled here at the flagship campus, and 90,000 students statewide.

Anonymous said...

FYI:

As per choice, Penn State is NOT a public university. There are only 14 public universities in PA. PennState only considers itself "public" when it comes to siphoning off funds set aside for true public schools. Otherwise, they feel that it doesn't benefit them in the least. PSU is the largest school in PA, and also one of the largest employers here also. They make their own rules. BTW...this study was conducted by PSU, the PA Dept. of Ed. contracts all research to Penn State (at a sizeable sum of money I might add).