March 30, 2009

Parental Involvement in Low-Income Areas

Uncle Jay has a good column on how the need for parental involvement in low-income schools isn't all that critical.

The story of his school and others like it suggests that the importance of parental involvement, at least in the most disadvantaged neighborhoods, has been exaggerated, probably because middle-class commentators have been imposing their suburban experiences on very different situations. Unchallenged, this misunderstanding of what works for low-income children could stymie efforts to improve the country's worst schools.

The best school leaders say that they don't need much parental involvement when they are hiring staff, creating class schedules and putting discipline procedures in place.

...

Low-income parents may often be distracted just trying to make a living, but they know what works. Once they see a school keeping its promises, they provide the kind of support found in suburban schools. But it's important to remember that good schooling must come before parental support, not the other way around.

That seems about right to me. Parents need to make sure their kids get to school timely and regularly. But then the school needs to demonstrate that it is doing its job. Or why would parents think its important to send their kids to school every day?

38 comments:

mmazenko said...

Overall, this is very hard for many people to accept, as involved parents think that the most important component of a child's success is parental involvement. They are always shocked when I point out the greatest predictor of success is socio-economic status. Now, I know Jay's focus is not on affluent schools. However, the reality is that affluent schools are generally "better." And many poorer parents are incapable of helping their students succeed, having been struggling students themselves.

Thus, parental involvement is an important factor, but not the most important. Quality schools/teaching is the key.

Unknown said...

Affluent schools are not "better", which is why you used the scare quotes.

Affluent schools are populated by the higher SES status families like you mention in the beginning of your comment, thus producing higher scores with the same teachers.

You both make the point that it is SES and parental involvement in childrens lives that makes the difference, then you go on to say that it is apparent, then, that teachers are the most important factor. Huh? Which is it? Parents involvement in their child's education, or the teacher?

Libby Maxim said...

since parental involvement and SES are impossible to control, whining over either is counterproductive,

better to build a better mousetrap, in that we need to create instuctional tools that are idiot proof so to speak,

make an instructional tool that delivers reliable results with all teachers and all students

and yes, this has been done, google the term SWRL and or read Dick Schutz's posts

but all this time wasting talk about teacher quality, SES, poor schools etc could be negated with better instructional tools

and i do not mean each teacher having his own tool bag, this then gets teacher dependent

make instructional tools the way we build things in industry

make em, road test em and then FIX them

instead of saying , well this works for this type of learner and not the other
well then, that instructional tool sucks

throw it out or fix it so that the instruction is replicable across the board

libby

KDeRosa said...

I'm going to have to agree with Libby in this one.

We have to take children as we get them and do the best job that we can regardless of their IQ, SES, etc.

Most schools do at least an adequate job with their high-SES populations and a less than adequate job with their low-SES populations. How we view the school's performance depends heavily on how many low-SES students the school has.

mmazenko said...

TFT,

My comment was narrow, thus the appearance of contradiction. Sorry, I was making a simple point about overestimating parental support when SES is so important. Otherwise, critics tend to simply say the schools are poor because poor people ignore their kids and don't make them work hard or do homework. That's a big dis-service to the discussion

Affluent kids come in with higher educated parents and higher vocabulary and expectations and cultural knowledge, as well as more classmates with similar qualities leading to a more "school success" oriented climate. Their teachers will more likely be content-skilled with better credentials and history. Chances are they just have better teachers and better resources, and will subsequently be more successful.

Thus, it's everything I mentioned is key. I didn't say one was most important, though I noted higher SES is the most significant "predictor." That's true for a myriad of complex reasons which I and Jay Matthews and others allude to.

Dick Schutz said...

Once again. Predictions do not constitute causes.

One could draw the same faulty conclusion about SES and medical/health services. But we don't.

The best database available, which few people look at as they chase "better data warehouses" is the ECLS-K.

There ARE cultural differences in how racial and SES categories interact with preschoolers, but the differences are not nearly as great as generally assumed. For example, with few exceptions, all parents or caregivers read books to little kids. So more than others, but these specific practices are NOT predictors.

Some kids DO enter K, "behind" but K teachers, in 1998, spent the largest amount time in reading instruction on teaching kids the names of the letters of the Alphabet, whether kids knew the names or not, and despite the fact that letter names per se have nothing to do with reading. By the end of the year, all kids knew letter names, but they had learned virtually nothing about letter/sound correspondences.

Any child who can speak in full sentences and participate in everyday conversation has the prerequisites to be taught how to read. But not with prevailing instruction.

I view parental involvement in their child's schooling as a good thing--but good in it's own right, not as an replacement for acountability and responsibility at the top of the ed chain.

It is the rare parent who doesn't take satisfaction in their child's accomplishments. But that's now what we show them. The talk focuses on child and parent "deficits," on "dumb" homework, and on contributions of time and money that are a hardship for many parents.

More and smaller reboots of the "system" are needed.

mmazenko said...

Libby,

The discussion is purely academic and intriguing for that reason - talking things out, I think, helps keep all factors in mind.

That said, I completely agree with you. It speaks to the issue of core competencies and how we get all kids where we want them to be, or where they want to be, keeping in mind that kids will and should be arriving at many different places - all ready for a four-year college and bachelors degree is unfair, unrealistic, illogical, and a waste of time and resources.

Thus, those "tools" should be relevant and adaptable to all student populations. In a side note, this would alleviate concerns about voucher programs that leave many schools with all the struggling kids. (I see the criticism coming on that one, by the way).

If we were more rational and pragmatic about what we want kids to accomplish, we could more easily identify those "tools."

Unknown said...

For a hundred years we've been trying to build a better mousetrap--to no avail.

Since you have given up on providing what parents need to be good parents (a lack of poverty, etc) you have nowhere else to turn but to schools.

This is flawed logic; trying to fix a problem by addressing things that do not impact the problem won't work. Never has.

And if you want teachers to be held accountable, you'd better attach risks to each population, and provide "handicapping" for the differences in intelligence, drive, and whatever else schools can't control but effect a child's ability to thrive in school.

If children are different and have different abilities, interests, proclivities, talents, maybe we shouldn't be trying to come up with ONE SIZE FITS ALL types of fixes.

Both mazenko and libby seem to discount differences in favor of onesizefitsall, yet they acknowledge differences. Very confounding!

KDeRosa said...

TFT, I think you are have some flawed logic.

We've also been trying to build a better mousetrap to solve the problems that go along with low SES to no avail as well.

Lack of poverty is not a solution to the low SES problem.

One of the primary problems with low SES households is the low language levels that are present which seriously impact the language abilities of students from these households. This is a problem that will not be solved by financial assistance. Moreover, schools continue to assume that all students are supposed to come to school with the language abilities of a middle-class child and that simply isn't the case. Few schools even attempt to remediate this problem.

Also, most attempts to build a better mousetrap in schools were not, and continue to be not, very good. But, there have been some better moustraps developed at the primary level and they have largely been ignored.

There are greater risks associated with educating low-SES students and those risks should be accounted for in any scheme that holds educators accountable.

Lastly, there's nothing wrong with one size fits all solutions provided that the solutions address the numerous similarities of students

Dick Schutz said...

Good points, Ken.
However:
"There are greater risks associated with educating low-SES students and those risks should be accounted for in any scheme that holds educators accountable."

Well, yes and no. Certainly, if you have a choice between being rich or poor, the keyed response is "rich."

A new book by Richard E. Nisbett, Distinguished Service University Prof at U Michigan, throws light on this matter.

"Intelligence and How to Get It: Why Schools and Culture Count"

There's a good NYTimes review:

www.nytimes.com/2009/03/29/books/review/Holt-t.html?8bu&emc=bua2

The nub for present purposes is that general ability doesn't differ by race. But if kids are not taught how to read, do math, and gain other academic expertise, that puts a lid on their general ability.

Low SES does not put kids at instructional risk. Mis-instruction puts them at risk. The current focus of instructional accountability is misplaced. As in every other sector of life, the focus should be at the top rather than at the bottom of the enterprise.

"there's nothing wrong with one size fits all solutions provided that the solutions address the numerous similarities of students"

It's certainly true that Edland has focused on differences while dismissing the similarities among kids. However, "the one size fits all" should reference the task rather than the kids.

There is more than one way to skin a cat, but all involve reliably separating the animal and the skin. And the way that is most efficient and effective will prove to be so across the board.

The notion that a teacher can mix and match instruction "to meet the needs of the individual child" does nothing more than to give a teacher license to inadvertently mis-instruct.

mmazenko said...

TFT,

I absolutely oppose a one-size-fits all system, and, in fact, I meant to write that in my last post, and regret not doing so. However, I was very clear about assuming the same end result for all students as unfair, irrational, and inefficient. I've argued against singular systems with singular ends for most of my career.

It seems you've misinterpreted the broad view of education reform I tend to take. That's probably because these short posts don't attempt to do everything or foresee every possible criticism. That said, my motivation is only what is best for the child at whatever level in whatever system. Nothing is off the table in my view.

I would argue that we haven't reformed to "no avail." There are many positives for a great majority of our student population, and as Dick and I have argued to Ken, we don't need a re-boot, as much as some realistic tweaking of a reasonably functioning system.

I would argue that it's a pretty practical statement that affluent schools are better schools than many of the poor urban ones. Jay Matthews will regularly cite numerous urban schools around Denver on his Challenge Index of "Best Schools." However, parents who can choose where they want their kids to go don't choose GW over Cherry Creek. They give a realtor boundaries of CC and say they want to live within them because they have better schools. By almost every measure they are right.

I've noted moves by Adams 50 to move away from grade levels to competencies because it works for their population. I in favor of charter and magnet schools that work outside of standard rules to the benefit of a particular student population. In fact, my son goes to one. I want to offer graduation at sixteen (like NH and much of Europe and Asia) for students to seek associates programs and trades if they desire.

I'm definitely in agreement with you on a desire to avoid a singular model.

Dick Schutz said...

Until we pin down what is meant by "model" and take "one size doesn't fit all" out of the realm of pejorative metaphor, dialog using these terms is empty.

Several scenarios MAY lead to the reliable accomplishment of a specified instructional aspiration. But those scenarios are typically much smaller in number than is commonly thought.

But without pinning down what want to accomplish and how we would know when the accomplishments have been made, the enterprise is "out of control."

Tracy W said...

I would argue that it's a pretty practical statement that affluent schools are better schools than many of the poor urban ones.

This vitally depends on your definition of "better". It is entirely arguable that many affluent schools are better because of their student intake, not because of anything the school itself does.

The NCLB data is showing that a lot of affluent schools suck at teaching poor kids about as much as poor schools. See http://d-edreckoning.blogspot.com/2007/09/poverty-nclb-and-excuses.html for an example provided by Ken himself.

Anonymous said...

I would say that high SES schools generally have better TEST SCORES. They may also have better teachers (but probably not better curriculum), but not necessarily. What they do have is a high number of kids who arrive at school with very high readiness factors, either because they have come from good preschool programs and/or they have been well-prepared at home, usually both. As their schooling continues, they continue to receive tutoring and enrichment (and teaching/reteaching as necessary) at home. The test scores are as much a reflection of the home/SES factors as they are of in-school factors.

In low SES schools, if the kids don't get what they need at school, they don't get it at all, because the parents are unable to provide it.

In other words, parent involvement in the sense of overcoming poor in-school instruction is much more likely to happen in high SES schools. Having had kids at such schools, I know that the parents provide a large chunk of the total academic input.

Teach3rd said...

I would argue that it's a pretty practical statement that affluent schools are better schools than many of the poor urban ones.

That is not necessarily true at all. The schools in my district are extreme. They range from neighborhood schools where the mean family income is $540,000 to ones where the mean is $15,000. Some would be surprised to learn this, but the wealthier schools don't always do better on the standardized tests; infact, many are mediocre. The top scoring schools in the district include many poor schools and some middle class schools.

There was a well-publicized research project where a very low SES inner-city school, just about the lowest in the district, was matched with a wealthy neighborhood school of similar size. I had an interesting chat with the lead researcher. She told me the kids at the wealthy school were far more to be pitied than the ones at the downtown school. They had lousy curriculum, uninspiring and lackadaisical teachers, much make-work and busy-work in class instead of engaging teaching and learning. The students did OK but not great on the tests, mainly because their parents provided them with supplementary tutoring and enrichment. The downtown kids got lots of enriching and exciting stuff happening at school.

Some of the rich schools have resources to die for because parents pay for computerlabs, athletic equipment and sports facilities, musical instruments and all kinds of things, but there's no real evidence in my district that these schools have better teaching and learning happening, while the high-risk schools have a lot of very dedicated staff who are willing to put in a lot of extra time and support for the kids.

I think it is largely true that affluent schools have better physical plants and "extras" than poor urban schools, but that is all. We have only one high-achieving affluent school. None of the affluent schools are in the pits, but many are just middling.

Anonymous, you sure hit the nail on the head:
In low SES schools, if the kids don't get what they need at school, they don't get it at all, because the parents are unable to provide it.

In other words, parent involvement in the sense of overcoming poor in-school instruction is much more likely to happen in high SES schools. Having had kids at such schools, I know that the parents provide a large chunk of the total academic input.


A friend teaches at one of the moderately high scoring schools. They make no secret that it is the parents' job to teach their kids spelling, math facts, handwriting, reading skills and much more, so that the school can concentrate on all kinds of fancy projects and integrated units. The parents all jump to provide their kids with personal assistants, laptops, Sylvan, you name it.

In my downtown school, parents are supportive but can't really teach their kids academics or pay tutors. The teaching during the school day is what matters.

Some of our low-SES schools are getting the job done, too, and outscoring the affluent schools. More important, the kids from those schools go on to good high schools and colleges. It's tough to sustain momentum at such places with budget issues, high mobility and teacher turnover, but teaching low-income kids well in school is definitely do-able.

KDeRosa said...

Let me post my graph of student performance vs. median household income for Pennsylvania's 501 school districts.

The high SES have a higher mean pass rate than the lower SES schools, but there are more than a few high SES district under-performers and quite a few low-SES over achievers.

School can make a sizable difference in achievement.

Anonymous said...

In no way will I argue the point that schools cannot make a difference in achievement, as they certainly can and should. I would like every child to attend a school with a strong, content-rich curriculum and a focus on the need for mastery of the basics as a start point. Then everyone will make progress. Parents should not be expected to do the teachers' jobs.

Dick Schutz said...

Your graph makes a good point. Even IF Income(SES) WERE causal, the R squared of .427 indicates that it accounts for less than half the variability in achievement. You'd get about the same thing with "Mental Ability"--which the insensitive achievement tests are actually a proxy for.

What we're short on are MANIPULABLE variables that can be used to control the other half of the variability. The most obvious place to look for these is in instructional product/protocols and that's where available evidence indicates that the control can be found. But that's the last place anyone is interested in looking. It's more profitable for the interests involved to look at society, parents, kids, and teachers--end of search, end of "accountability."

Anonymous said...

Students from low income households come to school without background knowledge and vocabulary. It becomes difficult for them to be successful without a knowledge base on which to build. Schools in low SES neighborhoods spend a large majority of their time catching kids up. This often takes away from the already stressed time table that is given to teachers. It is true that students in more affluent homes come to school with more knowledge than those from poorer homes. However, it is not true that all schools with affluent populations perform better than those with low SES populations. Instruction is the key.
Parent involvement I feel is more important in the area of value placed on education. If a parent from a low SES area emphasizes the importance of education and encourages their child to perform to the best of their ability, eventhough they cannot provide the extra help or instruction that is available in wealthier homes, the student can still succeed. Direct parent involvement can be a limited factor in determining how well a student performs.
I completely disagree with the one size fits all mentality. Every class has individual needs and those needs must be met in order for students to be successful. Ultimately, what we as educators hope to accomplish is to help students identify their needs and help them find the tools to meet their own needs so that they can become successful individuals who take responsibility for their own learning.

Anonymous said...

Anyone who has worked with at risk low-SES urban students from different cultural backgrounds, knows that homes that place a high value on education and children self-discipline, lead to sucessful student outcomes. This is true, regardless of whether parents can help with homework. Don't delude yourself into thinking parenting does not matter.

Parry Graham said...

Broadly speaking, higher SES schools tend to attract a larger pool of competitive teacher applicants. By "competitive" I mean that they are more likely to be experienced, to have an undergraduate major or masters in their primary subject area, to have held leadership positions in previous schools, etc. Higher SES schools attract this larger pool for a variety of reasons, such as offering higher salaries and experiencing fewer discipline issues.

Given that teacher quality is the most important school-based factor impacting student learning, being able to attract a more competitive workforce gives higher SES schools (and their students) an advantage over lower SES schools.

All of this is said with the caveat that any individual school is capable of attracting competitive teacher applicants, and of having incredibly talented personnel. But, on balance, higher SES schools offer higher salaries and more teacher-friendly working conditions, and that matters when you're competing for high-quality teachers.

Parry

Unknown said...

Engaging parents is essential on a classroom level and school-wide level. Not easy in Chicago where the urban school district cheats the students with only 900 hours of instructional hours per academic year. If anything, the students in the high poverty areas need additional quality instructional time and after school programs. It should not be an either/or question. As an instructor, social justice is providing the best learning opportunities for my students. What I have control of is the minutes that my students are with me. What we need is something along the lines of developing 90/90/90 schools that require more instructional time, more time for staff to plan and evaluate our progress.

Tracy W said...

Anyone who has worked with at risk low-SES urban students from different cultural backgrounds, knows that homes that place a high value on education and children self-discipline, lead to sucessful student outcomes. This is true, regardless of whether parents can help with homework. Don't delude yourself into thinking parenting does not matter.

No one thinks that.
What we are arguing is that schools can do far better than the average school is with those children whose parents don't place a high value on education and wouldn't know self-discipline if it upped and bit them.
The view is that there are at least two ways for kids to succeed in education:
1. If their parents (or equivalent figure) effectively teachs them.
2. If the school effectively teaches them.

Given that teacher quality is the most important school-based factor impacting student learning, being able to attract a more competitive workforce gives higher SES schools (and their students) an advantage over lower SES schools.

Only if your measure of competitiveness is the same as teacher quality. Also what is your basis for stating that teacher quality is the most important school-based factor? The research I am aware of on effective schools is that school-wide changes are very important - see for example http://www.sabine.k12.la.us/online/leadershipacademy/high%20performance%2090%2090%2090%20and%20beyond.pdf. The factors they list seem to be at least as much under the control of the principal as the individual teachers.

Dick Schutz said...

"Given that teacher quality is the most important school-based factor impacting student learning..."

That "given" has been given by teachers colleges and teachers unions.

It derives from the same sloppy research and insensitive measures that give rise to the belief that SES is causal in student achievement.

DI inquiry by Englemann and SWRL inquiry demonstrate that instructional product/protocols "are the most important school-based factor impacting student learning."

This is consistent with what holds in every other sector of life. It's not the individuals per se, but the "things they use" that account for differences.

Tracy W said...

Standardized tests were not used as measures of individual achievement before the mid-1960s.

This is false as I have pointed out to you before. For example, O-levels were introduced in Britain in the 1950s. See http://en.wikipedia.org/wiki/Ordinary_Level
And my own grandmother who went to high school and university in the 1930s reported sitting exams designed to measure her individual achievement, and in particular to see if she could get into university.

No other sector of life uses such convoluted and arcane measures. They'd be laughed out existence in a minute.

This is also false. Medicine uses standardised tests for diagnositic purposes - see http://eprints.soton.ac.uk/10479/

Tracy, you need to do more homework.

Dick, how would me doing more homework fix your errors? I quoted where the text you referred to contradicted your claim about IRT scores coming out of the computer forming a normalised test - that's all I can do. I don't see how me doing more homework will result in you starting to get things right, it never has in the past, I think that you are going to have to put some effort into fixing your mistakes yourself.

It's not possible to thrash out these technical matters in blog comments that quickly deteriorate to "tis-taint" and "so's your old man" level of dialog.

Your errors are far larger than merely technical matters. For example, as I pointed out above, you present a hypothetical situation about how to construct tests. You then draw conclusions that actually appear nowhere in the hypothetical situation you made, for example you state that " Second, we’ve sliced the latent trait into age/grade levels". But you didn't slice the latent trait into age/grade levels in your hypothetical situation. This is not a merely technical error, this is far more fundamental than that.

Of course I will continue to point out your technical errors as you make them, as I adore thrashing out technical matters like these. If you want to discuss them somewhere else on the WWW to which I can get acess, please say so. But drop the idea that you only have some technical errors to fix, that's just as wrong as anything else you've said.

Tracy W said...

Sorry for posting that response to Dick in the wrong place.

Parry Graham said...

There is a significant body of research literature identifying teacher quality as the most significant school-based variable impacting student achievement. Here are a couple examples, but I can find plenty more if you would like me to.

McCaffrey, D.F., Lockwood, J.R., Koretz, D.M., & Hamilton, L.S. (2003). Evaluating value-added models for teacher accountability. Santa Monica, CA: Rand Corporation. http://www.rand.org/pubs/monographs/2004/RAND_MG158.pdf.

Sanders, W. L., & Rivers, J. C. (1996). Cumulative and Residual Effects of Teachers on Future Student Academic Achievement. Knoxville, TN, University of Tennessee.

Wenglinsky, H. (2002). How schools matter: The link between teacher classroom practices and student academic performance. Education Policy Analysis Archives, 10(12). http:// epaa.asu.edu/epaa/v10n12/

Wright, S.P., S.P Horn, & W.L. Sanders. (1997). Teacher and Classroom Context Effects on Student Achievement: Implications for Teacher Evaluation. Journal of Professional Evaluation, 11: 57–67.

Dick, outside of the work you mention by Engelmann, are you aware of any third-party research literature that explicitly identifies the quality of instructional protocols as having a significant impact on student learning?

Parry

Parry Graham said...

Tracy,

There is also a significant body of research literature that suggests that school leadership (i.e., the principal) and organizational practices can have an impact on student learning. However, I don't believe I have seen any studies that suggest that these factors have the same level of impact as teacher quality.

Dick, to make you happy, here are a couple of additional links to studies conducted by decidedly non-union, non-education-school researchers. Hanushek is an economist at Stanford who has developed and implemented some pretty sophisticated quantitative models to look at the school-based factors that impact student achievement. He's not really a sloppy research kind of guy.

http://edpro.stanford.edu/Hanushek/admin/pages/files/uploads/teachers.econometrica.pdf

http://edpro.stanford.edu/hanushek/admin/pages/files/uploads/HESEDU2018.pdf

http://edpro.stanford.edu/hanushek/admin/pages/files/uploads/input_based.EJ.pdf

Parry

Dick Schutz said...

It really doesn't take "research" to "prove" that teachers are important and that they vary.

This holds from chefs who make soup to psychiatrists who treat nuts.

The thing is teachers are not a manipulable variable. We teach with the teachers we have, and studies of general "professional development" (unlinked to specific instructional/product protocols) have not shown that they do anything.

Follow Through was a major study.

My paper, "A Brief History of Programmatic Educational R&D,"

http://ssrn.com/abstract=1370088

chronicles experiences that independently parallel Englemann's.

Follow Through was preceded in 1963 by a major study "Cooperative Research Program in First Grade Reading Instruction" instigated by dedicated scholars. Chall, Bond, Clymer, and Durrell, compared five reading methods, involving 27 programs and some 9000 students.

The report is out of print but was reprinted in the Reading Research Quarterly in 1997. Several interesting findings:

(1)"One of the most striking findings was the persistence of project differences in reading achievement even after statistical adjustments were made for pupil readiness." (p.74)

(2)"A child who has the ability to read phonetically regular words also has skill reading words of high utility even though these latter words may be highly irregular." (p 73)

(3)"Teacher experience and efficiency ratings are only slightly related to pupil success." (p.73)

(4)Class size was unrelated to test scores.

As in Project Follow Through the analysis of the findings was convoluted and politicized conclusions bore little relationship to the findings.

The heuristic findings (1) and (2) have been ignored and the null findings (3) and (4) have been widely accepted.

“An enduring legacy of this study has been a greater focus on teacher and learning situation characteristics rather than on methods and materials. “

http://www.reading.org/Publish.aspx?page=RRQ-32-4-Readence.pdf&mode=retrieve&D=10.1598/RRQ.32.4.2&F=RRQ-32-4-Readence.pdf&key=B2818024-B0EE-4F63-9DFA-35AEC9C6A15C

The "legacy" endures. The $100million study of Reading First queried teachers regarding the core reading program they were using. However, the report of the investigation did not report the results or use the data in any analysis.

The French have a saying "At night all cats are gray." If you don't look at differences in instructional products/protocols, they all "look alike." And if you close your eyes to cats, you won't see them at all.

As I've said, this "legacy" holds harmless all the unaccountables above school sites.

Tracy W said...

"Evaluating value-added models for teacher accountability"
The discussion here in the text as far as I can see does not support the assertion that teacher effects are the most important. To quote from the article:
"The magnitude of effects and the relative importance of teachers compared with other factors influencing learning are difficult to assess, but our review suggests that these papers can be interpreted as demonstrating the existence of a teacher effect." pages 56/57 by the pdf count, 35/3 by page numbering.
http://www.rand.org/pubs/monographs/2004/RAND_MG158.pdf

Perhaps I have missed something and at some point in this paper they do identify teacher quality as the most significant school-based variable. It's a very long paper. If you tell me the page number you believe supports your assertion I will have a look at it, but at the moment I see no such support from this paper.

"Cumulative and Residual Effects of Teachers on Future Student Academic Achievement."
Again, unless I am missing something, this study does not attempt to look at whether other school-based factors have more of an impact on student achievement. It is only concerned with the impact of teachers. A worthy subject in and of itself, but not supporting your assertion.

"How schools matter: The link between teacher classroom practices and student academic performance."
Again, nothing in here to support that teachers are the most significant school-based variable. This study looks at the effectiveness of teachers' practices in the classroom, but not where said practices came from. If, for example, you have a principle monitoring and providing feedback on good practice in the classroom this study would not pick up on it.

Again, I might be missing something in the paper of course, and if there is evidence in here to support your claim a quote or at least a page number would be great.

"Teacher and Classroom Context Effects on Student Achievement: Implications for Teacher Evaluation"
Unlike any of the other references you have provided, this paper at least obviously attempts to look at other possible school-based effects on student achievement (of course, I may have missed something in the other papers and if I have I will be grateful if you point it out to me). However the variables they measure are limited. They have a control for school system, but not for the individual school, not for the curriculum used by the school, not for textbooks used by the school, not for school funding, not for school organisation, not for professional development, etc.
As one research study this may be useful, but it hardly supports your flat-out statement.

I think perhaps you have missed that there is an important difference between "teacher quality has an impact on student learning" and teacher quality is the most important school-based factor impacting student learning". 3 out of the 4 studies you quote only try to quantify the first statement.

Dick - your "study" does not apparently control for observer bias, in this particular case the Hawthorne Effect.

Teachers who presumably knew whether or not their students were participating administered the test. And there was an incentive - according to your description participating in the programme meant waving "the Title III evaluation reporting requirement for each district."
Of course you may have done better in your actual study than in your report of it, but this description is not convincing.

Dick Schutz said...

Two very ticky-tack points for Tracy. As far as the students were concerned, the instruction was "school as usual." There was no novelty effect.

The Title III reporting requirements were at the District level, far removed from teachers. There were periodic "tests" but they were intrinsic to the instruction. They were "no stakes" tests, far removed from an extrinsic, artificial "high stakes" test.

Tracy W said...

There was no novelty effect.

Did you test for the novelty effect?
If you did, what were the results?
If you didn't, how do you know there was no novelty effect?
Dick, you have asserted in the past that you didn't believe me because I didn't provide any evidence for my assertions. Why do you therefore continue making assertions without providing any evidence for them?

The Title III reporting requirements were at the District level, far removed from teachers.

So the district level had no connection to teachers? I find this unlikely. It could be true, more unlikely things have been true, but then again it could not be true. Again I notice you provide no evidence for your assertion.

Tracy W said...

I should say that for all I know Dick and his colleagues did test their project properly and their results could very easily be valid. And even if they didn't do all this that doesn't mean that the original argument is wrong. Just I feel that having pointed out wrong arguments by Dick on the topic of standardised testing I should at least try to apply the same critical thinking to a hypothesis I am inclined to approve of.

Dick Schutz said...

Hey, Tracy. Did you look at the scope of this investigation? Over 100,000 kids in hundreds of school districts in 18 states. The population was so large that it was possible to take independent random samples to make what would ordinarily be considered "very large studies." The independent samples yielded the same findings.

Moreover, the study was replicated with a second cohort of kindergartners the following year. Because personnel were more experienced the second time around, accomplishments were somewhat better in this replication.

Speculation about "Hawthorne effect" and "incentives" as causal is wacky.

The same kinds of robust results were obtained with the different treatments used in other large scale studies reported in other papers on the SSRN site.

Parry Graham said...

Tracy,

Great points. You made me go back and re-read those studies with an especially critical eye.

The first study (the monster one) actually makes your point for you. Starting on page 75 of the pdf file, the authors argue that few studies explicitly focus on disentangling teacher quality effects from school and district effects. As you said, an especially effective principal who helps teachers improve their practice through observations and teacher supervision could be picked up as a “teacher quality” effect, as opposed to a “principal quality” effect.

The third study, the Wenglinsky study, does not explicitly say that teacher quality is the most important school-based variable, but, by my reading of it, it does carry that implicit conclusion. The study focuses on whether or not “schools matter”, makes the point that “while the research… does find significant effects for school characteristics, the magnitudes of these effects tend to be modest”, and then draws the final conclusion that “the total impact of the teaching variables [is] comparable to that of student SES”. But your point is well-taken: the study does not attempt to tease out other potential variables that might impact those teaching variables.

I guess a more accurate and defensible statement on my part would be that teacher quality has a sizable impact on student learning, that I am not aware of studies of other school-based variables that document a similarly sized impact, and that “teacher quality” remains a black box of multiple variables. Based on my own experience, and my reading of less definitive research, I believe that “teacher quality” is primarily shorthand for the curricular, assessment, and instructional decisions that teachers make on a daily basis, which can in turn be influenced by a variety of factors.

I disagree with Dick that teacher quality is not a manipulable variable. In my opinion, it’s the one variable that you truly have to manipulate if you want to improve schools. But I’m late to pick up my kids, so no time to elaborate.

Parry

Dick Schutz said...

"I disagree with Dick that teacher quality is not a manipulable variable. In my opinion, it’s the one variable that you truly have to manipulate if you want to improve schools."

Well, you can select teachers.but selection is just arranging the chairs. It's not addressing the aggregate enterprise.

The Reading First Impact Study indicated that even highly focused "Professional Development" and a lot of it did nothing to effect kids'reading performance.

I can hardly wait to hear how you are going to manipulate the variable, "teacher quality."

You consider it "shorthand for the curricular, assessment, and instructional decisions that teachers make on a daily basis, which can in turn be influenced by a variety of factors.

It seem to me that's tossing the "whole enchilada" into the "black box."

Tracy W said...

Dick - for educational research to work you have to get a lot of things right. It doesn't matter that you covered 100,000 kids in hundreds of school districts in 18 states if your measure of success was biased.

What were the independent samples? Can you go into more details?

Speculation about "Hawthorne effect" and "incentives" as causal is wacky.

Such speculation is important regardless of its whackiness. The more criticism a hypothesis can stand up to, the stronger it is. I assume that you didn't test for the novelty effect, so you have no reason to assert that it didn't exist.

And I'm not that impressed with the SSRN site by itself, the quality of the papers there can be very low as anyone appears to be able to upload papers. Can you please tell me which papers have passed through a peer review process?

Parry - thanks for your openness. I think there have been some studies in NZ that came to the conclusion that principals were deeply important, but I can't find them again now. That's why I questioned your statement earlier.

Parry Graham said...

Dick,

There are tons of ways to try and manipulate teacher quality. For example:
• The recruitment practices used to try and “lure” people into teaching (this can affect the overall pool of potential teachers)
• The preparation that prospective teachers receive prior to entering the profession
• Hiring practices (i.e., the decisions made on who actually receives jobs)
• Whether or not teachers are paired with a mentor in their early years
• Mandating specific curricula
• Leadership practices that positively or negatively influence teacher working conditions
• The training (i.e., professional development) that teachers receive after entering the profession
• The supervision process used to monitor teacher effectiveness and encourage less effective teachers to leave the profession

I believe, however, that you are looking for manipulations that could have a positive impact on teacher quality. In looking at the partial list above that I created, the research evidence is all over the place, and I have found very little that suggests big-picture manipulations that can reliably have a positive impact. Teacher certification requirements don’t appear to have much impact on teacher effectiveness (although I know there is much debate around that point). There was an interesting piece by Malcolm Gladwell in the New Yorker recently suggesting that picking effective teachers is actually quite difficult. The Wenglinsky research article I cited above suggests that certain types of professional development can positively impact teacher effectiveness, but with multiple caveats. There was a large-scale research study done back in the 90s that looked at professional development, and it found positive relationships between pd and teacher effectiveness, but the study relied on teacher self-reports and did not attempt to make a connection to student learning (I can get citations for the above-mentioned studies and articles, if you’d like).

At the more local level (i.e., district or school level), my understanding of the research is that it is somewhat more clear, while still being pretty ambiguous (we’re dealing with varying shades of gray). Looking at the 90/90/90 school research that Tracy mentioned, there is some evidence that specific leadership practices can have a positive influence on teacher effectiveness. Research in North Carolina on the Teacher Working Conditions survey suggests links between teacher working conditions and student learning.

You suggest that by defining “teacher quality” as shorthand for the curricular, assessment, and instructional decisions that teachers make on a daily basis, I am tossing the whole enchilada into the black box. Yes and no. I am not defining teacher quality as a function of certification, as a function of class size, as a function of having a masters degree, or as a function of graduating from an Ivy League school. There is a lot that goes into teacher quality, but there is also a lot that doesn’t, and that’s my point (although maybe I’m not making it very coherently). Teacher quality is a complex phenomenon, but it’s the phenomenon I think you have to impact if you want to see improvements. Say your school sets a compelling new vision: if teachers don’t change their curricular, assessment, or instructional practices, your new vision doesn’t accomplish anything. Say your state creates a new set of standards, say your district buys sophisticated data analysis tools, say your principal goes and buys DI materials, say 10 new charter schools open in your district that directly compete with your school. Unless those changes somehow lead to positive changes in teacher decision-making in individual classrooms, student learning won’t improve.

So my litmus test for any school improvement proposal is: how specifically will it cause teachers to change their curricular, assessment, and instructional decisions? If the answer is “It won’t” or “We’re not sure”, then I am immediately skeptical.

Parry