1. High-quality schools can eliminate the achievement gap between whites and minorities.
2. Mayoral control of public schools is an improvement over the more common elected board governance systems.
3. Higher standards will improve the performance of public schools.
As worded, the answer to all these questions is that the research is insufficient. But notice for questions 2 and 3 how any amount of improvement will do, while for question 1 only improvement that will "eliminate the achievement gap." Such improvement would have to be on the order of about a standard deviation increase in non-Asian minority performance with no increase in white performance. A tall order indeed. In fact such a tall order, that no in-school or out-of-school intervention has ever achieved such results for the general population-- even the ones that Bracey supported and touted in this very report.
This is Bracey at his most dishonest--glaringly dishonest. Bracey had an agenda and he didn't mind bending the facts to fit his preferred outcome. He wasn't an honest researcher and this will be his lasting legacy.
A more honest researcher might adopt a more neutral standard of achievement such as "an increase in the performance of all students by an educationally significant amount (0.25 standard deviation)." That would be a laudable goal and also would serve to reduce the achievement gap. It's also the generally accepted standard in education research.
Under such a standard, Bracey would still get to criticize mayorial control and higher (national) standards as not having a sufficient research base; however, he'd have to acknowledge that there is a sufficient research base for higher-quality schools under this standard at least in the elementary school years.
Another problem with Bracey's reports is that they are peppered with his own assumptions about how to reform education that don't have sufficient research base or are contradicted by the data. Here are a few.
Students attending American schools run the gamut from excellent to poor. Well-resourced schools serving wealthy neighborhoods are showing excellent results. Poorly-resourced schools serving low-income communities of color do far worse. (p. 2)
Schools serving low-income communities of color tend to have resources above the median school.
I said above that if there are to be more high-quality schools (or at least, “high-quality” schools in terms of high or rising test scores), they will have to be developed in low-income neighborhoods. (p. 3)Bracey is implies that schools in higher income neighborhoods are doing a fine job educating low-income students. They aren't.
Before taking up the question of whether schools alone can remedy the achievement gap for poor children, we have to ask what is known about the effect of poverty on children. What are some of the out-of-school factors that contribute to poor children’s lower performance? (p. 4)
None of the studies Bracey, especially Berliner's, directs us to are capable of determining the causal link that Bracey implies. Bracey then proceeds to give us a few pages of various ailments and problems associated with poverty and attempts to draw a bleak picture of poverty's causal effect on student achievement. He has to resort to anecdote because the data a much less bleak picture. Poverty, or more accurately low socio-economic status (SES), is correlated with low student performance. But the amount of variance in student performance attributable to variations in SES is only about 18%. That means that 82% of the variance is attributable to non-SES factors. Bracey knows or should have known this, but misrepresents the data anyway.
These disadvantages all operate to attenuate achievement in schools. The question is, can “high-quality” schools alone offset them? (p. 7)
Bracey then looks at one ham-fisted study, Harlem Promise Academy, as a refutation. He ignores the other studies which have shown results larger than the 0.18 standard deviation gap attributable to poverty effects.
Bracey does a better job with the mayoral control and high standards issues. But the problem is that on the poverty/SES issue Bracey's non-research-based views are no better than those of the proponents of mayoral control and high standards.
Pot and Kettle meet Bracey.
51 comments:
> The last and thankfully final Bracey Report ...
Final, of course, because the man died. Trust DeRosa to cheer, rather than lament, that fact.
I lament the passing of Jerry, the man, as much as anyone. The passing of the edu-pundit not so much.
My own first-grade son is half-Asian, and my wife is a first-generation immigrant. Educationally, she takes the lead in terms of strategy, which generally involves the kids doing quite a bit of studying outside of school. My first-grader is about average intelligence, AFAICT, and is has about as much resistance to studying as any 6-year old.
Even though my son didn't know very much English, I started taking him through Engelmann's 100 Easy Lessons book starting at the beginning of kindergarten. It was very difficult, and I did a pretty bad job of it -- some of the exercises were just undoable because of his lack of English skills (blending "motor" and "boat" makes no sense when he doesn't know what either one is). Not to mention that it's not really as easy to follow a script faithfully as teachers pretend. Anyway, I got through the book just before the start of first grade, though I was not really entirely satisfied that I'd gotten him to mastery.
I just had my first PT conference. They were fantastically impressed -- the teacher had given my son a test in the first couple of weeks of school which involved reading a set of 158 words, in ascending order of difficulty. He got 128 of them right. She'd mentioned she had previously taught regular-ed students who got only 20 or so right.
It would have been easier if we were wealthy enough to afford private classes, of course. I think this is where the bulk of the .18 effect size comes from (and that the wealthy often expect more from their children).
Of course, his math at home has been kumon. Based on the experience of my older son, I don't expect him to be learning any math from any of his elementary school teachers.
Seems to me you've misconstrued the substance of the last Bracey Report, Ken.
Jerry was debunking the three assumptions that are popularly made by others.
"High-quality schools can eliminate the achievement gap between whites and minorities."
His take on this is pretty much yours. My take differs from both you and Jerry. If one looks for "gaps" one can always find them. And the reading and math tests being used assure the gaps.
Aggregate kids enter school with the minimum prerequisites to be taught to read, irrespective of SES. The disadvantages of poverty are still there, but the kids will be able to read, which is all that kids and parents are asking of schools.
This was demonstrated in large scale studies in the 1970's, and it's currently happening in some schools in the UK now.
On Mayoral control and "standards," You, Jerry, and I would write it up differently, but I think we'd agree that debunking is in order.
Personally, I liked the old Bracey Reports where he took potshots from all directions and at all comers better than this "analysis." The worst and best of Jerry was this kind of puncturing. He did have a few personal biases. For example, I know a guy who fiercely and regularly promotes DI. Nothing wrong with that.
Jerry died before finishing drafting the paper. Any beef really should be directed at those he finished and edited the paper.
Speaking of editing, that last line should read "who" rather than "he."
SpellCheck can't be trusted catch all typos.
"Aggregate kids enter school with the minimum prerequisites to be taught to read, irrespective of SES. The disadvantages of poverty are still there, but the kids will be able to read, which is all that kids and parents are asking of schools."
As a parent, and as an educator, I expect more of schools than to simply ensure that kids are "able to read".
Additionally, I know that you've defined "able to read" previously in other blog posts, but I want to make sure I understand. By "able to read" do you mean able to decode words, or do you mean something more complex, along the lines of being able to understand and assimilate information from a text?
If it's the latter, then there are (IMO and experience) huge disparities in this area on the front end and back end of K-12 education.
Parry
The question is OT, but Ken must be mulling over his misconstruing the Bracey report, working on the dinosaur post, or something.
By "able to read" do you mean able to decode words, or do you mean something more complex, along the lines of being able to understand and assimilate information from a text?
The latter, of course. Children entering school have this expertise in spoken communication. That is they can comprehend. They have not learned how to cope with written language.
That's what reading instruction is all about. And the link between spoken and written language is the Alphabetic Code. Teaching children how to handle the Code is the core of reading instruction.
.If it's the latter, then there are (IMO and experience) huge disparities in this area on the front end and back end of K-12 education.
Precisely. And schooling is in the middle. Since aggregate children enter school with the minimum prerequisites to acquire academic expertise, the disparity at the back end has to be a function of the schooling.
Toxic instruction is inadvertently disadvantaging many students. Some students learn to read without any formal instruction in reading. Others learn despite the toxic instruction. Prevailing tests permit schools take credit for this learning. The remaining instructional failures are attributed to deficiencies in the kids, their parents, or society.
The deficiency is not in any of these, and it's not in the water. It's in the instruction and in tests that are sensitive to SES differences but not to instructional differences.
As a parent, and as an educator, I expect more of schools than to simply ensure that kids are "able to read".
Me too. Everyone does. The point I was trying to make is that parents would be satisfied is schools would reliably teach kids to read and do math to get a job and/or go on to college, so the kids could be responsible grownups.
The ability to read is of paramount importance because it's the foundation for acquiring further academic expertise and for personal independence in assuming responsibility for ones own education.
Dick, can you give an example of a test that is sensitive to instructional differences (whether or not it's also sensitive to SES differences)?
buy cialis canadian
mail order cialis online
buy cheap cialis online us
Buy generic cialis overnight delivery
where to get cialis
discount cialis generic
Online cialis prescriptions
buy Cialis iceland 1buy cialis online
Dick, can you give an example of a test that is sensitive to instructional differences
The easiest example that comes to mind are the Advanced Placement Tests at the high school level. They could be improved, were the Item Response Theory props knocked out from under them, but by and large, "what is being taught" is sufficiently well defined that the measures provide information re what has been taught and learned.
"Standards" provide a spurious basis for instructionally sensitive tests. There are too many of them. Some are unteachable even if they superficially appear to be testable.
To get an instructionally sensitive test, you have to start at the other end: What instructional accomplishment/achievement are you talking about, and how would you know when it's completed?
The Race to the Top regulations don't talk about instruction at all. The regs define effective teachers and principals as those whose students achieve "at least one grade level in an academic year" and highly effective teachers and principals as those whose students achieve high rates. "one and one-half grade levels in an academic year."
And they mandate this with a straight face.
With effective instruction, the scores on an instructionally sensitive test will pile up at the top. Teachers are transparently effective by performance and thus demonstrate that they are "qualified." The reliable delivery of instructional accomplishments removes the correlation with SES.
Some conditions of poverty may indeed be be insuperable obstacles to an aspired instructional accomplishments. But if so, these will be student-specific and can be dealt with as such.
There is much more to be said about "program-specific tests," and "program-fair tests." It isn't complicated but it's addressed in several SSRN papers if anyone is interested.
http://ssrn.com/author=1199505
“Children entering school have this expertise [being able to understand and assimilate information from a text] in spoken communication. That is they can comprehend.”
Do you have evidence to support this? I would find it hard to believe that all students, particularly those with limited vocabularies and background experiences, arrive at school with the “expertise” to understand and assimilate information from a variety of spoken texts, especially once the curriculum moves past first or second grade.
I’m also not sure I understand what you mean by “toxic” instruction. Do you mean ineffective instruction, or something more negative?
Finally, when you say “the disparity at the back end has to be a function of the schooling,” isn’t that a pretty broad statement? Couldn’t disparities at the back end be a function of a variety of factors, in addition to schooling? I don’t mean to absolve K-12 education of its responsibility to teach all students, or its need to improve, but you’re making a pretty big argument.
Parry
I would find it hard to believe that all students, particularly those with limited vocabularies and background experiences, arrive at school with the “expertise” to understand and assimilate information from a variety of spoken texts, especially once the curriculum moves past first or second grade.
Hey, you're skipping grades. Kids enter school with sufficient personal assets to be taught to read. At that point, they're on the school's clock. Instruction is currently focused on their deficits, not building on their assets.
Do you mean ineffective instruction, or something more negative?
I mean instruction that inadvertently disadvantages kid. See the paper I cited for detail.
Couldn’t disparities at the back end be a function of a variety of factors, in addition to schooling?
The debilitating factors associated with poverty will still be there, but they are not causal. What happens in instruction is causal. The same logic applies to health care.
Anonymous, Dick has a bee in his bonnet about Item Response Theory (IRT). IRT is a way of estimating what a student's underlying expertise is, given which questions on a test they got right. It works with the data provided by existing test questions, and if some of the questions are bad (by which I mean a high-ability student is less likely to provide a right answer than a low-ability student) IRT's results will signal that there is a problem with the test questions. But IRT is not the test questions, and knocking out IRT will just remove a layer of error-correction from the design of standardised tests.
Be wary of Dick, he sounds ultra-wise in how he talks about tests, but if you ask him about details he falls apart and starts contradicting himself. Particularly on IRT. See http://d-edreckoning.blogspot.com/2008/07/apparently-poor-can-be-taught-to-read.html
IRT is a way of estimating what a student's underlying expertise is
This is true only if one believes in ghosts and other unseen phenomenon. What Item Response Theory yields is an estimate of a latent trait. Few people consider reading expertise and other academic accomplishments to be either latent or to be traits, but most people have bought into the IRT derivatives without understanding its workings.
and if some of the questions are bad (by which I mean a high-ability student is less likely to provide a right answer than a low-ability student) IRT's results will signal that there is a problem with the test questions.
Again, not. Items which most students get wrong and items which most students get right are discarded because they poorly discriminate the "latent trait" that IRT posits. This has the net effect of rendering the test instructionally insensitive.
The most valuable information for an instructor is the items most students missed and the items that most students got wrong. This information indicates where the instruction is weakest and where it is the strongest. Item Response Theory discards the grain and keeps the chaff.
Don't trust me. Tracy could be right. Check it out.
Jerry was debunking the three assumptions that are popularly made by others.
That was how I read it, too.
He certainly ripped David Frum apart for his support of Promise Academy:
A closer look at this highly publicized data point shows that it was true only
for one year, one grade, and one subject. For other grades, the students remained
substantially below white students, and for all three years of the comparison,
the gaps between whites and blacks on the English-Language Arts tests were
quite large. In addition, Columbia University sociologist Aaron Pallas has observed
that these students also took the Iowa Tests of Basic Skills, where they
scored only at the 33rd percentile despite their strong performance on state tests.
“Scoring at the 33rd percentile,” deadpanned Pallas, “is not a great success story.”
Pallas titled his blog on the topic “Just how gullible is David Brooks?”19
As for the achievement gap, I thought he was saying that "high quality schools" can't overcome the substantial environmental damages that cause cognitive problems in poor children. He's saying that the achievement gap can't really be closed. There's certainly more to the achievement gap than poverty, but I didn't see him offering any easy answers.
I did disagree with him about Sidwell being the optimal type of school, but I really don't understand how you read him as supporting those three assumptions.
This is true only if one believes in ghosts and other unseen phenomenon.
Well I don't believe in ghosts, but I do believe that there are unseen phenomenon that do exist. I can't see the electricity that's allowing me to write these messages on the Internet, but that doesn't mean that electricity doesn't exist. I can't see Dick's reading expertise in his brain, but that doesn't mean that Dick can't read.
What Item Response Theory yields is an estimate of a latent trait. Few people consider reading expertise and other academic accomplishments to be either latent or to be traits, but most people have bought into the IRT derivatives without understanding its workings.
Dick we have this discussion every time. The definition of latent trait, as defined in the source you provided, was:
Each of these is what psychometricians refer to as an unobservable, or latent, trait. Although such a variable is easily described, and knowledgeable persons can list its attributes, it cannot be measured directly as can height or weight, for example, since the variable is a concept rather than a physical dimension.
See http://echo.edres.org:8080/irt/baker/chapter1.pdf
If reading expertise is not a latent trait, then please tell me the units it can be measured in. I keep asking you this question every time you claim that reading expertise is not a latent trait, as defined by the IRT people, but you never answer.
As for "most people have brought into the IRT derivatives", I suspect that most people have never even heard of IRT.
Items which most students get wrong and items which most students get right are discarded because they poorly discriminate the "latent trait" that IRT posits.
Dick, in the past you have claimed that IRT-designed tests always return a normal distribution. To quote you: "IRT facilitates the norming by forcing the scores into a normal distribution." http://d-edreckoning.blogspot.com/2008/07/apparently-poor-can-be-taught-to-read.html
Now if you are right in your claim that IRT forces the scores into a normal distribution then any test designed using IRT is going to have to include some questions that most students get wrong (to provide the top tail of the normal distribution) and some questions that most students get right (to provide the bottom tail of the normal distribution). So your claims are in conflict.
Furthermore, if we look at the resource you provided on IRT, at chapter 8, the discussion of typical teaching goals, Chalres Baker says:
Tests used for screening purposes have the capability to distinguish rather sharply between examinees whose abilities are just below a given ability level and those who are at or above that level. Such tests are used to assign scholarships and to assign students to specific instructional programs such as remediation or advanced placement. (http://echo.edres.org:8080/irt/baker/chapter8.pdf, page 14 of the pdf)
Now if a test is intended to pick up students for advanced placement it is going to need questions that most students will get wrong so as to pick up on the unusually advanced students.
The most valuable information for an instructor is the items most students missed and the items that most students got wrong. This information indicates where the instruction is weakest and where it is the strongest.
You are contradicting yourself. To know where instruction is strongest, you need to know the questions which most students got right. After all, if something wasn't tested at all you can't draw any conclusions about the student's expertise, if you have me sit a reading test in only English you can't make any deductions about my bike-riding expertise. So therefore by your argument, your own premise is wrong.
Item Response Theory discards the grain and keeps the chaff.
Nope it doesn't. Item Response Theory simply is a way of estimating students' underlying expertise from the question answers. Nothing in IRT requires one to discard questions. See for example this definition of item response theory from http://echo.edres.org:8080/irt/
Item Response Theory is the study of test and item scores based on assumptions concerning the mathematical relationship between abilities (or other hypothesized traits) and item responses.
Don't trust me. Tracy could be right. Check it out.
Indeed, please, if anyone can find any evidence supporting Dick's claims above, please tell me. I like counter-intuitive events, and Dick makes some extraordinary claims, for example that IRT, a mathematical theory, can force test designers to throw out the questions that most students get wrong or that most students get right.
I can't see Dick's reading expertise in his brain, but that doesn't mean that Dick can't read.
Precisely. Reading expertise doesn't exist in the brain; there are no "latent traits" in the brain. Reading is a phenomenon that can be observed directly. Positing a latent trait only creates a fiction, which William of Occam reminded us a long time ago as not a good idea.
The fiction leads to the silly practice of defining "proficiency" in terms of arbitrarily-set cut scores on an ungrounded "test."
One doesn't need a paper-pencil, artificial instrument to "measure" expertise. Reading and other academic accomplishments are complex skills, and their acquisition takes the same form as the acquisition of expertise generally. It's useful to track the acquisition, and there are "how to" papers on methodology at
http://ssrn.com/author=1199505
Good luck chasing "latent traits" and administering standardized achievement tests that purport to measure them.
Reading expertise doesn't exist in the brain; there are no "latent traits" in the brain.
This is a breathtaking claim. If reading expertise doesn't exist in the brain, where else do you think that it exists? And what evidence convinced you that it resides in that organ and not in the brain?
Or have you decided to argue that there is no such thing as reading expertise?
As for the rest, I notice you don't defend any of your earlier statements about IRT. Before I start addressing your next statements, do you now agree that you were wrong about the previous statements you made? These were, to remind you:
"most people have bought into the IRT derivatives without understanding its workings."
"Items which most students get wrong and items which most students get right are discarded because they poorly discriminate the "latent trait" that IRT posits. "
"Item Response Theory discards the grain and keeps the chaff."
If reading expertise doesn't exist in the brain, where else do you think that it exists? And what evidence convinced you that it resides in that organ and not in the brain?
Brain matter exists in the brain, silly, not "expertise."
Reading is an observable behavior. It exits just like other behaviors, such as drawing false conclusions, as an empirical phenomenon.
The factual basis of my earlier statements is verifiable. The statements are not personal views or opinions.
It is extremely interesting for me to read the blog. Thank you for it. I like such themes and everything connected to this matter. I would like to read a bit more soon.
Brain matter exists in the brain, silly, not "expertise."
Again, I repeat my question.
If reading expertise doesn't exist in the brain, where else do you think that it exists? And what evidence convinced you that it resides in that organ and not in the brain?
If asking this question makes me silly then so be it. I would far rather be silly and informed than sensible and ignorant.
Reading is an observable behavior. It exits just like other behaviors, such as drawing false conclusions, as an empirical phenomenon.
However what most people are interested in is reading expertise, of which reading behaviour is only a manifestation (one cannot possibly test a child on every single thing they might want to read in the whole of their lives). Again, I repeat my earlier question, are you going to argue that there is no such thing as reading expertise?
The factual basis of my earlier statements is verifiable.
The only way that you can convince me that this statement is true is by verifying them.
Please provide the evidence that convinced you that:
1. That reading expertise resides in some other organ than the brain.
2. That reading expertise is an directly observable feature of people in the same way that height and weight is.
3. That most people have bought into the IRT derivatives. (We will leave the claim that most people have done this "without understanding its workings" until later).
4. That items which most students get wrong and items which most students get right are discarded because they poorly discriminate the "latent trait" that IRT posits.
5. That Item Response Theory discards the grain and keeps the chaff.
The statements are not personal views or opinions.
Dick, if you say a statement then I'm going to treat the statement as a sample of your personal views and opinions, unless you preface it with something to make it clear that you don't personally believe this.
Let's try another complex skill, such as driving or using a piece computer software. Do these matters of expertise exist in the brain? Do you really believe that the brain has little partial homunculi resident representing every human skill?
I would far rather be silly and informed than sensible and ignorant. So far you're following the "silly and ignorant" route in your arguments here.
To repeat. Reading expertise, or lack thereof, exists as any other behavioral phenomenon exists. The phenomena are dealt with empirically, not neuro-biologically.
I can't bring you up to speed in psychometrics via comments here. My point was that Advanced Placement tests would be more instructionally sensitive if they more closely matched the instruction, rather than matched Item Response Theory. The comment was made in passing, in response to a request for an example of an instructionally sensitive test.
We could have a tis/taint Monty Python kind of argument over the comment. But if you disagree, let's just say we disagree and get on with talking about something else. Or I'll be glad to retract the clause. It was a moot comment because the outfits constructing the Advanced Placement Tests are not going to drop IRT any time soon.
I don't have "a bee in my bonnet" about Item Response Theory. I've constructed tests using the theory, administered thousands of tests constructed per the theory, taught courses that included attention to the theory. But as measures of individual achievement, the theory yields tests that are not "fit for use." Not fit for use because the tests are sensitive to SES differences, but not to instructional differences. That's an empirical contention, not a theoretical contention. If you want to refute the evidence, we might have something to talk about.
None of this has anything to do with "Pot. Kettle. Bracey." I assume Ken is working on the dinosaur piece or something else, and hope that he gets back to his blog soon.
We could have a tis/taint Monty Python kind of argument over the comment. But if you disagree, let's just say we disagree and get on with talking about something else. Or I'll be glad to retract the clause.
So let me see. You see three options to deal with me disagreeing with you - having a tis/taint argument, agreeing to disagree, or gladly retracting the claim. What's missing here is the option of providing supporting evidence for the claim, thus attempting to convince me that it is right.
What's more, you say that you are glad to retract a clause. But if you believed that your initial clause was truthful, then why would you be glad to retract the truth? I see two options here:
1. You are well aware that your initial clause is false but stated it anyway.
2. You believe that your initial clause was true, but you don't care whether or not the statements you make are true, as if if you cared about the truth or falsity of the statements you made you would hardly feel glad at having to retract a statement that you believed to be true.
In either case, there is no reason to believe anything else you say, either you are consciously making statements that you know to be false, or you don't care whether anything you say is true or false.
As for your questions about expertises and brain, my currently-favoured hypothesis is that skills are stored in the brain in the form of changes in the arrangement of brain matter compared with the brain before it learnt the skill. The expression of the skill normally is dependent on functioning of other parts of the body. This hypothesis might be wrong, if you can convince me that it is I will change it, as unlike you I care about whether or not the statements I make and the things I believe are true or not.
Tracy-
Have you read Jane M Healy's Your Child's Growing Mind ?
It asserts that reading phonetically and regularly literally affects the physical neural structure of the brain.
When I read Shaywitz' comments about how dyslexia is real because it shows up on MRIs, I think of Healy's work. The lack of neural connections because of poor instruction means that the dyslexics' brain functions differently now, not that dyslexia was inevitable because of a difference in how the brain
functions.
For most dyslexic kids the differences in brain functioning that is showing up on MRIs is a result of poor or no reading instruction rather than confirmation that dyslexics' brains always functioned differently and this made it difficult to learn to read.
skills are stored in the brain in the form of changes in the arrangement of brain matter compared with the brain before it learnt the skill.
I certainly agree with that. All learning is stored in the brain.
Robin has a good explanation of what is going on with reading/dyslexia. Do you have fault to find with that explanation?
Robin - I would be surprised if any sort of learning did not affect the physical neural structure of the brain. Of course it is entirely possible for me to be surprised, I do not pretend to be an expert on brains. But if anyone theorises about learning being stored elsewhere I become deeply curious about where else they think it might be stored.
Dick, I will answer your questions once you have answered mine from earlier. Please provide the evidence that convinced you that:
1. That reading expertise resides in some other organ than the brain, in support of your assertion that "Reading expertise doesn't exist in the brain; there are no "latent traits" in the brain."
2. That reading expertise is an directly observable feature of people in the same way that height and weight is.
3. That most people have bought into the IRT derivatives. (We will leave the claim that most people have done this "without understanding its workings" until later).
4. That items which most students get wrong and items which most students get right are discarded because they poorly discriminate the "latent trait" that IRT posits.
5. That Item Response Theory discards the grain and keeps the chaff.
Or, if you can't provide the evidence, tell me that you now agree that those statements of yours were wrong and you were wrong to say them, and that you were wrong to say that "The factual basis of my earlier statements is verifiable."
My bad on misconstruing the purpose of Bracey's first argument. Though the specific deficiencies I point out still hold.
Dick says "His take on this is pretty much yours. My take differs from both you and Jerry."
What is your view, Dick?
Dick also says "For example, I know a guy who fiercely and regularly promotes DI. Nothing wrong with that."
The difference is that my personal bias has a research foundation, Bracey's does not. Most pundits tout lots of stuff with no research base, but Bracey was being hypocritical because he a. knows how to evaluate research and b. ignored the lack thereof anyway.
Dick also says in response to Parry "Precisely. And schooling is in the middle. Since aggregate children enter school with the minimum prerequisites to acquire academic expertise, the disparity at the back end has to be a function of the schooling."
Yes and no. There seems to be an out of school deficiency that schools haven't learned how to fully compensate for yet.
And i still can't make heads or tails of the IRT/latent trait issue.
All of my comments stand as stated.
The thread is about "Pot. Kettle. Bracey" I didn't intend to take us off topic and there is no reason to get further off, just to humor silly misinterpretations and faulty inferences.
Anyone want to talk about Bracey or the final Bracey Report?
According to last week's final RttT regs the Ed Dept "believes that the inquitable distibution of highly effective teachers and principals is a major cause of the achievement gap". (Page 412/775)
The regs makes it clear states and local districts will have to come up with plans to reassign such teachers and principals to high minority and high poverty schools so there is no longer a disproportionate presence in low minority and low poverty schools.
That makes sense except there was no attempt to do anything to increase the total number of effective teachers, require content mastery (except for alternative candidates), or require effective curricula.
Good instructional materials and a knowledgeable teacher allowed to instruct can take the high SES kids who travel, grew up with books, and talk politics and economics at the dinner table to an achievement level that's hard for low SES kids to ever get to.
Is our nation's emphasis on closing the achievement gap encouraging policies that hamstring our strongest students so other students can catch up?
If poor instruction and the resulting lack of practice to fluency in language, reading, and math can affect the physical structure of the brain, what are the long term consequences of emphasizing the gap instead a growth in achievement for everyone?
RttT is more of the same from DoE. It won't work and has no indicia of success to rely upon. This is the bane of every ed reform of the past and RttT is no different in this respect.
I'm not convinced that highly effective teacher sin affluent suburban school districts will remain highly effective when transported to low-SES districts.
Q:what are the long term consequences of emphasizing the gap instead a growth in achievement for everyone
A: The consequences are
(1) a belief that SES is causal. This belief is strengthened by achievement tests that are sensitive to SES differences but not to instructional differences.
(2) a belief on the part of many that schools cannot override the adverse conditions of poverty. That is, we have to "fix" poverty before schools can "fix" kids.
Meanwhile, every school district and public school in the country is headed to being termed "failing" by 2014 and the National Academy of Sciences has warned that each of the four RttT "reforms" has no scientific/technical foundation.
http://books.nap.edu/openbook.php?record_id=12780&page=1
Ken asked for my view of the issues Jerry treated in his last report.
I go along with Jay Mathews:
Jerry leaves the first two assumptions, about high-quality schools and mayoral control, looking like road kill.
http://voices.washingtonpost.com/class-struggle/2009/11/braceys_last_report--our_sick.html
I think Jerry also did a good job of trashing "standards."
Ken says, The difference is that my personal bias has a research foundation
True. Jerry did have a few Points of View which occasionally got in the way.
Another Bracey posthumous publication re testing is spot on and it may help clarify what has tangled up at least one re IRT.
I can't get the link to activate, so anyone interested will have to google for it.
The Big Tests: What Ends Do They Serve?
Published in the Nov. issue of Educational Leadership. What Jerry says about NAEP applies in spades to lesser standardized achievement tests.
There seems to be an out of school deficiency that schools haven't learned how to fully compensate for yet.
True. But kids, parents and citizenry aren't looking to schools to redistribute income. Just to teach them enough to be employable and/or go on to college.
And i still can't make heads or tails of the IRT/latent trait issue
It really isn't an issue. If you believe that academic achievement is a latent trait, you go along with the standardized testing industry (and Tracy W) and you're content with instructionally insensitive tests. If you don't believe in "latent traits," a notion that psychologists (other than misguided psychometrists) long-ago discarded, you look for alternatives. Alternatives are there for the implementing, but they are not currently being considered.
Ken says I'm not convinced that highly effective teachers in affluent suburban school districts will remain highly effective when transported to low-SES districts.
RttT regs define effective teachers and principals as those whose students make at least a year's gain on standardized tests.
That's a foolish definition, and it provides further incentives for teachers and principals to game the mandate.
I didn't intend to take us off topic and there is no reason to get further off, just to humor silly misinterpretations and faulty inferences.
And this is Dick all over. He's happy to make confident sounding pronouncements, but if you challenge him on them, he will either claim that he's right or will do his best to change the topic one way or another. Dick never backs up what he says.
A: The consequences are
(1) a belief that SES is causal. This belief is strengthened by achievement tests that are sensitive to SES differences but not to instructional differences.
Dick, do you read Ken's blog? In this very post on this thread, Dick says that the amount of variation in performance attributable to variations in SES is only about 18%. See http://d-edreckoning.blogspot.com/2008/03/statistical-illiteracy.html
And here Ken reports on Gering School District, which has moved to DI, and is now seeing higher performance on average than Nebraska despite the SES levels being below the state average.
If tests were sensitive to SES differences but not to instructional differences, then how come the Gering School District is doing so well?
I am going to now make a prediction. Dick is not going to answer my question in any meaningful way. He is not going to produce any evidence to support his claim that standardised achievement tests are insensitive to intrustional differences. Instead he is going to try insulting me, saying very firmly that he's right and changing the topic.
Ken: And i still can't make heads or tails of the IRT/latent trait issue.
Okay, here's my take on the latent trait issue.
We are interested in things like how well a person is able to read, or drive a car, or do basic arithmetic.
However we can't directly observe reading ability, or driving ability, or arithmetic ability like we can observe height or weight. All we can observe is the person doing, or trying to do but failing, various tasks.
In order to distinguish things that you can't directly observe, like reading ability or driving ability, from things you can directly observe, like height, weight, hair colour, Charles Baker, in the source on IRT Dick provided, calls the first category latent traits.
Before I start on IRT, do you have any questions for me about what latent traits are?
Before I start on IRT, do you have any questions for me about what latent traits are?
Yes, that's exactly where the entire argument falls apart for me.
I think I understand the IRT issues, but I don't see why a school that does a very good job teaching can't decrease the gap in an IRT test. If the students learned more, then the there should be a better chance that they know more and have a better chance of answering a random question correctly.
I see why this might break down if a large sample of students were to answer the random question correctly, leading to the question being thrown out.
However, I don't see why one school or one district can't increase their mean performance under an IRt test.
I also don't undersatnd what a latent trait has to do with any of this.
though I agree with Dick that tests should be more sensitive to instruction.
It may be a foolish definition of Effective but it can still cause great mischief. The comment to RttT (page 418) makes it clear that involuntary reassignments will soon be the norm.
It's fascinating that the feds rejected expressly requiring or even awarding points for states who implemented the NMAP's recs on math teacher prep programs and licensing requirements.(Page 130/775)
What conclusion should be drawn about "reform" that has a weak definition of "effective" teachers AND kept rejecting all attempts to strengthen instructional materials and the knowledge and skills of the teaching corps based on research as to what works?
The conclusion I draw is that when middle-class parents start finding out that they can't buy their way into good school districts with better teachers since all the better teachers will have to be shipped to the low SES schools, support for public education and the enormous tax burden it causes will fall. In advertantly, the feds are hastening the demise of public education as we know it with such measures. That's probably a good thing.
I'm going to answer your question, Tracy, so you were wrong about that prediction. But you are likely right in predicting that you'll say the answer is not "meaningful."
Incidentally, it's Frank Baker, not Charles Baker. Frank is not a leading authority on IRT. He explains it in what I regard "in a meaningful way. But he's descibing the IRT pitch. IRT proponents slide right from latent a latent trait which must be homogeneous and univariate into skill ability. Reading expertise is clearly neither a homogeneous or univariate thingy. It changes it's manifest nature as the individual moves from novice to expert.
Height and weight are manifest variables amenable to measurement with equal interval scales as temperature can be measured. Reading and driving are not amenable to equal interval measurement but they are amenable to measurement with Gutttman scales analogous to measuring spoons or the Snellen eye chart that is used in vision tests.
If tests were sensitive to SES differences but not to instructional differences, then how come the Gering School District is doing so well?
The extended explanation is in a couple of SSRN papers:
All Achievement Tests are Not Created Equal
Program-Fair Evaluation of Instructional Programs
http://ssrn.com/author=1199505
But I can give you the short answer. The Gering data show that Gering is doing relatively better in teaching reading per the test norms. I've repeatedly said that a legitimate instructional architecture can over-ride SES differences and DI is one such architecture.
However, Gering has nudged the distribution of test scores upward. It hasn't changed the shape of the distribution which would be necessary if the instructional accomplishments were attained--no child left behind.
Nor has Gering eliminated the "racial gap"--nor likely the "gender gap." These gaps are largely a function of the instructionally insensitive tests rather than of the instruction.
Ken says Inadvertently, the feds are hastening the demise of public education as we know it with such measures.
It really isn't inadvertent. The "progressive" agenda for elhi education is exactly the same as the educational agenda of the far-right agenda . When you see Newt Gingrich playing a starring role on the "Road Tour" along with Secretary Duncan, with Reverend Al there for "diversity," you know where the Feds are heading.
We've got a real "education crisis" going. The National Academy of Sciences has warned that each of the four reforms has no foundation. If you look at the 500 point rating sheet at the end of the RttT, it's clear that the "reformers" don't even know how to construct a decent rating form.
In giving their assurance that they will implement the "reforms" states have little wiggle room to be "innovative."
The public schools are a well-established institution and 80+% of parents are satisfied with the school they are sending their kids to.
Charter schools have barely made any inroads into public schools.
The only question is how and when the "Race to the Top" will implode.
Jerry Bracey would have speeded up the implosion. But it will happen without him.
Gingrich is hardly far right, Dick.
The "progressive" agenda for elhi education is exactly the same as the educational agenda of the far-right agenda . When you see Newt Gingrich playing a starring role on the "Road Tour" along with Secretary Duncan, with Reverend Al there for "diversity," you know where the Feds are heading.
Do you mean that both th eprogressive and far right want to destroy public education. I think neither is the case.
Gingrich is hardly far right, Dick.
Point taken. That used to hold, but the right wingnuts have moved farther to the right.
Do you mean that both the
progressive and far right want to destroy public education. I think neither is the case.
It's no secret that the Republican Party agenda is to privatize public education. Rove/Dubbya wanted to effect this with school vouchers in NCLB, but that didn't happen.
Newt Gingrich says that the only part of the Obama administration agenda he takes issue with is that he would prefer vouchers (spun as Pell Grants)instead of Charter Schools.
RttT promotes Charter Schools and adds three other killer reforms to the agenda.
Secretary Duncan and the Progressives see none of the technical flaws in NCLB that will brand every public school district as "failing." The aim is to rebrand the title of the legislation.
My reading is that this amounts to wanting to destroy public education, akin to I'm going to kill (reform) you to save you from yourself.
This is so contrary to the promises President Obama made in the campaign and to the values of the base of the Democratic party that the American public doesn't yet know what is going on.
Reform, high quality, world class, standards, removing unqualified teachers. Who could be against such? But each term is either empty or aimed directly at the core of public schooling.
The "race" is going to me the mother of all education muddles.
Yes, there is a portion of the far right that curves around meets back up with the far left, though both are at each other's throats.
Having said that though, I do not believe that the progressives, misguided though they are on almost every single issue, want to destroy public education (i.e., government run schools). Like everyone else, progressives want to a) be in power and b) improve the education of children. The progressive's favored way of accomplishing things is with a heavy governmental hand. But that isn't working in education and the base is growing dissatisified with the educational status quo and the players. Advocating charters is a good way to maintain control of schools while appearing to upset the status quo.
Repulicans, like Gingrich, believe that the government running things like schools is a bad idea. A properly regulated and competitive free market will yield better results. They still believe in government subsidies and/or funding. That doesn't strike me as a desire to destroy public education. It's just another belief for how to improve education outcomes which is consistent with their political view.
In this case the politics is making strange bedfellows but neither side wants to abolish public funding for education.
I'm going to answer your question, Tracy, so you were wrong about that prediction.
This makes it then one of those times that I am happy to be proved wrong (there are plenty of times when I am proved wrong). Thank you for taking the time to answer my question. I am afraid though that I am not convinced by your answer.
IRT proponents slide right from latent a latent trait which must be homogeneous and univariate into skill ability.
Nothing in the maths requires a latent trait to be homogenous or univariate.
. Reading and driving are not amenable to equal interval measurement but they are amenable to measurement with Gutttman scales analogous to measuring spoons or the Snellen eye chart that is used in vision tests.
Measuring spoons are based on known relationships of volume, for example a standard tablespoon is 3 standard teaspoons. It is not obvious to me what it means for person A to have 3 times as much reading ability as person B.
The Snellen eye chart is a set of standardised lines of letters of different sizes, so if a subject can at a distance of 20 feet (in the USA) read the 4th line accurately but not the 5th line the optician can work out the difference in resolution at that height. Again, it is not obvious to me what the equivalent is in terms of reading ability.
Driving ability and reading ability are complex, someone may have learnt to drive on an automatic, never driven a manual and thus be incapable of doing a hill start but has driven safely in all sorts of traffic and weather conditions, are they a better or worse driver than someone who recently learnt to drive on a manual and can do a hill start without thinking about it?
Another thing I notice about your analogy is that measuring spoons and eye charts only measure traits that are homogenous and univariate. Snellen eye charts only measure visual acuity, they do not measure other aspects of vision such as colour-blindness. Measuring spoons only measure volume, not weight or temperature. If reading ability and driving are amenable to meausrement with Guttman scales then reading ability and driving ability must be homogenous and univariate, the very thing you wrongly sneer at IRT proponents for claiming.
So we have a contradiction here. Dick simultaneously falsly criticises IRT proponents for assuming that things like reading ability are homogenous and univariate, while also saying that reading ability can be measured by analogy with methods which can only cope with homogenous and univariate features.
As for your referred papers:
In "All Achievement Tests are not Created Equal", you say:
Tests referenced to the general curriculum both showed a high level of performance "at level" on matters they had been taught and a lower level of performance on matters "above level" that they hadn't yet been taught.
In other words you found as an empirical matter that standardised achievement tests responded to instruction.
As for "Program-Fair Evaluation of Instructional Programs", this is an old paper where you and Ralph Hanson appear to have developed a methodology for measuring what programmes are trying to achieve. It looks fine to me, but I can see nothing in there that explains how come Gering School District can be doing so well on tests if those tests are not sensitive to instructional differences. Admittedly it is a long paper and the scan's hard on my eyes, I could easily have missed something. If you could specify the relevant page in the pdf that would be great.
However, Gering has nudged the distribution of test scores upward.
Interesting claim. Where did you locate the distributional data for Gering High school? I couldn't see it in the links in Ken's post.
So to summarise, of the responses you make:
- One of your supplied references directly contradicts your claim that tests are not sensitive to instructional differences.
- I cannot find any support for your claim in your other reference.
- You make a claim about the distribution of Gering High School data but do not link to where you found this data.
- You criticise IRT proponents for assuming that latent traits like reading ability and driving ability are homogenous and univariate while recommending a measuring system for them that can only measure traits that are homogenous and univariate.
It hasn't changed the shape of the distribution which would be necessary if the instructional accomplishments were attained--no child left behind.
Actually, if Gering High School provided as good an education for its top students as it did for its bottom then the shape would still be the same. In other words, let's say a school sucessfully teaches every child to read well enough to read and comprehend the local newspaper. While it is teaching the necessary skills to the slowest learner, it also advances the instruction of all the other learners, bringing say the top performing group up to reading Chaucer in the original, the next level up to reading Shakespeare, and so forth. So my hypothetical school would have achieved its instructional goals, and still have a broad distribution of results.
Ken: I think I understand the IRT issues, but I don't see why a school that does a very good job teaching can't decrease the gap in an IRT test.
Neither can I. This is why I keep questioning Dick whenever he claims this.
Incidentally, I had better say explicitly that I assume that we are talking about a test designed using IRT that properly measures whatever the school is doing a very good job teaching. If the test in question was aimed at measuring musical composition which the school didn't teach I'd still expect a gap obviously.
I also don't understand what a latent trait has to do with any of this.
Again, I don't either, this is one of the reasons why I say that Dick has a bee in his bonnet about IRT.
Ken - actually I think I can say what a latent trait has to do with IRT.
A latent trait is more difficult to measure than something direct like height or weight, or something homogenous and univariate like visual acuity. So all the statistical complexity of IRT is useful in measuring latent traits. You could use it on measuring height, or visual acuity, but why bother when simpler methods work just fine for the simpler measurement problems?
Right. Either way you get a measurement and I don't see why going through the more complicated IRT route yields invalid results.
maybe I should read through the papers again, but last time I did I didn't see this question being adequately dealt with.
I think the problem with measuring reading ability (i.e., comprehension in the later grades) hinges on knowing background kniowledge which correlates with IQ/SES because it it often not explcitly taught in school and therefore must be largely learned via the out-of-school envirnonment.
Tracy, that "answer" would earn you a failing grade in any pschometrics course, but this isn't a psychometrics course. Your misunderstandings are so twisted and convoluted that I can't untangle them here.
No problem. We're commenting on a blog and got sidetracked on a matter that has no bearing on the thread.
Ken has gone on to other matters. Me too.
I don't understand why it is so difficult for politicians to one, realize that they have no business in deciding what's best for public education, those who are involved and see it every day are the experts in this category and that they are unbelievably hypocritical. Bracey himself states that the wealthier neighborhood schools do better than the poor, so why is it then that funding is being taken away from these poor schools and given to the wealthier schools? To increase the high and low SES gap. There is a direct correlation between poor neighborhoods and achievement, the money isn't going to their schools the same as wealthier neighborhoods and the students then don't have the supplies, space or teachers that they need. If a school is short on funds a teacher is going to have more students in their classroom to decrease the number of teachers needed to save money. A teacher with 30 students cannot teach as effectively as a teacher with 15 students, it's easy to see. I have another question that I would like to ask politicians and that's how is that they're so smart to "know" what students need if the education system in this country has been so bad? How did they get so smart? What kind of education did they receive? What kind of education are their children receiving?
Post a Comment