First he gives the Grey Lady's education reporting way too much credit vis-a-vis the Obama Effect.
Even the New York Times weighed in with a story that made the Obama effect appear based on science (relying on a single study; am I alone in thinking that was sub-NYT standards?) by writing up a study claiming that black test takers upped their scores post-Inauguration Day, apparently the dividend of a "Yes we can" self-esteem movement.
Sadly, this kind of education reporting for the NYT is very much the rule and not the exception.
But on to the Whitmire's closing pathways.
First he claims that College is not sufficiently accessible. I'm not sure that's really the problem. State colleges already admit many students who aren't sufficiently prepared for college level work. Most of these ill-prepared students aren't going to make it out of college anyway, so I don't see accessibility as the problem; lack of preparation is the problem.
Whitmire recognizes this lack of preparation as a problem, but unfortunately blames the wrong culprit:
The stimulus bill proposed by the House would bump up Pell grants for poor students to make college more affordable, but that does not solve the biggest problem faced by these students: As a result of attending subpar high schools, they are not ready for college work.
The achievement gaps are present long before high-school. It is debatable whether elementary schools have really improved, as Whitmire claims, but one thing is clear they haven't improve enough yet. Middle-schoolers remain woefully unprepared for high-school level work, so why are we blaming high-schools for being unable to deal with all these ill-prepared children?
Next Whitmire claims that "[n]ational education reforms have pushed curriculum demands lower into the grades, handing kindergartners the verbal tasks that two decades ago confronted second graders." This is only partially true. Today's kindergartners are still doing the same stuff that many kindergartners of twenty years ago did. Teh only real difference is that back then we allowed the struggling students to wait until they were "developmentally ready" which has been proven to be a large waste of valuable academic time. Yet the problem remains that we are still often not too successful in teaching these at-risk kids. In this respect Whitmire has a valid point and literacy rates will have to soar for there to be an improvement.
WHitmire's next point that black boys need to be rescued is also a valid point, as long as if by rescued he means to provide them with the effective commercially available curricula that has existed for decades.
Last, Whitmire jumps on the teacher quality bandwagon with both feet:
Knowing what we know about the value of a high quality teacher, we should be on the verge of delivering those teachers to inner-city students.
Really? I though the "research" pretty much indicated that we don't have the foggiest idea how to make average teachers into superstar teachers. Actually do know how to improve the effectiveness of all teachers: hand them a effective curricula and teach them how to use it, but this isn't what most people mean when they talk about teacher quality.
Ultimately I disagree with Whitmire's major premise that the lack of pathways are what's holding back students. The pathways have been in place for all children of a certain ability level and family stability to take advantage of. It is the access to those pathways that need to be improved to accommodate a level of student ability that has never been able (or had the opportunity) to take advantage of them up until recently.
13 comments:
Whitmore says: "National education reforms have pushed curriculum demands lower into the grades, handing kindergartners the verbal tasks that two decades ago confronted second graders."
Well, yes and no. There have been no "reforms" but misguided activities under the banner of "phonemic awareness" have been pushed into K. However, the ECLS-98 data indicate that K teachers a decade ago were teaching much like they always had.
NCLB has greatly increased the amount of classroom time devoted to reading, but the Reading First Impact Study found that the increased time didn't deliver any better results.
I recently discovered the Social Science Research Network which is a very neat way to make research findings accessible.
http://ssrn.com/abstract=1336604
Copying and Pasting that into your browser should get you to a paper that presents the Technical Lexicon that defined the structure and substance of 6 school subjects in the 1970's.
My take is that the TechLex in each subject has changed very little, then-to now. The only thing that has changed much is the rhetorical fog that engulfs the Lexicon. The fog isn't any better or worse, it was thick then and it's thick now. The spin is just different today.
Whit goes on to say: "Most hurt are those likeliest to have grown up in homes with few books, always-turned-on TVs and little conversation."
Again, yes and no. Those characteristics do tend to characterize black and low income homes. But again Mom's of black preschoolers read to their kids almost as frequently as the Mom's of upper class white preschoolers do. (ECLS-K data, again)
Whit makes the all-too-frequent mistake of confusing correlation with causation. All aggregate kids enter K with more than adequate spoken language expertise to make teaching them to read a very feasible proposition.
As Ken keeps reminding us, ya gotta have a legitimate instructional program to get the job done. And instructional programs are the last place anyone (present company, and a few others excepted) anyone wants to look.
Whitman and USAT do beat Dillon and NYT on this matter on my scorecard, for what that's worth.
I think President Obama got it right. His election did give many, including but not limited to Blacks, an emotional boost. Emotional boosts are good, but they're ephemeral. To recall BF Skinner: "Take care of the operants and the afferents will take care of themselves."
That's what the President appears to be trying to do.
With friends who promote "The Obama Effect," and appointees who have "tax problems," he really doesn't need enemies.
"Drs. have it good, they bury their mistakes. Teachers have it better, their mistakes bury them."
That may be an old saw but it was new to me. It was made by a Dr Vincent J. Worden in 1970. Worden at the time was Asst. Prof at State College at Bridgewater Mass.
I'd add, "Media people have it even better. Their mistakes constitute the stuff that history is made of."
Speaking of history the doc is the proceedings of a conference, "The Use, Misuse, and Abuse of Test Results."
/www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/37/59/28.pdf
It's a "knock your socks off" document. People were grappling 40 years ago with exactly the same testing issues that are now current. What's further sad, is that the were doing a better job framing the issues and resolving them than is being done today.
Going forward to the past would save us a lot of time and disappointment in the "new" strings that are being attached to the economic recovery package.
"Yes, we have done it!"
"Actually, we do know how to improve the effectiveness of all teachers: hand them a effective curricula and teach them how to use it."
I have to disagree with you on this one, Ken. I think that this may be true for basic literacy development in the primary grades. But, as the complexity of content areas and the skills necessary to master those content areas increase (i.e., as kids move on to upper elementary, middle, and high school), I am not aware of any curricula that have been proven to be successful irrespective of the teacher.
Parry
"I am not aware of any curricula that have been proven to be successful irrespective of the teacher."
That is not a fair criteria.
What we want is curricula that work noticeably better than what we have now, on average, with the teachers that we have.
-Mark Roulo
Parry says:" . . .as the complexity of content areas and the skills necessary to master those content areas increase (i.e., as kids move on to upper elementary, middle, and high school), I am not aware of any curricula that have been proven to be successful irrespective of the teacher."
I agree, Parry. But it's not a function of the content and skills. It's a function of the fact that we aren't locking in the expertise that was attained early on. In reading the instruction is subsumed under the fog and muck of "English Language Arts" and is extended throughout the grades K-12. The imaginary "Strands" in math create a similar condition.
Kids are "all over the place" and the prevailing standardized achievement testing practices provide cover for maintaining the status quo, irrespective of the rhetoric.
Dich Schutz, I doubt that people 40 years ago were doing a better job of resolving testing issues than we are today, if they had we would not be having the same problems still.
The document you linked to is remarkably supportive of tests, given that you linked to it:
"Counselors should thoroughly review and critique the tests being used in their systems by raising such questions as: do the tests have content validity? From what sources do they derive their concurrent validity? How reliable are they?" page 20.
"Testing and related methods of objective evaluation have more relevance than ever in the area of the individualised curriculum. The emphasis of the day is on accountability and the need is for more precise and accurate ways of evaluating educational outcomes in oder that the methods of teaching, the content of instruction, and the organisation of the schools may become more effective." pg 27 [emphasis in original]
The document illuminates the misuses and abuses of tests that are very much with us today.
Take the quotes you picked, Tracy. How many counselors today scrutinize the content validity of the tests that are being used?
Take "Accountability." The notion of accountability was not introduced by NCLB. In 1970, "Accountability is at the forefront as SCHOOL BOARDS TAKE ON HEIGHTENED IMPORTANCE WITH THEIR STRONG INVESTIGATIVE AND APPRAISING ROLE.
School boards today are "out of it." No one is performing an investigative role. And appraisal is in terms of "Proficiency" as arbitrarily cut-scores on ungrounded tests.
The doc clearly draws attention to the hazards and limitations of grade and age equivalents: "The assumption of that a 12-month gain really occurs in the 9 or 10 month school year, the unwarranted extrapolation to levels not covered by a given battery, the lack of equality of units from level to level and from test to test, all argue against it's use."
The doc looks forward to a non-graded system within a decade with measurement relying on mastery tests rather than norms.
NCLB locks in interpretation in terms of "grade level" and gives no attention to "mastery."
The misuse and abuse of tests that were recognized in 1970 are now prevailing practices.
Indeed, Dick, the things you identify are still problems today. That's why I'm surprised that you said "they were doing a better job framing the issues and resolving them than is being done today."
If they were resolving those issues, why are we still having the same problems today? Why aren't counsellors examining the validity of the tests? Why have school boards given up performing an investigative role since this paper was published? I don't see how you can attack testing practices so firmly today and simultaneously believe that the past characterisation of them was so good. I mean, you may have a story here, about how the forces of test validation and so forth were overcome by some new rage of nuttiness, but at the moment I can't think of a reason why, if the blokes in the 1960s were doing a good job of resolving stuff, we'd have the same problems today. I'm all in favour of learning from the past, and sometimes the thing to learn is that what was tried in the past didn't work.
The NCLB incidentally does not lock in interpretation in terms of grade-level. It requires that states test students at particular grade levels in particular subjects, but the particular tests are left up to the state.
"If they were resolving those issues, why are we still having the same problems today?"
The problems today are much larger and more consequential. How we got to this point is a long story but some of the factors involved can be mentioned.
In part, we can blame "computers." Today the computer does all the heavy lifting in test construction and reporting. So the entire operation is mindless. The tests are "confidential," the underlying statistics are arcane, and the computer spits out numbers on an ungrounded scale.
The testing industry has become "big business." The management of outfits like ETS and the major test publishers come from the Corporate sector, not the Education sector.
The rhetoric of "Standards" has hurt rather than helped. People pay lip service to them, but they do nothing more than provide cover for shoddy istructional practices.
In fairness to the test pubs you could say that the tests have changed with the instructional times. In earlier eras a standardized test battery typically included tests of Vocabulary/Word Meaning, some form of Decoding measure, Comprehension and Spelling. However, with the domination by Whole Language Instruction and "reading for meaning, a single measure does the trick, and a single measure is needed to make the IRT wheels turn.
Similarly in Math. You had separate Computation and Problem Solving/Word problem subtests. Today the two are confounded, and the results correlate as highly with the Comprehension Test as the reliabilities of the tests permit.
Why aren't counsellors examining the validity of the tests?
Because we don't have many counsellors any more. And today, testing is "not their department" There's a "Testing and Accountability Department" for that. And locals have no choice of tests, so why bother with content validity. You administer whatever comes down from the State.
"Why have school boards given up performing an investigative role since this paper was published?"
Because the Federal Government has usurped these functions. Teachers and students today are "accountable" School boards and state and federal officials are "unaccountable." It's bonkers.
The upper echelons annually issue press releases that "we're making gains" The media phone or click it in.
It's true that NCLB only mandates the grades that the tests are administered. The interpretation in terms of grade equivalents is inherent in prevailing test usage.
Test specialists in 1970 recommended use of the 99 point Percentile Scale and even better the 9 point Stanine scale. They also advised that the focus be on the class, school, of district, not on the pupil as the key unit.
Dick, you are doing a fairly good job of convincing me that people 40 years ago failed to resolve testing issues.
One point - you say that a single measure is needed to make the IRT items turn. Firstly, I am not sure what measure means here. IRT does require the assumption that there is an attribute, or ability, or expertise, that can be represented as increasing, so that people with a low level of the attribute (or to use the IRT guys' terminology latent trait), generally have a lower probability of getting a test question right than people with higher levels do for the same question.
But if you want to test multiple skills in the same test, IRT won't prevent that. Say there are some kids who are absolutely awesome at Computation, but have never been taught how to answer Word Problems and consequently tend to get them wrong. You can divvy the test questions up into two groups and then analyse the answers using IRT separately, just as you could with classical test theory, except that IRT allows you to estimate what score a person would get on a test with a different level of difficulty (assuming that no change in their underlying ability/attribute/expertise took place between taking the two tests, and that the student doesn't get the answers either all right or all wrong).
Well, we're getting "off thread" again here. My point was not that people 40 years ago had resolved all testing issues. The point was that they then recognized misuses and abuses that educational and governmental officials today regard as "standard operating procedures" and that the media echo.
The point about the IRT wheels turning didn't relate to the internal workings of IRT but to the fact that that the IRT "machine" cranks out a single scale at a time. Of course, it's possible to posit a different latent trait and set up the wheels to crank out a different scale.
This is commonly done for different school subjects, like math and reading, but not common within a subject--one subject, one scale. this inherently precludes any diagnostic or prescriptive information in the test results. Again authorities and media are mindlessly oblivious re the workings.
Parry: I have to disagree with you on this one, Ken. I think that this may be true for basic literacy development in the primary grades. But, as the complexity of content areas and the skills necessary to master those content areas increase (i.e., as kids move on to upper elementary, middle, and high school), I am not aware of any curricula that have been proven to be successful irrespective of the teacher.
I agree with this, Parry. We still need good commercially available content area instruction for lower performers. I suppose the need for this is still too low to be commercially feasible. I know, for instance, that the demand for the upper DI levels(like reading, writing, and math) don't sell as well as the lower levels because so few kids have the skills needed to successfully pass the upper levels unless they actually went through the lower levels. I suspect, there'd have to be a critical mass of students who've gone through all the DI levels (or their equivalent) before there'll be a demand for content area curricula.
The point about the IRT wheels turning didn't relate to the internal workings of IRT but to the fact that that the IRT "machine" cranks out a single scale at a time.
IRT does not crank out a single scale. It assumes that the latent trait (or, if you prefer, non-directly-observable ability, or however you want to describe it) can be regarded as a single scale, in which the more you have of the latent trait the higher your probability of getting the test item right. Eg the better I am at driving, the higher my probability of being able to successfully complete a 3-point turn, all else being equal. The assumption of a single scale is used going in, it's not cranked out.
IRT depends on analysis of the test answers from an initial group, whose ability is known (or for a more accurate description, observed by some other method), to work out how correct answers relate to ability. If the higher-ability students get answers all over the place on the questions, as they would if the skills the test questions measure aren't correlated in the high ability group, this alerts the developer that there's a problem. (It doesn't tell the developer what the problem is, it could be that the test questions are badly written, debugging is a separate skill).
this inherently precludes any diagnostic or prescriptive information in the test results.
Which is why the test questions and the children's answer sheets should be returned to them, or to the school. And another reason for releasing this is to pick up on badly-written test questions, and incorrectly marked ones, and thus maintain pressure for quality in the tests themselves.
Again authorities and media are mindlessly oblivious re the workings.
They may be in the USA. In NZ, the questions are public knowledge and a bad test question is front page news in the national newspapers.
Post a Comment