Nevermind that we're eight years away from that threshold, the moaning has already begun in earnest. Educators are quick to throw lots of kids they don't think will be capable of ever becoming proficient under the bus. That nasty ol' NCLB is so unfair. It's an impossible standard.
Of course, what educators fail to tell you is that that 100% proficiency standard excludes up to 1% of students with significant cognitive disabilities who are permitted to take an alternate assessment under NCLB as Edweek reports ($):
Under federal rules for testing under the No Child Left Behind law, up to 1 percent of a district's or school's testing population can take alternate assessments and have their "“proficient"” or "“advanced"” scores count in the district'’s or school's calculation for showing adequate yearly progress'— the law'’s key accountability measure. According to the federal government, those assessments must meet the same technical standards as other tests given by a state under the 4 ½-year-old law.
So where did the Feds come up with this paltry 1% exclusion? Did they just make it up? No, not exactly.
The department does say that research suggests no more than 1 percent of the student population "—about 10 percent of all students with disabilities" should fall into that category.
I'm willing to bet that 1% doesn't include any kids with the bogus "learning disabilities" tag which school districts have used to throw kids they've mis-educated into the special-ed ghetto enabling them to collect government booty while simultaneously shirking their responsibility to educate.
The altenate assessment rule pre-dates NCLB. Of course, before NCLB it was routinely being ignored like many of the Fed's requirements for the money they were doling out:
First mandated with the 1997 reauthorization of the Individuals with Disabilities Education Act, alternate assessments were, in some states, tied to the individualized education program, or IEP, of the particular student. The fact that such students were being tested at all was considered more important than the quality of the tests, and the tests were not included in accountability measures.Now things have changed:
The No Child Left Behind law changed that, by requiring that all tests be included in accountability determinations and meet high technical standards. In December 2003, the Education Department issued regulations on the technical qualities such tests must meet in order for the results to count under the federal law.It is bitter medicine indeed for our educators who've grown accustomed to the institutionalized failure their faddish education techniques have caused.
23 comments:
I can only comment for Massachusetts; but the standard chosen by Massachusetts is high enough that the best performing schools in the state will struggle to achieve the eventual scores that will be mandated of every school.
For example, Boston Latin Academy scored 98.4 for composite English Language Arts last year; but to be "100% proficient", they would need to score 98.5.
This isn't about subgroups, special ed, or anything like that. Boston Latin has no special ed, no limited english speakers, and they pre-screen admissions so they've only picked the cream of the crop.
The eventual standard in Massachusetts is arbitrarily high. So high that a finding of "inadequacy" is meaningless.
When combined with the mandated sanctions - turn control of the school over to the state, fire all or most of the staff, hire corporate management, etc, it becomes arbitrary and capricious. Most parents don't want the federal government to oversee the dismantling of the schools their kids attend.
Presumably, the next federal congress will recognize this, and change the law, before the people change the congress.
"When combined with the mandated sanctions - turn control of the school over to the state, fire all or most of the staff, hire corporate management, etc, it becomes arbitrary and capricious."
It sounds like you have issues that have nothing to do with the specific calibration of the standard.
The good old dismantling theory of NCLB. It couldn't possibly be about improving education. And so many democrats voted for it too.
The major problem with NCLB is that it didn't implement national exams -- well, that, and it didn't fire every educrat in every school system.
Sure, I've got issues with the sanctions. They've never been demostrated able to achieve the scores that are mandated, among other things. For me, personally, the issue is control of our town school. We used to be independent, but now our regional superintendent is so scared of losing his job that we come in third in line, after state and federal bureaucrats.
But that's a separate matter. If the standards are arbitrary, that makes the whole premise that we can hold schools accountable arbitrary. Accountable for what? If accountability is just a ruse for labelling schools inadequate and turning control over to state and federal bureaucrats, that isn't accountability.
First, the complaint is that the standard is too high. Next, the complaint is that the standard is arbitrary. These are combined with the "dismantling theory" and the "ruse theory".
Why don't you give us a link to sample (is it MCAS?) tests or questions and explain why kids at each grade level shouldn't have to know that material. Most of the tests I have seen (especially NAEP) are trivially easy. I have heard that Massachusetts is setting higher expectations than other states, but that may not mean much. Let's see some examples. Also, add in a link to how the tests are scored. I am actually really interested in this. Often, people argue (one way or another) about state tests, but nobody has actually seen the questions on the test or the grading scheme. I would love to see an example of a tough state test.
However, I am skeptical about your arguments when you first complain about the level and then complain that any test (apparently) is arbitrary. Then you complain that it's all a plot to take away local control of schools. Actually, I perfer completely local control of education, but that means parental control and choice. Then, schools don't have to worry about any standards and tests, they just have to worry about their jobs.
The example I provided is the scores.
One of the best exam schools in the country, Boston Latin, barely passes ELA composite and fails a couple subgroups. In Math, they would fail AYP already, but are excused by the "safe harbor" score for the time being.
Every student attending Boston Latin takes the SAT test. 95% go to college, and many of those go to elite colleges.
If the standard set is high enough that a school no one questions is excellent, and which screens out 98% of the population; if that school is only excused by a "safe harbor" exemption, then mandated scores are arbitrarily high.
You might as well be identifying inadequate schools with a lottery, unless you intention is simply to identify nearly every school as inadequate.
"The example I provided is the scores."
But I want to see the tests and the scoring system. Your example might sound persuasive, but I want to see the details. Have you seen the tests, or are you just relying on this one piece of data?
The problem is that your complaint goes far beyond just the calibration of the test. You are using this one piece of data to justify throwing out the whole system of accountability - and as the basis for your "ruse" and "dismantling" theories.
"For example, Boston Latin Academy scored 98.4 for composite English Language Arts last year; but to be "100% proficient", they would need to score 98.5."
I don't know about this scoring in Massachusetts, but in our state, these "proficiency" scores are calculated in a funny way. Nobody seems to explain how they are calculated exactly, but it is definitely not a linear scaling of the exam results. 98.5 is not a percent that has a direct relationship to how many questions the kids get right. (I assume that this is a target for 2014.) It has to do with the percent of kids that get over some minimal proficiency level that is set by the state, not NCLB.
I can't find the particular formula for our state, but you can see how the scores relate to exam resluts. There are 4 proficiency levels that kids are placed in as a result of their grades on the test. Before the test is given, a large committee of lower school teachers take the test and work out the the grading and cut-off points. There are some multiple choice questions, some short answer, and some longer answer questions. For the longer answer questions, they set the requirements for partial credit. This is a very normal process for designing a test, but the key ingredient is that the teachers themselves calibrate the results.
The students take the test and the grades are determined and scaled to between 0 - 80. This is where the first funny business comes in. There isn't a linear relationship between the raw test score (based on the teacher-selected maximum number of points) and the 0-80 score. Forty is the midpoint and the cut-off for achieving the "proficient" level. As far as I can tell, (unless I do a curvefit or regression of some scores I see), this non-linear scaling raises scores below the 40 cut-off point and lowers(?) scores above the 40 cut-off point. This score, between 0-80, is used to put each child into one of four proficiency levels. Levels one and two are below the 40 proficiency cutoff, and three and four are above. After that, the process of determining a "proficiency index" gets fuzzy, but it is clear that the score, often in the 90's like BLA, is not directly related to the average of raw test scores. It is based on the percent of kids who achieve the MINIMUM proficiency of 40. Looking at our state's sample exams and scoring, the 40 level (in spite of any non-linear scaling) still means that the student got only half of the questions correct based on the teachers' own criteria. My guess is that BLA's target 98.5 percent proficiency rating has to do with the percent of kids that get above some minimal cut-off point on the exam. So the goal is not to get all of the kids up to some unusually high standard; it has to do with not leaving anyone behind some minimal cut-off level of expectations. This is how you can have a great school that doesn't quite meet the expectations. NCLB is truly about leaving no one behind. The tests are very simple and the cut-off points are low - about a 50 percent grade in our state.
My issue with NCLB is not accountability, but the fact that it will institutionalize low expectations and allow the schools to avoid doing more. Our state is very pro-active about the test. They get to set the criteria and then use it as cover for schools. Most (non-urban) schools in our state easily hit the top proficiency level and then use that rating to pacify parents into thinking that the schools provide a good education. The real result is only that they leave few behind - which is not very difficult in an affluent town. Then, our schools and school committee argue that because of this high rating, no child should be allowed to go to charter schools. There is currently a moratorium on charter schools in our state. they think it's "their" money.
I'm actually with those who think a 100% pass rate for those taking the exams is unachievable.
Weird things always happen. A kid will have her first epileptic fit ever in the middle of the exam. A student will have just been dumped by his girlfriend and be in a screw-everything mood.
A 95% or even a 98% pass rate would be much more reasonable. The objective, I hope we all agree, is to get all kids actually learning. A school can't predict which kids at the start of the year will have some special circumstance which interfers with exam performance, so a very high but not 100% rate would still maintain the pressure to ensure every kid learns.
Tracy, read through Charles Murray's piece on NCLB. It was printed in the WSJ and reprinted here .
Murray comes at this from the right. I, as a progressive libertarian. Murray's kids lost their best english teacher to NCLB testing mandates. My kids, their best math teacher. Murray does this stuff professionally. I program computers professionally, and didn't even know what AYP was until last month, when it was the last straw in a year filled with bumbling bureacurats screwing up my kid's school.
One of Murray's basic points; you can't create a test with a single measure of proficiency that doesn't have a bell curve, unless it is a trivial test. If you want a 95% pass rate, your definition of proficiency can't match the MCAS definition or NAEP definition of proficiency - not even for Boston Latin Academy, where they prescreened to select only top performing kids.
As things are, schools are leaving some kids behind because they need to get from 60% to 70% proficient to stay ahead of sanctions. That means they triage resources away from the top 55% and the bottom 25%, so they have a prayer of getting past sanctions for another year. If they pass, the principle keeps his job. If they fail, the principle gets the axe. What would you do if you were in charge of allocating resources to meet a mandate to improve passing rate next year or lose your job?
I agreed with some, but not all, of Murray's article here.
Murray is wrong on the single cut point though. The object of NCLB is to get all students passed a cut point. The fact that those above the cut point will be normally distributed is irrelevant.
And don't think that because Boston Latin is an elite school means they are doing a superior teaching job. frequently, these schools teach just as poorly as other less elite schools and rely on their students for high scores. Nothing prevents kids from learning.
I agreed with some, but not all, of Murray's article here.
Murray is wrong on the single cut point though. The object of NCLB is to get all students passed a cut point. The fact that those above the cut point will be normally distributed is irrelevant.
And don't think that because Boston Latin is an elite school means they are doing a superior teaching job. frequently, these schools teach just as poorly as other less elite schools and rely on their students for high scores. Nothing prevents kids from learning.
"... unless it is a trivial test."
Where is the link to the sample tests? I have yet to see a test where the grade-level problems are not trivial. On top of that, the cut-off is very low; 50 percent in our state. The shape of the curve (bell or otherwise) does not change just because you set a lower cut-off point. As Ken says, the shape above the cut-off doesn't matter. Even the difficulty of the test is less important if the cut-off is low enough.
"Murray's kids lost their best english teacher to NCLB testing mandates. My kids, their best math teacher."
Look at it from the teacher's viewpoint. Let's say you are a seventh grade math teacher and many of the kids coming into your class don't know the times table. Your job is to get everyone to meet some seemingly impossible (given what you are starting with) goals. If I were the math teacher, I would not be happy. If I could not change the "system", I would definitely think about another career. The mandates force the teacher to focus on the lower end kids at the expense of the more able kids. As a parent of a willing and able child, I would not like this situation either.
But the problem is not the test. (I have, however, seen some bad tests, but they are usually so easy that it really isn't a big issue.) The problem is a system that uses unproven curricula, poor teaching methods, and allows students to slide from grade to grade with little or no expectations. NCLB tries to change this by setting (very low) proficiency levels that everyone has to meet. Some would like to pin all of the blame on external reasons (kids, parents, poverty, health care, society), but this is a cop-out and an excuse for not dealing with major problems.
For purely selfish reasons, I would like our schools to ignore the tests and give more time and attention to the better students. It would be easy to blame the poor performance of the other students on external causes. However, the bad curricula and teaching methods that cause the problems in the first place also affect my son.
Once kids get into high school, with tracking and honors courses, I worry less personally about my son. If I can blame the testing results of the non-honors kids on external reasons (and by high school, they really look external), then I don't have to worry about my desire to improve society.
For the lower grades, I can't get away with this. I see the assumptions, philosophy, math curricula and teaching methods schools use. They are horrible and no amount of external excuses can be made to cover up the problem. However, the NCLB solution, good for many kids, is not good for my son. There is no guarantee that schools with change their fundamental assumptions, curricula, and teaching methods. All they will do is to shift their resources (using their same inefficient and unproven methods) to the lower ability kids.
NCLB is a very poor way to fix the system. The proficiency levels will never be high enough to provide a proper path for kids to get out of poverty. There is no way to force schools to change their curricula and teaching methods. However, the answer is not getting rid of the test. The real benefit of the test will be to show parents that schools (especially K-8) have big difficulties getting many kids to learn much of anything.
Hopefully, this will lead to the ultimate in local control of education - full school choice and vouchers. Only competition will break the inefficient and pedagogical straglehold of public education.
Steve - from my limited knowledge, the states have very different standards. Maybe your state doesn't have a particularly rigorous test. States that had the chance to define their standards after the mandate was made have tended to create standards that allow average suburban schools to pass, at least in until around 2010 or so. Compare your state's proficiency level with NAEP scores; but god forbid, please don't nationalize our testing and curriculum by asking your state to switch to using NAEP. NAEP should be used to calibrate state efforts, not as a means to nationalize education.
Massachusetts set up its test and defined proficiency before No Child Left Behind added a 100% proficiency mandate. The state doesn't offer any guidance on "what is proficient" but I suspect it is about a B- grade on average statewide. Or a C- at the best schools.
Ken- Boston Latin has a sink or swim mentality. I suspect it is reasonable to postulate they could get half the kids who score "high needs improvement" above the threshold, if they change their mission from challenging kids to achieve excellence to assuring bureacrats all the kids will achieve mediocrity or better.
But then, Boston Latin started with only "advanced" kids, who score something like a standard deviation above proficient to start with; unlike the rest of the schools in the state.
"The state doesn't offer any guidance on "what is proficient" but I suspect it is about a B- grade on average statewide. Or a C- at the best schools."
No guidance? Look at the test questions. Where is the link? What is the formula for your "proficient" score? "a B- ... or a C-". I don't believe this. Please show some evidence.
Even if your test sets a higher standard than other states (good for them!), What, exactly, is your complaint?
1. The test is unfair for your kids?
2. The test is unfair for poor kids?
3. Testing takes resources away from your kids?
4. Testing is wrong by definition?
5. The grading is unfair? To whom, the kids or the school?
You have already trotted out your "dismantling" and "ruse" theories, so perhaps you see all of this in strictly political terms. You need to cut through the politics and define exactly what your problem is.
" if they change their mission from challenging kids to achieve excellence to assuring bureacrats all the kids will achieve mediocrity or better."
[First, you complain that the tests are too difficult, then you complain about the mediocre goals they set?!?!?]
Anyways, this seems to best define your complaint. So, it's number 3 - "takes resources away from your child". I can sympathize, but you're not going to win points for your self-centered approach to solving the problems of education.
"The state doesn't offer any guidance on "what is proficient" .."
See Mass Frameworks
http://www.doe.mass.edu/frameworks
Sample MCAS tests
http://www.doe.mass.edu/mcas/2006/release/default.html
The tests look simple to me, but I was not able to find the formula that defines the proficiency cut-off point and the formula that defines the proficiency index.
There is a common mythology about state tests being too difficult. There is also a mythology that there is some other "higher" learning that makes not knowing what is on the tests acceptable.
Those who complain about the tests have to be specific about their complaints.
"There is also a mythology that there is some other "higher" learning that makes not knowing what is on the tests acceptable."
Indeed, and educrats cannot define what they mean by "higher-level thinking," nor can (or will) they give any examples of what it might be.
It's like "critical thinking," which twenty years ago meant learning about the logical construction of argumentation and identifying logical fallacies, but now means nothing more than toeing the liberal groupthink party line.
Steve, as I've said - my objection to the tests is that they were not conceived as a test to set an arbitrary level for "proficient" that all students could ever be expected to achieve.
That is demonstrated by the fact that few if any of the best schools in the state currently match the proficiency rate that will be required of all schools in a few years.
What more do you need to know? The standard is arbitrary, as confirmed by a cursory review of actual CPI scores in the best districts.
I also object to the fact that the state and federal government are stripping control of our local school away from parents and our school committee, and that the prescription for fixing schools identified as "inadequate" has no better chance of bringing most schools to "adequate" than the status quo.
The combination of mandate plus prescribed sanctions is arbitrary and capricious. That's a legal standard, but given that there will be 20 million suburban parents up in arms about state takeovers of their kids schools within a few years it's also an electoral litmus test.
The mandated scores alone are simply arbitrary.
Where the cut point is set is not arbitrary, but rather is a political decision because education is controlled by elected politicians.
You think it's too high, others think it is too low. There is no right or wrong based on this criteria.
Steve is trying to get to the point that using aobjective criteria, there is little doubt that the cut point has been set low.
As things are, schools are leaving some kids behind because they need to get from 60% to 70% proficient to stay ahead of sanctions. That means they triage resources away from the top 55% and the bottom 25%, so they have a prayer of getting past sanctions for another year. If they pass, the principle keeps his job. If they fail, the principle gets the axe. What would you do if you were in charge of allocating resources to meet a mandate to improve passing rate next year or lose your job?
So this means that whatever the school was doing for remaining 20% (100% - the top 55% and the bottom 25%) of students before the NCLB, they weren't learning to read and do basic maths sufficiently well enough to pass the tests.
Now the NCLB is pushing the school to ensure that this 20% of kids learn to read and do basic maths, when before they weren't.
It is a grave pity that the school isn't attending to the bottom 25%, but that will come as the required passing rate increases. Too late for the individual students I agree, but before the NCLB they also weren't learning to read and do basic maths.
This shifts us on to the question of the deprived top students. If schools cannot both teach every kid (except the severely cognitively-disabled) how to read and do basic maths, and challenge the stronger students, then it is a value judgement as to which is most important. And, as someone who was massively bored at school by its slow pace, I favour teaching every kid (except the severely cognitively-disabled) to read and do basic maths, even at the price of directing resources away from the stronger students to the poorer ones. The impact on the lives of those who are illiterate and innumerate strikes me as a far worse outcome. Of course, this assumes that schools cannot accomplish both goals. If a school can both teach every kid (except the severely cognitively-disabled) to read and do basic maths and challenge the stronger students, then I am all in favour of it.
What would you do if you were in charge of allocating resources to meet a mandate to improve passing rate next year or lose your job?
Allocate the resources to teaching more kids how to read and do basic maths, and assuming I achieved the outcome, feel happy that I'd gotten 20% of my students passing the exam who weren't passing before, and depressed that 25% were still failing, and fret about how to improve efficiency so that 25% were also brought up to standard.
What principal could regard 45% of their students not being able to read or do basic maths as a success? Surely your principal would be tearing his/her hair out over this regardless of the NCLB? If it's only the risk of losing the job that makes your principal redistribute resources to make sure more kids learn to read and write, your principal is not being very professional by my books. The NCLB is just requiring the principal to do what the principal should be doing anyway.
" my objection to the tests is that they were not conceived as a test to set an arbitrary level for "proficient" that all students could ever be expected to achieve."
Are you saying that some other "kind" of test is required? It's not just a matter of setting a proper cut-off point?
"The standard is arbitrary, as confirmed by a cursory review of actual CPI scores in the best districts."
Arbitrary? You could complain that the levels are too high, but how are they arbitrary? Do you think the results of the tests do not give us any useful information?
"I also object to the fact that the state and federal government are stripping control of our local school away from parents and our school committee ..."
What are they stopping your schools from doing?
" ... and that the prescription for fixing schools identified as "inadequate" has no better chance of bringing most schools to "adequate" than the status quo.
The status quo is better? I don't think so. NCLB doesn't tell schools how to solve the problem, just that they have to recognize and quantify the problem. You may argue about the cut-off point, but it's up to the schools and local communities to solve the problem.
Tracy W says it best:
"The NCLB is just requiring the principal to do what the principal should be doing anyway."
There is nothing arbitrary about this.
Ken, I wouldn't have expected you to be a relativist on the idea whether there is a right or a wrong threshold.
If you support the idea of a threshold at all, you ought to be able to justify it based on something more rational than politics.
You can't identify an appropriate threshold because there isn't one. A threshold appropriate to challenge a school in a district already a standard deviation above median is not appropriate for a school at the median, or below the median. Any threshold-based score distorts incentives, but a threshold that isn't calibrated to an achievable goal - on a school by school basis, or better, on a kid by kid basis if you really want this to be about leaving no child behind - doles out random punishment and rewards.
As I've demonstrated, the standard in Massachusetts eventually mandates schools reach a point on the curve about three standard deviations above the current median CPI.
"As I've demonstrated, the standard in Massachusetts eventually mandates schools reach a point on the curve about three standard deviations above the current median CPI."
This is just wrong.
First, I doubt that the threshold is arbitrary or based only on politics. However, any cut-off is based on opinions that will be influenced by politics. In our state, local teachers are used to carefully evaluate the test and define the cut-offs for each level of proficiency. You need to find the document for your state that defines how this is done to prove your case that it is arbitrary. As I said before, I haven't ever seen a minimum proficiency standard that was anything but trivial.
Second, the test is not about the average skill or testing level of all students at a school. It is about a minimum level of proficiency. To meet the requirment does not require a school to be as good as BLA. It just means that they can't ignore the kids at the lower end. It is easy to see why BLA might not meet the requirements. Perhaps they assume that if a student doesn't do well in school it must be their fault. It is easy to let lots of kids fall through the cracks.
"Any threshold-based score distorts incentives..."
Of course, the problem is that a criterion based only on the low end (which should be really easy to achieve), might cause a school to shift most of their resources away from the better students. And, there is no guarantee that it will improve the curriculum or teaching methods.
That is the best criticism of NCLB that I can come up with. The cut-offs are not arbitrary. They are so low that there is no need to custom-fit them to individual schools or individual students. If you push that position, then you need to show some real test questions and explain how it is OK for students to pass on to the next grade without knowing that material. In fact, in our state, they only have to know half the material on the trivial test.
Post a Comment