August 23, 2007

Technology for Technology Sake

Let's face it.

Most schools don't know much about education. Sure, most of them do a swell job of educating the easily educable--the bright kids who are self-motivated to learn. Schools which are blessed with many of these kids handily make AYP and are viewed as being successful schools. That's because it doesn't take a whole lot of skill to teach these kids. Educating these kids is like surfing; as long as you point yourself in the right direction and are somewhat careful you can ride the wave all the way in without falling unless you do something really stupid (i.e., anything "student centered").

On the other hand, these same successful schools do an awful job of educating the dim and/or unmotivated. And, woe be to the school which is burdened with many of these kids. We call these schools failing schools because they don't make AYP and their students lag behind grade expectations.

In this way, we conveniently label schools based on which category of kid predominates in the school. But, ostensibly, most schools are the same as far as ability to educate goes. That is, not very good. Schools know this. It is their dirty little secret. That's why they hate NCLB--it draws attention to their short-comings, and they'd rather you not know about it.

The result is that almost all schools are looking for a solution to their education woes. Maybe they have a school full of under-performers. Or maybe they have a persistent pocket of under-performers. In both cases, the problem isn't going to go away any time soon. This is because schools don't know how to make the problem go away. More accurately, they don't want to institute the rigid curricular and quality control measures that are necessary to improve educational outcomes.

So, they look for the next best thing. The magic bullet. And, the magic bullet of choice is new technology.

New technology is great if you are a clueless educator. You get to spend money on the latest must-have toys. Lacking this money, you get a good excuse to plea for increasing your budget to acquire said new toy. In any event, eventually you'll get your new toy, special interests dominate diffuse interests. And, with new toy in hand, you'll get to show parents that you're actually doing something about improving education. You are a caring educator, a pillar of the community. If you are media savvy, you'll be able to snare a gullible journalist, who surprisingly knows even less about education than you, who'll come over to look at your new toy and write a breathless story about you and your new toy.

Kinda like this story in the Arkansas Democrat Gazette.

Here's your script. When journalist arrives to see your new toy make sure you have a young student available to demonstrate something that looks like education. Bonus points if the student is photogenic and young enough to cuddle. Extra bonus points if the student comes from an historically underperforming minority group. Super bonus points if the student is mildly retarded or has some other disability, preferably one of those fashionable learning disabilities.

The day took a decidedly atypical turn for one class when Angel Chavez, 5, wrote his name on the classroom board.

Instead of chalk or marker, Angel picked up an electronic pen and scrawled his name, as best he could, across the class’s new interactive white board, a computer screen sensitive to touch that also runs computer programs, streaming videos and Web sites. It’s one of 25 going up in Springdale classrooms this fall.


Next, display as many of your new toys as possible making certain to explain how much money you're spending (in education and technology more money = good) and how the technology is needed to connect with today's tech-savvy youth.

Arkansas educators will spend millions this year upgrading classroom technologies. The rollout figures to be particularly striking in Northwest Arkansas, a region state officials say is already home to the most hightech classes in the Natural State. Whether it’s electronic white boards, software that teaches students to read or even Global Positioning System devices for math classes, the new gadgets are aimed at one goal: plugging a generation of computer-savvy youth into learning.


Then trot out your technology director (if you don't have one dress up a custodian) and have her say something positive about your new toys. Bonus points for cramming in as many edu-buzzwords, like learning styles, differentiated instruction, and diversity, as possible.

“Kids in school now live in a multimedia world,” said Kathleen McInroe, technology director for the Bentonville School District. “They text message. They use cell phones. They do YouTube.

“ Learning in a visual, interactive kind of way has become their best learning style. We can’t ask them to leave that at the school door because we have other ways we are more comfortable teaching.”


Now, hold out the tin cup because even if you're flush with money you can never have enough of it. Bonus points for incorporating a heartwarming story of how you're already fleecing Uncle Sam to buy your existing toys. Message: you care.

FINDING THE MONEY Melanie Bradford, head of the Arkansas Department of Education’s research and technology division, said there is little state or federal money earmarked for technology in schools.

The only significant revenue stream from either source is the federal Enhancing Education Through Technology grant program, Bradford said. The program sent $ 2. 4 million in competitive grants to Arkansas schools with large numbers of low-income students last year.

Beyond the federal dollars, it largely falls to individual schools to squeeze money out of existing budgets to bring technology to their classrooms, Bradford said.


Now bring the journalist to a classroom in which many of your new toys are being used. Noe do a little compare and contrast, i.e., this is how we used to do it in the dark ages and this is how we do it now in the gilded age. Be sure to claim that educational outcomes are improving. Journalists are either too lazy to check the stats or won't understand them if they do. Bonus points for calling your profligate spending an "investment."

EVOLVING TECHNOLOGY Barbara Brannan has seen the region’s investment in technology first hand. Brannan is a science teacher at Holt Middle School in Fayetteville. She’s seen technology transform the way she teaches after teaching science in Fayetteville for more than 30 years. Before, she used transparencies, slides, and a video — if her school’s one television set was available — to teach concepts like the scientific method.

Today, Brannan and her students use laptops, document cameras, classroom clickers that let students answer questions electronically, and even GPS units to master the same concepts.

Incorporating technology is such a priority at Holt that administrators are starting to consider digital offerings when deciding whether to adopt new textbooks.


If the journalist does the unthinkable and asks whether student performance has improved with your new toys don't panic. Suppress your urge to run. Remember, he's not looking for the truth; he's looking to find pretty numbers that validate the prevailing viewpoint which includes the meme that technology is good for education.

The solution. Lie. Better yet, find someone with credentials to misrepresent the facts for you. Bonus points if your "expert" can actually spin the truth enough to get the truth printed while fooling the journalist. Hint: The word "correlation" is your friend and, in education at least, the word "research" can include things that aren't research. Be bold.

THE BENEFITS Northwest Arkansas educators give different answers when asked whether investing in technology leads to increased student achievement. Part of the reason for the discrepancy may be because there is limited research on the correlation between technology and learning.

The Metiri Group, a California based educational consultancy, put out a report in 2006 summarizing the existing research on the educational impact of technology in the classrooms.

The report found that technology provides a “small, but significant” boost in learning when implemented carefully.


Genius. Almost no one who isn't an economist understand that there are opportunity costs in adopting something that yields “small, but [statistically] significant” instead of something that yields results that are large and educationally significant.

The story practically writes itself. Now go out there are do some good, the world needs more heroes.

29 comments:

CrypticLife said...

I was looking forward to the picture of the cuddly minority learning-disabled kid.

I feel cheated.

Anonymous said...

Ok, so what works?

Direct Instruction gets the students to the 4th grade level.

But it doesn't teach enough high frequency vocabulary to get students to the 8th grade level, right?

Why not use technology to teach the general word list and academic word list? (Please google "academic word list.)

It's less than 3,000 words. The gains should be the dramatic ones we are looking for...

But even if it works, it still leaves critical reasoning skills as the final hurdle to get students to grade level...

Can't technology be used to develop those?

Sorry to keep revisiting the same issues, but it's hard to criticize the edublob if we don't have better alternatives.

KDeRosa said...

Crypticlife, you and me both, but I had to make an executive decision on that one.

eric said...

Here's the technology I would use to teach word lists. As for critical thinking, read the WaPo Pogrow piece.

KDeRosa said...

Anon,

DI can get students up to a legitimate sixth grade level in math, reading, spelling, writing, grammar, reasoning, and history which is to say up to the content area subjects. At that point the need for a specialized DI-like curiculum diminishes, but using DI-like teaching techniques can certainly be helpful for scaffolding these difficult to educate children.

Of course, there is little hard research for these later grades.

We're talking bout the full school implementation which has all the vocabulary, fact, reasoning, and concept knowledge that's needed.

And there probably are alternatives to DI depending upon the population of the school.

I'm not saying that there is no role for technology, just that most of that technology isn't really all that useful if the instruction is still substandard and really isn't needed as much if the instruction is improved.

It's mostly fluff.

KDeRosa said...

Eric, isn't it true that HOTS hasn't been legitimately researched yet (as in with control groups), I remember reading that somewhere?

ts said...

Twenty years ago when I was in grad school (in telecoommunication systems), one of my IT professors said that we do too much with technology just because we can, not because we should. It appears that nothing has changed.

Anonymous said...

While I do agree that throwing tech and $ has no benefit beyond the soundbite. You are guilty of buying into the same mythology as the clowns with the tech.

Read the article in the current Harpers. Ed is in no more trouble now that it has ever been.

You suck for blaming the kids. Our anti-intelect culture and other family factors define the kids. But you are right in that the school is a very small factor. Kid "A" will perform in any school.

But kid "A" isn't the problem now is he?

Anonymous said...

Your article ran SO TRUE. As a teacher who has been in education for several years and worked at several different schools in different socioeconomic areas, I can testify to the main points of the article.

I once worked at a school that was big on "technology". It is a Title I school that has barely made AYP. But the principal liked the toys and gadgets and the ability they gave him to brag about his school.

We had LCD projectors in the celings of classrooms, mobile wireless carts, digital cameras, color laser printers, color scanners, new computers (so many laptop computers, in fact, they were left lying around in classrooms and the media center unsecured). Did it improve student achievement? NOPE!!!!!!!

We had teachers who had kids playing on the computers all day. Surfing the web. Typing on blogs. Typing stories and writing on Word. Were the teachers teaching reading? H_ll NO!!! Were the teachers teaching math, science, and social studies? H_ll NO!!!

Our system is currently "rolling out a pilot program" called 21st Schools or some garbage like that. They are going to put LCD projectors, Smartboards, etc in all the classrooms. The program costs MILLIONS.

Student achievement will not go up one inch. Most of the teachers will be forced to undergone "training" to "effectively integrate technology to improve student learning." After being "trained" on the new garbage, I mean equipment, FEW teachers will actually use that stuff-- PERIOD.

Reality is that it takes more effort to actually use the stuff than what the students get out of it. Sure there is some technology and programs that are useful, but they don't supercede good traditional teaching.

NYC Math Teacher said...

One cannot overemphasize the do-nothing Technology Coordinator jobs that result from this overinvestment in gadgets. My school has teachers who teach nowhere near a full schedule but who somehow coordinate technology efforts throughout the school.

And here we thought Tony Soprano's construction guys had it easy.

KDeRosa said...

Anon 6:03,

1. I don't claim that public education ever had a halcyon era.

2. I'm not blaming the kids. I am merely recognizing that some kids are more difficult to educate. Ultimately schools should be responsible for educating the ones who are educable, which includes all but about 5% to 10% in the lower grades and perhaps less in the higher grades.

3. We don't know that culture is causing the performance problem. The causation could be the reverse with low IQ persons causing the anti-intellectual culture.

4. Schools could be having a much larger impact on education outcomes than they currently are if they knew how to educate, which they don't.

KDeRosa said...

NYC Math Educator, it could be that your Tech guys are Tony Soprano's no-work construction workers.

Eric said...

Eric, isn't it true that HOTS hasn't been legitimately researched yet

Poor vetting of best practices is likely the nation's premier educational risk. Niche interventions like HOTS simply wouldn't hit the radar of "legitimate research." I've seen good data from HOTS, albeit via Dr. Pogrow himself.

FWIW, I believe HOTS made the National Diffusion Network when it was around. Unlike other programs on that list, I believe HOTS deserved to be used more widely.

One might hope that "legitimate research" would mimic Follow Through by comparing the "most likely to succeed" contenders. From what I've seen, it's more likely to compare the favored solution to random controls. If "legitimate research" were designed to find solutions, I'm sure HOTS would have received due attention.

This is a case of choosing "whatever works" when the "what works" list is empty.

Eric said...

Ken, you'll like this:

The Politics of Knowledge: The Carnegie Corporation, Philanthropy, and Public Policy by Ellen Condliffe Lagemann

Page 231 (on Carnegie-funded education experiments):
These experiments varied considerably in orientation. Some were essentially behaviorist, as was certainly true of the "intellectual pressure cooker for children from the slums" operated by Carl Bereiter and Siegfried Engelmann at the University of Illinois in Urbana. Here, four- and five-year-old children were drilled to recognize, describe, and differentiate among letters, colors, numbers, and shapes. According to Maya Pines, a freelance writer who prepared a book about early learning experiments for the Corporation, the goal was to "do systematically , and artificially, what middle-class parents do naturally at home"; though very controversial, it appeared to work." According to J. McVicker Hunt, children enrolled in the program "gained about one year of psycholinguistic ability" in a three-month period.

KDeRosa said...

I think Engelmann alluded to that "study" in his book last year.

Eric said...

children enrolled in the program "gained about one year of psycholinguistic ability" in a three-month period

The first time I read this, I said WOW! After a bit of thought, I realized that's an effect size of 3. I can only say WOW WOW WOW!!!

Anonymous said...

Thank you Kderosa and Eric for responding to my questions. I really appreciate it.

kderosa:

Can you point me to the evidence that DI will get low income students to the 6th grade level?

My understanding is that while the program goes up to 6th grade, many 6th grade students will test around the 4th grade level.

Is there a school that has actually gotten grade level test scores with 95% of its low income 6th grade students?

Eric: The game you refer does not seem to teach words according to the frequency research. (see "academic word list")

All: Will someone please google "academic word list" and comment?

I think this is a missing link.

And everyone seems to agree that there is no high quality research on solutions for teaching 8th grade level critical reasoning skills, correct?

KDeRosa said...

Anon, see the Baltimore City schools that were part of the Baltimore Curriculum Project from 1998-2003. See especially, the City Springs School which tested very highly on both the math and reading portions of the CTBS. These students were completing the sixth level of the DI programs in a whole school implementation. The effect size for this school approaches 2 standard deviations for the 2003 school year. The average effect size was about 0.75 sd.

The better implemented schools were getting students to score above the 40th percentile on the fifth grade CTBS in 2002.

So, no DI will typically not get 95% of severely disadvanted fifth graders to grade level (as measured by the CTBS), but it should get about half of them there.

I interpret this to mean that a typical school should be able to get about 90% of its fifth grade students to grade level (based on a legitimate measure).

KDeRosa said...

And everyone seems to agree that there is no high quality research on solutions for teaching 8th grade level critical reasoning skills, correct?

I'll go one further and say that there is no such thing as a 8th grade level critical reasoning skill.

What is an 8th grade level critical reasoning skill? The ability to do algebra and geometry? The ability to comprehend and summarize an eighth grade text?

dweir said...

Technology that is nothing more that packaging for the same poor content will do nothing to improve learning. Good technology -- and good uses of technology -- extends the reach of an effective teacher. For example, I used a music program that allowed students to drill exercises or even sections from their music. This freed me up to work with a smaller subset of students or it allowed me a way of conducting individual performance assessments while still teaching the rest of the class. After school, I could go back and review the work done by the students who worked with the computer.

While it is still unfortunately common to have experienced educators who are not computer literate, this should be a non-issue in the next 5-10 years. Younger teachers know they can be more effective with technology:
- communication with parents
- providing online office hours at night
- tracking and analyzing performance data -- and most importantly being able to respond to that data

That said, if there isn't already a good low-tech means of tracking and using data in place, introducing technology will not help. Technology can only build upon good practice, not invent it.

Anonymous said...

KD-

Thank you very much for your thoughtful responses to my posts.

I am really trying to learn as much as I can.

I have been convinced for years of the value of Direct Instruction.

That said, City Springs test scores seem to have "spiked" with the 60-70 students you cite.

For example, only 18% of city springs 5th graders are above grade level on the maryland standards test in 2007. (at least according to greatschools.net)

More telling: Less than 60% of the 7th graders are at grade level on the 2007 msa.

I don't point this out to diminish the accomplishments of city springs, which are fantastic.

Rather, I am wondering if Direct Instruction really gets the students all the way to home plate.

8th grade critical reasoning skills are those cognitive skills required of a student who is fluent and possesses an 8th grade level vocabulary in order to score at grade level on a standardized test.

Also: Can you please google "academic word list" and comment?

Thanks!

KDeRosa said...

two things:

1. I believe the Maryland scores include lots of transfer students whose scores aren't indicative of the education they received at city springs

2. city springs changed what it was doing after 2003 and NIFDI stopped consulting as a result, hence the spike.

Anonymous said...

"The better implemented schools were getting students to score above the 40th percentile on the fifth grade CTBS in 2002."

--- 50th percentile is grade level, not 40th, so even the better implemented schools were not getting students to grade level, correct? ---


"So, no DI will typically not get 95% of severely disadvanted fifth graders to grade level (as measured by the CTBS), but it should get about half of them there."

--I don't understand how this follows logically from the statement above it.

In any case, it seems like the best example of DI we can find gets less than 50% of students to the fith garde level, correct?

We are still a ways away from the ability to read at the 8th grade level, which I think may be a crucial "tipping point."

--Kederosa: Would you be willing to check out the research on the academic word list? These words are not taught by any DI programs, but they may be part of the solution to raising student achievement beyond the fifth grade level. Just google "academic word list".

Respect your thoughts very much and would like to see what you think.

KDeRosa said...

Anon, the project only lasted five years 1998-2003. These fifth grade scores are the first cohort. Now look at the first grade scores for the best performing school, City Springs. City Springs began the implementation back in 1996. So the 2003 cohort was the fourth cohort whose scores for first grade (1999) had still not peaked. SO I am extrapolating a bit from the data when I suggest that the better implemented DI schools should get fifth grade CTBS scores to at least the 50th percentile based on City Spring's performance trend.

And bear in mind that 50th percentile on the CTBS is a much higher benchmark than most state's NCLB tests.

The best example of what DI can do is considerably higher than 50th percentile. City Spring's scores are at the 80th percentile. That's up from below the 10th percentile in 1996. I don't know how well those fifth grade scores will translate to 8th grade scores in three years.

i will take a look at the academic word lists,, but I think it takes a lot more than learning a bunch of high frequency academic words to give these kids the background knowledge they lack.

Anonymous said...

Thanks for the excellent response.

The academic word list was generated from a corpus of college textbooks.

Mastering this vocabulary won't prepare a student to read the arts section of the New York Times. (That requires lots of background knowledge.)

But it will allow them to recognize 90% of the words in college textbooks.

Graduate students from other countries often lack "background" or "core knowledge" such as that in the Dictionary of Cultural Literacy. But they are still able to get advanced degrees from American Universties because they understand academic vocabulary--often because they crammed it before going to grad school.

If we can prepare low income students to succeed at the college level, that sounds good to me. :)

Anonymous said...

KD:

Also, I should note that I don't think you can extrapolate a test score trend.

It's especially shaky with direct instruction students.

DI does a great job getting students to decode. But fluency only gets you to the third or fourth grade level.

Beyond that, vocabulary and reasoning skills are critical.

DI teaches only a limited number of vocabulary words, perhaps due to the high number of repetitions required by the DI philosophy.

I think this is reason DI students have trouble getting to the 8th grade level.

DI does a better job of teaching advanced critical reasoning skills, such as the completion of syllogisms.

I really think high frequency vocabulary might be the "missing link."

Giving students background knowledge is also a great idea.

The concept of the "core knowledge" programs is very much like high frequency vocabulary instruction.

Hirsch says in the Dictionary of Cultural Literacy that it was developed by looking at the frequency with which cultural knowledge is assumed by the writers of the New York Times.

For example, "Paul Revere's Ride" was included as it is frequently referenced (and assumed familiar) in the popular press.

But it doesn't come up nearly as often at the word "system," for example.

So it is very inefficient to improve reading comprehension by teaching "background knowledge."

But by teaching students the vocabulary needed to comprehend high school and college textbooks, we equip them to develop background knowledge through both coursework and independent reading.

KDeRosa said...

Anon, I'm not extrapolating to 8th grade, I'm only extrapolating to 5th grade based on expected trends to fifth grade of similar schools.

By my count DI explicitly teaches about 400-500 words per year in the reading program alone at the third and fourth levels and beyond. That's quite a bit of vocabulary and doesn't include all the implicitly learned vocabulary. Is there a a program that is actually capable of teaching lower performers more vocabulary than this per year? And by teach I mean that the students have actually learned the words which is no guarantee in most programs.

Bear in mind that Wes Becker, one of the original DI guys, was one of the first educators to recognize that vocabulary acquisition is the major stumbling block in getting lower performers to perform at grade level in third grade and above. See here. The problem is not that educators don't recognize that this is a problem, the problem is actually teaching all that vocabulary and concept knowledge. This is where programs like CK and the AWL fail: the specify the what but don't know about the how to teach.

I think we can agree that DI, if well implemented, can get low performing students up to a fifth grade level (unlike most other academic programs) and that moving forward it will be necessary to further develop the advanced vocabulary and background knowledge that these students need and lack to make further academic progress.

Anonymous said...

KD:

Thanks for your excellent response.

We are on the same page: DI is a national treasure but it does not get low income students to the middle school level in reading comprehension. If implemented well--which is rare, I think-- DI may get them to the 5th grade level.

"The problem is not that educators don't recognize that this is a problem, the problem is actually teaching all that vocabulary and concept knowledge."

I would argue a few points:

1) awl is not a program, it is just a list of words students need to know

2) the genious of the awl is it is only 550 word long. As you note, you could teach the whole list in a single year!

3)

The problem is not HOW to teach -- DI principles of effective instruction can be applied to anything. We know "how."

The problem is WHAT to teach -- Direct Instruction programs simply do not teach the right words. Or at least not enough of them.

How does that argument strike you?

Good exchange by the way.

KDeRosa said...

I agree, the comments have been constructive and have moved the discussion forward.

DI goes up to fifth grade so I don't think it is a flaw that it isn't getting kids to a middle school reading level. That's the middle school's job. DI's job is to get the students prepared to do middle school work by getting them to a fifth grade level.

So, the question remains is DI preparing kids well for middle school work? But, I don't think the data exists for a definitive answer. Part of the reason why the data doesn't exist is because pretty much nothing but DI can get low performers up to a fifth grade level by fifth grade and even then DI is difficult to implement because the quality control/feedback mechanisms are foreign to today's educators. How can we do legitimate middle school research on kids that don't exist in the real world or that take five years to get through a well implemented DI program (which itself takes a a few years to stabilize)?

I think that reading comprehension takes more than knowing the words on the AWL, though certainly those words should be known by the time they become needed. I'm thinking this is about the high school level. Perhaps this could be part of the focus of a middle school curriculum?

I do think that when it comes to teaching reading comprehension that the HOW is still unknown. Today, we teaching things that we think contribute to reading comprehension, but no one nows how to teach it directly, assuming it can be. DI just happens to be an efficient way to teach, so to the extent that it excels in this area, it is do to the inherent efficiency not because it has the HOW part down.

I think the problem is a combination of what and how. Studies do exist that show that when low performing students have the domain knowledge needed to comprehend a dense reading passage that they can perform better than higher performing students that lack the needed domain knowledge. The problem remains teaching all the needed domain knowledge to comprehend a wide variety of academic reading passages. It is a time issue because teaching domain knowledge is not amendable to acceleration. Plus, lots of learning takes place outside of school. It's a difficult problem to contend with.