February 16, 2007

Higher order thinking test

(Update: See Part Two here.)

Supposedly the goal of education is higher order thinking which can be defined as:
A complex level of thinking that entails analyzing and classifying or organizing perceived qualities or relationships, meaningfully combining concepts and principles verbally or in the production of art works or performances, and then synthesizing ideas into supportable, encompassing thoughts or generalizations that hold true for many situations

It's commonly thought that these higher order thinking skills can be taught directly apart from the relevant domain knowledge (i.e., a narrow portion of knowledge that deals with the specific topic of interest). The thought is that you don't need to learn (i.e., memorize) all those messy facts because you can use your fancy higher order thinking skills to figure out whatever you need to know. Thus, instructional time is concentrated on higher order thinking skills and the learning of facts is downplayed.

Let's put that theory to the test.

No doubt if you enjoy reading (or at least take the time to read) an obscure education blog you went to college, are highly educated, and are smarter than the average bear. In other words, you have higher order thinking skills in spades. Let's test how well you can use them.

Consider the following:

You have two identical glasses, both filled to exactly the same level. One contains red dye, the other water. You take exactly one spoonful of red dye and put it in the water glass. Then you take one spoonful of the mixture from the water glass and return it to the red dye glass.

Question: Is there more red dye in the water glass than water in the red dye glass? Or is there more water in the red dye glass than red dye in the water glass? In other words, the percentage of foreign matter in each glass has changed. Has the percentage changed more in one of the glasses, or is the percentage change the same for both glasses?

Use your superior higher order thinking skills and intuit an answer. First try to do it without resorting to outside sources. Then try answering it using whatever reference source is handy, such as google.

NB: This only works if you don't know the scientific principle involved. If you happen to know the right scientific principle, you're relying on your domain knowledge to answer the question, not your higher order thinking skills. Also, no fair if you know the source of this problem.

I'll let you stew on it for a while.

Partial Update: Hint--Instead of water and red dye, think of red balls and white balls. Assume that each glass starts out with 100 balls of a single color. Now remove a number of red balls from the red-ball glass and put them in the white-ball glass. Then return the same number of balls from the glass with the “mixture” and put them in the red-ball glass. Do this with different numbers of red and white balls.

15 comments:

Parentalcation said...

Oh... pick me... I know... I know!

(Actually, it would be cheating if I answered, since I recently read the same thing you did)

MikeZ said...

I worry about your saying "If you happen to know the right scientific principle, you're relying on your domain knowledge to answer the question, not your higher order thinking skills."

Surely, HOTS must be based, at least partly, on domain knowledge.

I don't know if this is a good argument, but is it "fair" to resort to the balls analogy?

Another similar problem is the "two jars" problem. The most recent one I saw is this (boiled down to its basics): You have two unmarked jars, one 10 oz, the other 6 oz , and an ample supply of water. Measure out 8 oz.

If part of your HOTS involve modulus arithmetic, it's not that hard.

I would also suggest that the goal of education is not HOTS, but CT (critical thinking). It may turn out that the two are related, but CT involves weighing arguments to detect flaws, considering and evaluating alternatives, and probably a few other things I can't think of right now.

Clearly (to me, at least), critical thinking skills are a requirement for getting through today's world of mass media barrages, of a Web where everything is available - and some of it true.

rightwingprof said...

"I would also suggest that the goal of education is not HOTS, but CT (critical thinking). It may turn out that the two are related, but CT involves weighing arguments to detect flaws, considering and evaluating alternatives, and probably a few other things I can't think of right now."

Really? According to whom? Among the academics I know, CT involves repeating the PC party-line, and has nothing to do with thinking or reasoning at all. Rhetoric, logical argumentation, and what was once termed CT are all out of favor in educational institutions.

nbosch said...

MikeZ poses this problem:
You have two unmarked jars, one 10 oz, the other 6 oz , and an ample supply of water. Measure out 8 oz

Fill the 6 oz; pour 6 oz in the 10 oz jar; fill the 6 again; pour in the 10 oz (2 oz left over) empty the 10 oz down the drain. Pour to 2 oz into the 10 oz; fill the 6 oz and you have your 8 oz. We've done a ton of these problems in our class--the hard ones are when you can't dump the water out!

I've taught gifted kids (K-6) for 21 years; a lot of them just think differently. Many gifted kids think most of the time in higher levels like synthesis, analysis and evaluation. It becomes apparent when you "listen" to them think. Can you teach higher order thinking? For arguments sake, I'll say "no". But one of my frustrations with education today is there is no time for kids to think, period!! The day is filled with so much info that will eventually be tested that kids, at least in our district, don't have time to think, wonder, ponder, "what if", or daydream.

Michael said...

Among the academics I know, CT involves repeating the PC party-line, and has nothing to do with thinking or reasoning at all. Rhetoric, logical argumentation, and what was once termed CT are all out of favor in educational institutions.

Geez, what academics do you know? No one outside of the Economics Department at my schools looks at critical thinking that way.

Michael said...

That should be "school," not "schools." Yeesh.

Tracy said...

CT involves weighing arguments to detect flaws, considering and evaluating alternatives

By this definition, I'd say that critical thinking is a subset of Ken's definition of higher order thinking.

While weighing arguments to detect flaws and considering and evaluating alternatives are of course highly important skills, they are not the only useful skills. Some other ones are making up your own arguments (as well as detecting the flaws in other people's arguments) and creating alternatives (as well as considering and evaluating them). Another HOTS is outside the line of arguments - art and performance are highly important - such as the creation of a beautiful home.

A school that only limited itself to teaching critical thinking skills would be ignoring a lot of human possibilities. I personally prefer the education system to take a wider view of what students should learn than you advocate here.

rightwingprof said...

"Geez, what academics do you know? No one outside of the Economics Department at my schools looks at critical thinking that way."

Sounds like you don't have much contact with anybody outside your department, then. Check out your local English department (they're supposed to be teaching that in freshman comp), join an academic mailing list, go to teaching symposia at your local ed school.

Logic hasn't been taught in universities for ten years. It's cultural imperialism. Just one "narrative" out of many, all of which are equally valid.

It's called postmodernism.

Michael said...

Oh, I know plenty of academics outside my department. Most of them are too busy teaching to worry about pc-ness of either right or left. I used economics to make a point: unexamined orthodoxy isn't good for thought, and you'll find it nowhere more than in econ departments.

rightwingprof said...

"unexamined orthodoxy isn't good for thought, and you'll find it nowhere more than in econ departments."

Why do you think so many business schools have their own econ departments, when the university also has an econ department?

Tracy said...

I used economics to make a point: unexamined orthodoxy isn't good for thought, and you'll find it nowhere more than in econ departments.

This is not my experience of economic departments. I don't think your economics department can be very good at preparing students to be economists. For example, there is a long-running debate in macroeconomics between what is broadly termed the Keynesians, the monetarists, the neo-classicals, and the real business cycle theorists. An economics department that didn't teach students about these different ideas would leave them unable to understand many economic articles, let alone the real world.

Also in macroeconomics there is a long-running debate between different theories of economic growth. Again, an economics department that didn't teach students about the arguments on either side of these debates wouldn't be preparing their students very well to understand economic articles.

And of course, in teaching these debates, a professor must at least be weighing arguments in front of students and considering and evaluating alternatives, which you define as the core of critical thinking skills.

In my experience most economics schools do seem to teach these debates, as I have had arguments about economics with people who studied at a completely different university to me and they generally know what I mean when I say something like "monetarists" or "neo-Keynesians". So I think you must have run into a few incompetent economics departments, which is a shame.

Tracy said...

And of course, like all sciences, in economics the status goes to those who challenge the orthodoxy. Not much status goes to people who just make the same arguments again and again. Look at the people who win the nobel prize in economics - Robert Lucas for overturning adaptive expectations and completely changing monetary policy, Milton Friedman for taking on the Keynesian orthodoxy, Kahneman and Smith for starting to integrate psychological insights into economic analysis, Coase for completely changing environmental economics.

The incentives of science encourage amibitious scientists who want to be famous to challenge the current orthodoxy. Furthermore, I suspect that many people who are smart enough to be good scientists don't want to spend their lives repeating the same stories, they prefer the excitement of working on something new and challenging, even if they don't care about professional respect and status.

If your economics department is turning out people who just repeat economic orthodoxy then, to be blunt, it's not got any future Nobel prize winners in it (unless they're getting their critical thinking skills elsewhere). It's going to have a hard time attracting good economists and it's going to go downhill as all the interesting and high status work is done by other economics departments.

rightwingprof said...

"Most of them are too busy teaching"

They must not be tenured, then. I've got news for you. Four or five classes per year isn't "busy teaching"; it's not "busy" anything. Five, six, or seven classes per semester is "busy teaching."

dweir said...

First, can someone please clarify -- is the working definition of HOTS used here the one from Bloom's taxonomy?

Second, lovely problem. You've titled it a "higher order thinking test", therefore I assume you mean to use it to assess HOTS. How do you see that playing out?

A couple points in the comment thread caught got me thinking -- Can HOTS be taught or assessed? Are arts/performance defacto HOTS?

I used to teach music, and I used to teach math. I always found it interesting that people would describe a student who performed above their peers in music as "talented". I almost never heard anyone describe a math student as "talented".

Surely there is innate ability. But, that doesn't mean HOTS isn't teachable.

A colleague of mine (an art teacher) said it best: Talent is the observable manifestation of acquired skills.

Feynman learned portraiture by acquiring the skills. The most "talented" musicians are those with the best skills. Most jazz musicians started by memorizing their scales and modes. Does any other human endeavor have shortcuts?

BTW... I ran your puzzler by my ivy-schooled, smarter-than-me husband. While he came up with the same answer as I, he started first with something else. I won't say anymore until you've given the answer.

MikeZ said...

rightwing prof: "According to the academics I know ..." - that says it all. Academia has tried to bury CT under a death-bed of Political Correctness, and seem to be suceeding.

To skip to another of your comments, tenure is what made academia what it is. In business, you have to deal with people with different viewpoints - many of them are your customers, and some are your co-workers. So you get to hear different opinions, and you have to deal with them. It's probably no surprise to you that in academia, if someone has a different viewpoint, you just don't vote them tenure, and you don't have to suffer all those obviously wrong ideas.

tracy: "A school that only limited itself to teaching critical thinking skills would be ignoring a lot of human possibilities." That's a good point. Thinking is a good thing, but you have to have things to think about. I read somewhere that the Japanese model of school is that the teacher presents information, and the students absorb as much of it as they can. Our model is a little better - we present information and talk about it, about what it means, the consequences &c. If all we taught was CT, all we'd get is students who were very good at logic puzzles.

But if we don't teach thinking skills, we end up with a population that thinks news about A. N. Smith and Britney Spears is supposed to be front-page stuff.

dweir and nbosch wonder if "thinking" can be taught. I think it can. Just as some talented people can play the piano far better than others, and some other talented people (I believe there's a big overlap here) can solve differential equations, some people seem to have a knack for "figuring things out". It's all a matter of practice. The 1960s were a great time for writing about creativity. One I remember is Alex Osborn - the guy who came up with the concept of brainstorming. The there's the Betty Edwards book, "Drawing on the Right Side of the Brain", ... I just found another one: Lateral Thinking, Edward de Bono, 1967.

Perhaps a psychologist could say why that movement seems to have died out. Is it because there wasn't enough "there" there, or because it's just "too hard", or because we all lost interest?

I think you can teach thinking the same way you teach everything else: by example, by practice, and by applying the principles outside of class.