September 29, 2009

Learning Science and Everything Else

Last April, Science Magazine, published a study in which researchers administered  a pair of physics tests and  a test of general scientific reasoning to new freshman engineering and science majors in America and China.  Not unsurprisingly, the Chinese students performed very well on the physics tests (forces and mechanics:86% and electromagnetism: 66%) since they had received many hours of physics instruction in high school.  The American students performed not so well (forces and mechanics: 49% and electromagnetism: 27%) because only about a third of American students take physics in high school, the rest learn physics concepts from general and other science courses.  In contrast, both groups peformed identically on the test of general science reasoning (74%). From this the researchers concluded:

[This] finding defies conventional wisdom, which holds that teaching science facts will improve students’ reasoning ability.

Our study shows that, contrary to what many people would expect, even when students are rigorously taught the facts, they don’t necessarily develop the reasoning skills they need to succeed,” Bao said. “Because students need both knowledge and reasoning, we need to explore teaching methods that target both.”

As I indicated in this post, this is not a fair conclusion from the study:

What? This isn't the conventional wisdom. The conventional wisdom is that that learning facts in a domain will improve the ability to reason in that domain. This wasn't tested in the study. What was tested in the study, via the FCI and the BEMA, was the students' understanding in the domain (physics) which was significantly higher for the Chinese students compared to the American students. Not surprisingly, the American students didn't understand much physics since they didn't learn many physics facts and their "scientific methods" instruction failed to fill the void. Constructivists take heed.
Naturally, I got some pushback from the usual sources.
When your long commentary concludes that the investigators misinterpreted the results of their own study, I know something amiss - with your commentary.

Perhaps there was something wrong with my analysis.  But if there were, there's also something wrong with biologist Paul Gross' more-detailed analysis in this month's American Educator which essentially concludes the same as I did. (HT Education Next blog)

Here, then, is an alternate view of the Bao et al. results. The Chinese students know physics. The American students don't. Now both groups face a different challenge--different enough from the standard physics problems so that the Chinese students' superior conceptual and problem-solving skills in physics provide no immediate advantage. The new challenge is to think about problems of a very simple scientific character, but in forms and subject-matter domains that neither group has encountered before. As the authors explain in their online supplementary materials, the LCSTR "measures fundamental reasoning components with simple context scenarios that do not require complex content understanding. This test design can improve the measurement of the basic reasoning abilities by reducing the possible interference from understandings of content knowledge." But if so, both cohorts will handle most of the questions on the LCSTR (or any challenge like it) the same way. The will need to think through each question from scratch--to find the best answer starting from elementary principles. That kind of thinking is slower and more error-prone (Ed: see this post for why) than the thinking available to a physics-savvy Chinese student taking the FCI or BEMA.
It's a very good article and analysis (and not just because Gross agrees with me) so you should read the whole thing.

The article is also relevant to our ongoing discussion on expertise and knowledge.  Gross gets right to the heart of the current debate.

Arguments for much more reasoning and less content (a necessary tradeoff, given the time constraints) in K-12 science begain decades ago.  Eventually, the idea became a catch phrase.  "Content" was redefined to function as a synonym for "facts" (or "mere facts") independent of reasoning.  But defining content that way is nothing more than a rhetorical move. No honest study of  science textbooks and lessons nationwide, not even from the benighted decades preceding the launch of sputnik, could conclude that just memorizable facts were required, with no reasoning.  Facts were (and are) taught, and facts must be learned if any discipline is to be understoood and practiced. The rhetorical flourishes of those arguing for more scientific reasoning have affected some people's perceptions, but they have not changed the reality that, in general, science curricula have never been exclusively lists of facts to be memorized, devoid of the means by which those facts are discovered and gain acceptance in the scientific community. (emphasis added)
Exactly.

Proponents of inquiry/discovery learning, 21st Century skills, more scientific reasoning are basically arguing against a strawman they've created to push their "agenda."  No one is arguing for the mere memorization of facts.  Simialrly, no one is arguing for the merely learning how to reason generally independent of any facts. 

Everyone wants students to increase their ability to reason generally.

One side believes that this can be accomplished directly in an environment in which the learning of content is downplayed or minimized in favor of various learning activities centering around learning the scientific method, data collection techniques, formal logic, critical reasoning/reading, performing scientific experiments, and the like.  Since time is limited in K-12 education, so devoting time to all these activities cuts into learning content.

The other side believes that this general reasoning ability is not something that can be taught directly but is the result of learning how to reason within a wide variety of different domains which entails learning the content inherent in each domain.  Sadly, there is no shortcut to learning, no mtter how badly you'd like there to be.  And, yes, some areas of the areas of learning pointed out by the 21st century learners do need to be improved; however, they need to learned independently for each domain.  For example, if you want students to know about how to conduct an experiment, you're going to have to teach in in physical science, then again in biology, and again in chemistry, and yet again in physics.  Yes, there will be some transference between domains, but the research tells us that the transference with be minimal.  each domain has its own special considerations which are doman specific.  I tried to illustrate this point in the context of critical reading in this post.

The debate is really about where the line should be drawn between how much content should be taught and what (often ignored) skills should be emphasized given the time constraints of K-12 education such that the student will be able to best build on their acquired knowledge base as they continue their learning after K-12.

That's the debate we should be having, but sadly aren't.

2 comments:

Stephen Downes said...

> Proponents of inquiry/discovery learning, 21st Century skills, more scientific reasoning are basically arguing against a strawman they've created to push their "agenda."

I don't have an agenda, at least, not so far as your world is concerned. You want to screw up your education system, feel free. It's not my issue; I'm not American and I don't have a stake in the American educational system.

I have attempted to respond to what I perceive as your misunderstanding of facts and reasoning, but the reason I do it is mostly because your blog shows up as part of my regular reading in educational technology, and while items worth citing are few and far between, I nonetheless see the rest, and sometimes, because I dislike foolishness, I respond.

Don't read any deeper motives into it than that. I really have no stake in your interminable campaign. What motivates you? I can't say. What end are you trying to achieve? I can only speculate. Do I care? No.

A sentence like this, for example, is just ridiculous: "if you want students to know about how to conduct an experiment, you're going to have to teach in in physical science, then again in biology, and again in chemistry, and yet again in physics. Yes, there will be some transference between domains, but the research tells us that the transference with be minimal."

It makes me wonder whether you've actually _studied_ biology, chemistry, and physics. They are not completely separate domains with separate sets of 'facts' that must be acquired (or as you put it, 'learned'). They form a part of a single system of reasoning, with multiple areas of intersection and overlap. And thinking experimentally is less a discipline, less even a skill, as a way of looking at the world.

Goodness, before telling us what science is, or how to learn science, I wish you'd read some people who have actually studied science, people who publish not in the American neoconospehere but in real academic publications, people like Imre Lakatos, Larry Lauden, Paul Feyerabend, Thomas Kuhn, Bas C. van Fraassen, to name just a few. And maybe brush up your history of the discipline with some Popper and some Hempel.

This isn't a plea for you to earn some knowledge or acquire some 'facts' in the discipline. A reading of these writers won't offer you a srtore of facts (or, more accurately, if all you get from such a reading is a store of facts, you will have misunderstood the readings and misapplied your time).

It's a plea for you to become immersed in the subject you are writing about, not necessarily instead of the political writers you so obviously enjoy, but at least alongside them, so you can acquire through this immersion a facility with the sort of skill reasoning in this field requires.

If you were immersed in this literature, and conversant in learning and science, you would see - you would _recognize_ how facile and how utterly _futile_ a characterization of the issue as "one side believes..." is.

And when you see something like this: "Yes, there will be some transference between domains, but the research tells us that the transference with be minimal. each domain has its own special considerations which are doman specific," you will see how utterly utterly wrong that is.

Anyhow, enough pro bono work for now. Back to my day job, where I eat stuff like this for lunch.

KDeRosa said...

Stepehn, your mistake in your exceedingly defensive comment was thinking that my post was directed at you, which it wasn't.

Also, to the extent you have an agenda, it is irrelevant since you don't set or influence policy. Just like me. Further, I put scare quotes around agenda to indicate I wasn't using the word in the traditional sense.

It makes me wonder whether you've actually _studied_ biology, chemistry, and physics. They are not completely separate domains with separate sets of 'facts' that must be acquired (or as you put it, 'learned').

Strawman: No one is claiming that they (or any other domain) are "completely" separate domains with separate sets of facts. There is much overlap. Of course, the non-overlapping areas fill years of undergraduate and graduate courses and all that fancy "general reasoning" doesn't work all that well until all that content is acquired/learned.

They form a part of a single system of reasoning, with multiple areas of intersection and overlap.


To the extent that there is an underlying "single system of reasoning" it is of little value in a domain with deep, specific content, such as biology, checmistry, and physics. That's what this study demonstrated.


And then the comment devolves into what is possibly the worst form of advocacy. Ostensibly, Stephen wants to show that my argument is wrong and that he knows better than I do. If my argument was weak, unsupported, and/or wrong, a truly knowledgable person, as Stephen claims to be, would be able to make short shrift of my argument with a flawless, concise counter-argument demontrating my ignorance and his erudition. And, yet Stephen utterly fails to deliver. Not only that, but he gives an argument riddled with appeals to authority which undermines his main claim to be a knowledgable critical reasoner. That's a double fail.