The "Story of Science" series by Joy Hakim tells the history of science with wit, narrative depth and research, all vetted by specialists at the Massachusetts Institute of Technology.
The series, which has drawn acclaim, chronicles not only great discoveries but also the scientists who made them.
...
"These books humanize science," Pavlekovich said.
"We teach students this equation and this theory or this topic and that idea, but we never discuss the scientist behind it or how that scientist made the discovery," he said. "It helps students to understand how they struggled and overcame great obstacles to do what they did."
That's lovely. But does it work?
The answer is: nobody knows. That's because in education they don't test their new products before putting them on the market. The publishers didn't set-up a controlled experiment in which they tested this brave new textbook series with a group of students and a competent teacher to see if the students learned as much or more than a typical science textbook. There's a name for this conduct: educational malpractice.
Here's the testing the publishers did:
Scientists and educators say that there are many ways to teach science but that Hakim's approach makes sense.
...
Hakim said MIT scientists, including senior research scientist emeritus Edwin Taylor, checked each chapter of the Einstein book.
That's nice, but does it work?
Nobody knows and nobody cares.
5 comments:
> The publishers didn't set-up a controlled experiment in which they tested this brave new textbook series with a group of students and a competent teacher to see if the students learned as much or more than a typical science textbook.
Just a few thoughts.
First, the question, "does it work," begs the question. Does it work at what?
The objective of these textbooks - as clearly stated by the publisher - is to tell people about the lives of the scientists that made the discoveries.
In this, they are almost certainly more successful than those textbooks that do not describe anything of the scientists. This follows from the general (and generally inassailable) principle that "A textbook containing X is more effective at teaching X than a textbook that does not contain X."
Second, the design of the research suggested in the post ("a group of students and a competent teacher") does not account for the fact that textbooks must also support incompetent teachers, parents, self-learners, and more.
Third, there is the presumption that a "controlled experiment" can tell us whether or not a textbook 'works'. this is almost certainly false. A controlled experiment can measure for simple cause-effect relations ("did the introduction of textbook A cause efect Y"). But learning is not a simple cause-effect environment.
Things to consider in the design of an experiment ought to include (but almost certainly won't) the nutrition level of the student, the textbook being replaced, the prior experience of the class (eg., were they always lecture based, do they have good reading skills, etc), religion, the height of the teacher, the time of day (and day of the week) the material is taught, and more.
All of these - and others - can have as much of an impact on the outcome as the textbook. Yet it is impossible to design a testing regime which accounts for all these variables - you can't simply hold them constant, because the textbook may have an impact in some situations and not others.
The best - and only - test is the one the publishers are actually undertaking: publish the textbook, make it available to the entire teaching population, see how many of them start using it and how many of them continue using it.
There is no excuse for calling this "educational malpractice". Such an epithet presupposes a legal certainty about educational testing. But the opposite is true. The testing proposed to assess the impact of the textbook is based on a naive and incorrect model of both learning and science, and ought to be dismissed out of hand.
Sometimes I get the impression that educational "research" is based on making topics interesting for the teachers (who have to teach the same stuff year after year) rather than the students (who have to actually learn the stuff).
Stephen, here are some of my thoughts on your thoughts.
The objective of these textbooks - as clearly stated by the publisher - is to tell people about the lives of the scientists that made the discoveries.
I disagree. The author believes this textbook to be a replacement for standard science textbook. See here. So, I'd say that the stated objective of the book is to teach science content by focusing on the lives of great scientists.
Second, the design of the research suggested in the post ... does not account for the fact that textbooks must also support incompetent teachers, parents, self-learners, and more.
Those might also be desirable subjects for research. But, since I am only concerned with how the book performs in a typical school setting, I'm satisified with the limited research results we would obtain from my proposed experiment.
Competent teacher was probably a poor choice of words. I should have said a teacher with the skill set presumed by the author. Ideally, the etxtbook would be tested in enough different classrooms to minimize teacher effects anyway.
With respect to the efficacy of education research, I'll direct your attention to Science Versus Basic Education Research for the rebuttal and the limits as to what we can gain from education research.
At least some of the factors you list are important to control for, should be, and can be.
The best - and only - test is the one the publishers are actually undertaking: publish the textbook, make it available to the entire teaching population, see how many of them start using it and how many of them continue using it.
This might be a valid test in a properly functioning free market with properly aligned incentives and rewards. But schools today do not function in that environment. Today, all this test would tell you is the preference of those who select science textbooks for schools, the incentives of these people are not necessarily aligned with student performance.
Such an epithet presupposes a legal certainty about educational testing.
I think you're seriously misinformed about the level of "scientific research" that has formed the basis of many malpractice decisions. Correlation studies with p < .95 are typically good enough.
I've read quite a few Hakim books from both the history and science series. They are not textbooks. You could call them an engaging narrative. However, the flow of the narrative (especialy of the science books) is constantly interrupted by inserts or sideshows. More integration would have been helpful.
I would recommend the series just for sheer reading pleasure or as a supplement to textbooks. The history series is marred by a strong leftist bias.
http://www.textbookleague.org/121hakm.htm
Second, the design of the research suggested in the post ("a group of students and a competent teacher") does not account for the fact that textbooks must also support incompetent teachers, parents, self-learners, and more.
On the other hand, if we carried out the test DeRosa proposes and we discovered that with a competent teacher and set of students the new textbook didn't offer any advantages over the old ones, then why bother testing for what would happen with incompetent teachers, parents, self-learners, underfed people, etc?
The best - and only - test is the one the publishers are actually undertaking: publish the textbook, make it available to the entire teaching population, see how many of them start using it and how many of them continue using it.
?!
What an expensive and dangerous way of testing a product! Your average engineering firm would go bankrupt if they did that sort of test. It is massively expensive to fix bugs in a product after it has gone to market. And the odds of any complex project like a textbook having no bugs is basically zero.
Post a Comment