22 Oct 96
Tom Reeves

Reading these statements about truth, superiority, and winning, I was reminded of the work of Suchman (1967), one of the staunchest advocates of the experimental approach to research and evaluation. Suchman wrote: "The logic of this design is foolproof. Ideally, there is no element of fallibility. Whatever differences are observed between the experimental and control groups, once the above conditions are satisfied, must be attributed to the program being evaluated" (Suchman, 1967, pp. 95-96).

It is difficult to argue with those who write in terms of their approach being foolproof or infallible. It is also difficult to oppose an approach that is entrenched in the minds and actions of educational researchers brainwashed through semesters of inferential statistics and quantitative research methods, with scarcely an introduction to contemporary philosophies of science or alternative approaches to inquiry. I began to wonder about Steve Tripp's perspective on philosophy of science. And then he told us:

Ahh, now there is the problem. A positivist? Has anyone noticed that there is distinct contrast between the advocacy of experimental methods by educational researchers such as Steve and the critique of these methods by contemporary philosophers of science? Phillips (1987) remarks on this irony, "New approaches to the design of evaluations of educational and social programs are being formulate? that make the 'true experiment' seem like a lumbering dinosaur, yet some folk persist in thinking that dinosaurs are wonderful creatures" (p. viii).

My own, admittedly fledgling, understanding of the philosophy of science that undergirds contemporary research in the social sciences is most influenced by Karl R. Popper (1968) and Thomas S. Kuhn (1970). Popper dismissed the type of determinism evident in the statements by Tripp and Hedden. He wrote, "Determinism means that if you have sufficient knowledge of chemistry and physics you can predict what Mozart will write tomorrow. Now that is a ridiculous hypothesis." (Quoted in Horgan, 1992) The issues of predictability and generalizability are what got the positivists in trouble more than 50 years ago. Positivists assert that they can logically infer certain truths about the world through repeated empirical observations. I agree in the sense that if we ran lots of experiments like Steve's and came up with the same results, we would be able to say that print is more effective for learning than audio PROVIDED THE CONDITIONS WERE THE SAME. However, as others have noted during this discussion (e.g., Walt Wager and Cathy Gunn), it is these conditions that make all the difference.

What can we say to practitioners based upon the results of research like Steve's? Print is better than audio!?? Under what circumstances? For which tasks? And how does this "law" (print > audio) interact with individual differences such as intelligence, metacognitive ability, field dependence/independence, locus of control, culture, sex, age, visual and aural acuity, learning styles, analytic/relational skills, anxiety, tolerance for ambiguity, motivation, risk taking, prior experience, interests, attitudes, and disabilities? And what about our objectives? Are we teaching facts, skills, rules, schema, mental models, problem-solving, or creativity?

Chaos theory (cf., Gleick , 1987) indicates that even relatively simple physical systems such as weather are impossible to "determine" because of the complexity of the variables involved and the sensitivity of the interactions of these variables to small differences in initial conditions. If this is true of weather, can we really imagine (as positivists do) that we can ultimately determine which media are best for learning when the phenomenon of interest is as complex as human learning? We have conducted decades of media experiments, and yet we have a pitifully small foundation of theory upon which to help practitioners desperate for guidance in the design of interactive multimedia, electronic performance support systems, etc. The sorry nature of this foundation is obvious in opinions like the one in Steve's original essay that Clark's "media are mere vehicles" statement is "arguably the most famous theoretical dictum in the field of instructional technology." Geez? (This kind of opinion must drive researchers like Dave Merrill crazy.)

With respect to my own philosophy of science, I agree with Kuhn (1970) who suggested that the social sciences (including sociology and psychology explicitly, and education by implication) are "pre-paradigmatic" in that researchers in these fields have not been able to reach consensus concerning the fundamental theories, methods, and assumptions for conducting inquiry. (By the way, even dealing with the physical and biological sciences, Kuhn rejected the notion that as science progresses, we move closer and closer to "truth." He wrote: "Does it really help to imagine that there is one full, objective, true account of nature and that the proper measure of scientific achievement is the extent to which it brings us closer to that ultimate goal?") Kuhn maintained that instead of ultimate truth, the goal of science is simply to change our perspective of the world.

Now there is a laudable goal for research--changing our perspective of the world. To do so, I believe you must situate your research "in the world," not some artificially controlled environment. The empirical methods of positivists like Steve work well within the carefully controlled contexts of their experiments, but their results do little to help us change anything worth changing.

Finally, as Ian Hart noted, I did say "embrace complexity" (Reeves, 1991), and I'll keep on saying it in the hope that a few people will hear it. Does anyone know if the latest paleontological research indicates that dinosaurs had keen hearing? If this is "true," there may even be hope even for them!

References