I'm becoming a little concerned at the development of the pseudoscience debate as it appears to be in some danger of verging into the realms of an academic flame war. Words are the most powerful weapons academic have and once I see "cranks," "crackpots," and "Josef Mengele" mentioned--no matter what the context--I must urge us to call "Whooa!"
Let me try to step back a little here and work out where we are and how we got here:
Steve Tripp (and he's not alone in this) sees merit in the media attributes proposition. Lots of educational technologists do--after all their very budgets depend on it!--and "common sense" tells us that print is different to sound. Steve is also concerned by the oversimplifications which have stemmed from the 14 year old "trucks and nutrition" metaphor. For the sake of debate on ITForum he sought to refute Clark's criticisms of 50 years of research in media comparisons by setting up an experiment that answers what he perceives to be the main problem (.e., that the uncontrolled variables can be controlled if one is rigorous enough). He set up a hypothesis and tested it by having students do the same task in the same environment but varied the means of delivery. (This is a gross simplification, but I'm only trying to make a point here). If Steve is guilty of anything it is his dogged defense of the validity of his conclusions in the face of some very strong arguments. But that's what we ask of people who put up papers to ITForum (I suspect Lloyd has a bottom drawer full of agreements written in blood).
Tom Reeves took Steve to task, arguing that his methods had the same shortcomings that Clark (and others) had pointed out and furthermore that his experimental design and his defense of it is in danger of falling within Tom's well-argued definitions of "pseudoscience." This term is highly emotive and Tom admits he only used it "in hopes of getting the attention of the educational technology research community." He certainly did that! But calling something "pseudoscience" in a public debate is about as grave a criticism as anyone can make of a "scientist" and it is debatable that it should be used at all in relation to people who do not have a deliberate intention to deceive. Lloyd's list of religious fundamentalists and nostrum sellers is appropriate, but I don't think Steve's work needs to be put in the same category, no matter what its methodological shortcomings.
In an effort to shift the ground (and cool the passions), I and several others have argued that the positivist models of research put forward by Steve and criticized by Tom are of little interest or relevance to our concerns as educators. We side with people like Bob Kozma and Dave Jonassen who believe that the entire debate needs to be reframed from "What is the most effective medium to learn from?" to "How can students learn effectively with media?" We believe that it is not possible adequately to address the question using hypothesis-based, experimental methodology and we veer towards more ethnographic, hypothesis-generating approaches which seek insights rather than (Steve's unfortunate use of all-caps) TRUTH.
Like Tom I am very concerned with anything that poses as "science" or "research" that doesn't pass appropriate tests of rigor (though not necessarily the scientific rigor of Newton, Darwin, and Farraday). Qualitative research methods are most vulnerable to this criticism and phrases such as "soft science" even "Mickey Mouse science" are bandied about by the numbers men & women. Action research is a case in point: in my experience it is too often an excuse for poorly-designed and conducted research projects in Education. "It's just action research..."
You see, action research is not research in the sense that Steve uses the term. Action research is a process which uses the methods and rigor of research to achieve change.
ACTION RESEARCH
It's interesting to imagine how action research might be used to tackle the type of question posed by Steve.
1. First of all, the problem would need to have arisen naturally (e.g., there has been a curriculum based solely upon reading and some teachers believe that this is disadvantageous to some students and audio would be a better way to provide information--shades of the Australian foreign language curriculum 1955-1985).
BY THE WAY: I'll bet a steamer of cha siu bao that Steve's experiment did not arise as an abstract proposition--Steve is a language teacher (I guess) and he has a special interest in presenting information in written versus audio form. ("Socially responsible" in Tom's definition)
2. The advocates of change present evidence from the literature (any literature, e.g., brain research) to make the case that it is worth trying. They probably also present evidence that there are problems with the current methodology (e.g., the failure rate). Some students may be being disadvantaged.
3. A group of faculty agrees to try the audio method. This is important--action research requires commitment (not objectivity). Carefully designed strategies for change are agreed together with forms of continuous evaluation: controlled observation, teacher diaries, interviews, assessments, etc. There may be a need for some teacher in-service retraining. Regular meetings are held with all participating teachers where information is shared.
4. Because this is formative evaluation, the process is cyclic. Let's assume that evidence arises that the way Tom (the French teacher) combines audio with colorful wall charts appears to be more intrinsically motivating than Ian's method of running the tape recorder for the whole lesson. Evidence from interviews and observations might be collected to show that this was, in fact, happening and then Ian might try out Tom's method and compare the effects on his class. They debate it at one of the meetings, etc., etc.
5. When does this cyclic process of act-observe-reflect come to an end? Maybe it doesn't:--after all Mao believed in continuous revolution. More likely it's like the Japanese game of "go" where players agree that it's over. The new procedure is agreed among the faculty and the results are written up--probably not in a learned journal, more likely as a "poster" exhibition.
In action research, therefore, the conclusions are not as important as the process. Research is a tool for change, and its rigor (the careful observation, data collection, indexing, analysis, and debate) is what keeps the process of change on track.
Now if I claimed that my action research project proved that the audio method was better than the reading approach so therefore everyone should do it, I guess I'd be guilty of pseudoscience.
But I wouldn't. What I would say is that the experience of carrying out this research has made me a better teacher, more aware of my students' needs and the possibilities of the media, more conscious of the value of careful observation and documentation.
As Professor Fred Dagg used to say: OK, I be letting you get back to work now...
Tune in next week for water divining.