Now that Steve Tripp's discussion is "officially" over, I'd like to follow up on one of the tangents we touched on toward the end of the week--specifically, the concepts of "socially responsible research" and pseudoscience. I like Tom Reeves' idea of promoting socially responsible research and I vote in favor of it. He presented some wonderful suggestions for any would-be researcher. I also support his ideas of the value of doing developmental/action research. Some of my most important research, that with my son who is mentally retarded (selfish though it may be), falls into this category. However, I'm left with some troubling thoughts on how to correctly judge research to be socially responsible and very concerned about inaccurately (and unjustly) deeming some research not to be so. It seems that if we are not careful too much good research will be inappropriately maligned. I also admit that I continue to be confused by his descriptive list of what research should be classified as "pseudoscience." For example, clear examples of pseudoscience to me are creationism, astrology, phrenology, etc. (note that this is not meant in any way as an insult to one's religious faith). I just can't seem to use Tom's list to figure out which examples of educational research clearly belong in that camp (good research vs. poor research is a different list I'm assuming). Most important of all, I'm concerned that too many people (graduate students?) will ultimately decide that there are just too many personal risks associated with trying to do serious research, so why bother?
By the way, I know that Tom is just trying to take educational research seriously and to assure that good research prevails. Yet, I think some clarifications are needed here for folks who don't work on the same floor as Tom!
I'll refer to my own work as a starting point. I do my research both in the field (elementary classrooms) and in the "laboratory" (computer labs at the university and in elementary schools). I feel that the laboratory is the best place to study research questions that focus on the psychological framework of learning within computer environments (I use classical physics for the content). The "unit of analysis" is a single person and his/her learning. To take my work with adults as an example, I admit that I rely on undergraduates to participate voluntarily in my laboratory research. It is heavily quantitative (but not exclusively so). The learning experience usually only lasts about an hour for the participants. Tom's pseudoscience list really seems to target research of this type.
The criticism, of course, is that this is not an authentic situation for these participants. Few are expected to have any genuine intrinsic interest in the content. The majority actually sign up for my research with the carrot of getting some extra course credit and also because of the promise that the experience will demonstrate a use of simulations in education--as preservice teachers, this is relevant to them. Despite being open to the criticism of inauthenticity, I constantly remind myself (and others) that most high school students also find little relevance and authenticity in the things they are learning in subjects such as physics. (By the way, I'm trying to build a liaison with folks in our physics department--I haven't had much luck yet.) The limited exposure is not an issue given the purpose of the research--psychologically, a lot of interesting learning goes on in one hour. By the way, you can imagine my satisfaction when many of the undergraduate participants, uninterested with physics at the start of the experiment, become intrigued with physics as a result of their research experience. I get lots of comments like "Well, if I had these kinds of experiences in high school, I would have liked physics more!" I also get lots of requests from these participants for copies of my simple physics software.
I feel this laboratory work is important because it informs me as to the psychological factors underlying learning within these sorts of learning environments. It influences my field work and my teaching about the design of these sorts of environments (i.e. games/simulations). I also hope it informs others through my published reports of this work. Yes, I am open to legitimate criticism on several fronts, but so is all research. I do my best to recognize the flaws and discuss them openly in my publications. Hopefully, other researchers who see value in my work will take up the challenge of designing research that corrects these flaws in ways that I was not able. So, I accept the fact that some people will value one part of my research efforts over the others. But, I feel the way my work in one area informs my work in the other is very important and I will continue to follow this pattern of research (e.g. in my field work, I use a different "unit of analysis"--groups of students working collaboratively). I also feel strongly that none of my work is pseudoscience, yet using Tom's list doesn't make this clear.
As another example, one researcher whose work and ideas I've long admired is Richard Mayer. Yet, if we again use Tom's guidelines, it seems that one could easily judge his work to be social irresponsible and pseudoscience as well. Frankly, lots of examples of important research begin taking on the pseudoscience aura, such as all the pioneering work of John Bransford, Alan Lesgold, Robert Glaser, etc. in cognitive psychology, because of its strong laboratory emphasis. (By the way, I was an undergraduate at the University of Pittsburgh in the l970s and participated in many studies sponsored by the Learning Research and Development Center--research that I dare say has had a great influence on educational psychology and instructional technology. It was also socially responsible in my view as an undergraduate--it gave me lots of lunch money!)
I guess my point is that a lot of research can quickly be viewed as inconsequential when a single study is isolated and dissected. Yet, when viewed in the context of an established research agenda and the published literature it becomes incredibly important and very socially responsible because of the way it informs the professional community--even if it is heavily quantitative laboratory stuff! That's why publishing one's work and holding it up to scrutiny is so important. I'll repeat that my greatest fear is that we might convince too many people to give up on the idea of pursuing research as an integral part of their careers before they even begin. (An "established research agenda" begins with the first study!) Let's face it, research in the social sciences is difficult, complex, and imperfect. We need as many creative people as possible willing to join in. I also believe that research is very much like other complex processes (e.g., writing, instructional design) in that you understand it by doing it.
Frankly, when it comes to defining pseudoscience, I'd prefer that we stick to the following list put forward by John Casti in his book Paradigms Lost (a work that I know Tom favors as well):
With these, I can more easily judge creationism as pseudoscience and my work (and Richard Mayer's) as social science (I'll let others more qualified than I tell us if social science is a "real" science). I don't think any of this counters Tom's basic premises, but hopefully clarifies them further.
Casti, J.L. (1989). Paradigms lost: Images of man in the mirror of science. New York: Morrow.