I would like to add my two cents to this discussion regarding pseudoscience and social relevance. There is much in Tom's paper to provide food for thought for researchers in instructional technology; in this mailing, I would like to discuss one train of thought which arises from the paper. Principally, I would like to suggest that these two issues--quality of research issue (pseudoscience issue) and social relevance issue--might usefully be considered as separate issues. I know that they overlap, but they are parts which are separate. How about asking ourselves the following questions:
Tom concludes in his paper that a high proportion of articles published in Educational Technology Research & Development and Journal of Computer-Based Instruction (about 60%--70% over a five year period) can be regarded as pseudoscience. Question: Would these studies be more socially relevant if they had been performed in a more rigorous manner (e.g., to the level of the 30% not described as pseudoscience)? Would these studies have a greater effect on society even if, by the criteria Tom has used, they could be categorized as "proper" science? My answer is that it is certainly debatable how much effect on our society (and educational practice, in particular) such improved studies within the empirical-quantitative paradigm would have. I cannot presume to speak for Tom, but my feeling is that he's also doubtful: "Although we may eventually be able to conduct valid, socially responsible analytic studies in instructional technology, that time has not arrived" (page 10). I would be interested to read what other readers have to say on this point--to me, these questions are the kinds of questions that we should discussing as individuals and as a discipline.
This leads to my next point: If any of us have doubts about the social relevance of any particular research paradigm or research program that we are in or are considering pursuing, then we have some very important personal decisions to make about what kinds of research we do. If we genuinely feel that a particular research program is of value, then let's go right ahead and do it! If any of us wants to conduct analytic studies, interpretivist, theoretical, or any other kinds of studies, I say go for it irrespective of what other people might say about our work! I, for one, would never consider telling anyone that they should not/cannot do a particular type of research, but I am quite happy to tell anyone, if asked, what I think might be the value of such research. (I am not suggesting that Tom is doing the former.) More of this later.
Now, we cannot of course do our research (or at least publish our research) in a vacuum--if our research is not considered to be the kind of research considered important (some would say fashionable) at a particular time, we will not get published. So, at any particular time, a discipline (and in particular, journal editors and members of editorial review boards) DO make value judgments on the appropriateness of manuscripts submitted in terms of research paradigms. I am not on the editorial review board of any journal, but if I was, I would be quite happy to say whether or not I think a particular research was of use or not and using that as one criterion to vote for/against publication. I do not see any contradiction between this and honoring another researcher's freedom to let him/her decide what they would like to do. So, it would appear to me, as a relative newcomer to this profession, that when one looks at manuscripts that appear in the journals Tom mentioned one can conclude one of (1) reviewers decided the research is socially relevant and paradigm is appropriate or (2) reviewers did not question the social relevance, but found the study to be reasonably well done within the paradigm within which it was framed. Which is true? I honestly do not know. I would like to hear from people who have been involved in such review processes to respond on the list.
So, the point of all this? Discussions of the social relevance of our research activities in instructional technology are to be welcomed. I certainly think it is a very fundamental responsibility for us to think deeply about (1) the goals of our research programs and (2) how successful are the paradigms we are using in achieving these goals. The terms "social relevance" and "socially responsible research" can be interpreted by each of us in our way, but in whatever way we see these terms, we can then ask ourselves (1) am I using the most appropriate paradigm(s) and (2), more broadly, what paradigm(s) do I think are most useful what goals.