[quoting Rieber, 24 Mar 97] I do both field work and laboratory work and don't apologize for either. The postings I have read during this discussion so far seem to be advocating abandoning laboratory research outright (regardless of its quality). I think that would be a mistake. Am I the only one?
I'd certainly not be an advocate of banning it outright, but in my experience laboratory-style research is seen as "easier" than field research (none of those messy variables) and faster (something you can accomplish in a term). If the imperative is time, and I guess it is in course-based research projects, then laboratory research is going to look quite attractive. My contention is that if it is all you do, then the results may not bear scrutiny in the light of day. Historically speaking, this has often been the case. (I recall a series of laboratory experiments in the 1960s which demonstrated that the use of color was not important in instructional graphics or instructional television. As a producer of educational media--even then--I failed to see the application of this research to my work.)
[quoting Wager, 24 Mar 97] Unfortunately most graduate students don't engage in research early enough in their programs to know what they want to do for their dissertation. It is by doing research that you learn what a good question is, and what confounding variables are. Then, by the time they get to the dissertation, they are in a hurry, and are victim to poor logistics (even if they do have good questions).
Which is saying much the same thing. If the motivation for doing research is only the course requirement, than the likelihood is that the question will be trivial, the methodology will be flawed, and the conclusions will be irrelevant.
But back to Lloyd's project, which I find fascinating and an excellent example of how I think research should be conducted in our field. He writes about a project in which he and a doctoral student are engaged:
[quoting Rieber, 24 Mar 97] David and I are clearly proponents of gaming. However, our results clearly indicated a negative consequence of integrating game elements to simulations (even though participants really like the game, they got so wrapped up in it that they forgot they were supposed to try to learn something from it!) Therefore, we were forced to confront our own biases favoring game design and think more deeply about our views. We also needed both methodologies to come to our conclusions. If we had only used qualitative methods, I fear our biases would have gotten the better of us and we would have concluded that games are great. Fortunately, the quantitative results clearly showed that the game interfered with explicit learning--we could not "get around" that. Instead, we used the qualitative sessions to help us understand why this was so (although we had predicted, based on previous research, that the game had the potential for distraction).
So Lloyd is proposing a solution to many of the issues that have come up here:
(a) The supervisor is clearly involved in the project with his student. This is the model favored in the sciences, where graduate students all contribute to a larger project under the control of the supervisor. By following this model, it is guaranteed that the project is not trivial and that the work is properly supervised.
(b) The research is not hypothesis-driven--the hypothesis is being developed over the course of the project. And the research team's biases are clearly recognized and taken into account.
(c) The research includes a variety of tools: both quantitative and qualitative. In this case the "laboratory" phase is obviously appropriate (although I'm not sure that Lloyd's and my definitions of "laboratory research" coincide).
Walter concludes by asking:
[quoting Wager, 24 Mar 97] Maybe things are better at other universities--I'd like to know.
They're certainly not better here, but in recent times university departments in Hong Kong have been required to nominate "designated researchers" as part of the funding exercise. Departments with more designated researchers get more money and can take more graduate students. To be a designated researcher means that you have attracted external research funding and have international respectability as a researcher. The assumption is that your research students work with you and learn from you. People who are not designated researchers can still take on graduate students, but it costs the department money to allow this.
This is not a new model--in fact it's the oldest model there is: it's known as apprenticeship. And it is exactly what Lloyd was describing. Speaking as an outsider, with little knowledge of U.S. universities, apart from what I read in journals and on forums, I have the impression that this model is not as accepted as it ought to be--that it is increasingly common for students to undertake independent research with minimal or inadequate supervision.
I'd be interested to know if you think this is a problem.