[quoting Wager, 21 Mar 97.b] However, I am very suspicious of research like this: A study of how students use resources in a collaborative learning environment on the web, or phrased as a question: "How do students use resources in a collaborative learning environment on the web?" Even with detailed (read tedious) description of the learning environment, the types of learning activities, and transcriptions of individual verbal interactions, my first question is: "What, as a professional, can I learn from your study that might have some utility for future design decisions, or theory building?"
After thinking this over, I can't see a problem with the research question Walt refers to. My problem with the scenario is the description of the types of data collected/reported. Personally, I would love to read a case study that utilized an emergent design by a researcher whose goal was true engagement of the learners rather than polite responses to what the learners thought the researcher might want to hear. In my opinion, for this topic, in which there is so little quality research, the questions should come from the learners/participants rather than from the researcher.
I think the problem with the scenario Walt presents comes from what one of my committee members called "straddling the fence." The methodology is positivistic but the question is objectivist. I believe that research that seeks to form a foundation for future research doesn't start with a comprehensive literature review; the literature review comes in response to the data as it is collected. The open-ended questions used to initiate the research become more specific as the research progresses. The methodology isn't set in stone because the purpose of the research isn't to find "scientific truth" but to identify the "claims, issues, and concerns" (see Guba & Lincoln, 1989) of one group of participants. And, those "claims, issues, and concerns" may be different for another group of people.
Why must one single research study contribute to the knowledge base? To me this desire comes from the tradition of using large enough samples that we can statistically generalize to the entire population. That kind of research isn't possible without a solid understanding of the types of variables that might be encountered and the possible controls that could be applied. But first you have to identify the variables and the controls. Isn't that where we are right now as a field--in the process of identifying those factors that influence the impact/affect of technology upon learning?
But trying to accommodate an emergent research design into a prospectus format that must be approved by people whose philosophical framework is grounded in the scientific method is an almost impossible challenge. I just threw out my dissertation research because the questions I was trying to investigate had no meaning to my students, they had different issues altogether so that their polite responses didn't triangulate with the work they produced or their reflective reports. Nothing in the literature prepared me for what I found in practice. If I hadn't been working from a pre-approved prospectus and human subjects review form that had to be rigidly adhered to, I could have shifted the focus of the research just slightly and ended up with what I think could have been a good study that identified some previously unreported variables. If I had more experience as a researcher, I could have possibly made these adjustments within the proposed methodology. But dissertations are formative review. Knowing about isn't the same as being able to do. My advice to doctoral students is to be proactive in finding research projects to work on before you start working on your dissertation. It's back to the drawing board for me.