21 Oct 94
Doris Shaw

If we are educators, and particularly educational researchers, we should care whether students are mentally engaged. The difficulty is measuring the quality of "covert" mental activity. It seems that the consideration of overt and covert responses has not been fully explored.

Yes, there can be interactive reading of a book. My father used to read his favorite snatches at the dinner table. When such interaction takes place, however, evidence can usually be observed. Head scratching, taking down another book from the shelf, conversing with another person, scratching a note or a diagram, even gazing out the window or pacing the room are behaviors that may be overt indicators of high level interactions. Interaction with a computer may be accompanied by observable behaviors--over and above audit trails or judged responses.

We have a tendency to label responses that we have anticipated and plan to measure "overt" and anything else "covert." I maintain that all mental activity results in some kind of overt response, though it may be difficult to observe and may even occur at some time in the future. Short of measuring mental activity with electrodes or computer anxiety from blood samples, we might make good use of ethnographic methodology to discover behavior patterns that indicate the quality of learning transactions--those that associate with successful productivity models.

As far as performance support systems are concerned, their success is very difficult to measure by traditional methods. The motivation for their use must be intrinsic (Malone stuff), since setting up extrinsic motivation such as fantasy, credit for completion, scoring points, or bells and whistles is not part of their design. Who is to know what piece of the system satisfies each student's intrinsic curiosity or job requirement? That again suggests that process/product comparison is the way to measure transactions in the case of performance support systems.

Doris Shaw

Phone: 217-373-6729
Fax: 217-373-6724
E-mail: shaw@bambi.cecer.army.mil