12 Oct 95
Steve Alessi

Since I haven't seen any reactions to Spector & Davidsen's article yet, I'll jump in with some comments, and probably become shark-bait as a result.

I have been enthusiastic about the Systems Dynamics (SD) approach ever since reading Nancy Roberts' (and several other authors) book Introduction to Computer Simulation: A System Dynamics Modeling Approach. I teach SD and Stella in my graduate course on simulation design, and students use Stella to do modeling and then use those models inside other authoring systems to create instructional simulations. Of course, the Roberts' book was really about using modeling as a primary instructional strategy for teaching thinking and problem solving (not for designing CAI simulations) and I have always found their arguments for that to be very convincing (much more so than the arguments for using Logo to teach thinking and problem solving). I especially liked the argument by Roberts and her co-authors that our national (and local) policy makers look for simple answers, trying to change complex systems by manipulating a single variable (e.g., more taxes to "fix" the deficit or more prisons to "fix" crime). Invariable their simple fixes change the complex systems in unintended ways, often with the exact opposite outcome they wanted. If only more of those policy makers had studied Systems Dynamics--well, you get the point.

However, the approach has, sadly, never caught on (as Logo, unfortunately in my opinion, did for a while). There are probably a number of reasons. As Bill Barowy points out in his work, kids have a lot of trouble understanding the concept of modeling. It is very difficult for teachers to teach. It is not in the school curriculum. And, to be honest, I'm not sure we have evidence yet that it works. Mike and Paal referenced Mandinach & Cline (1994), Davidsen (in press), and Morecroft & Sterman (1994) claiming they show "very promising" results. I have only read Mandinach & Cline. Their book gives a very good description of designing and implementing a Systems Dynamics based curriculum in schools, but by their own claim (Chapter 6) they had not yet begun research on effectiveness at the time of that book. I hope their continued research will show positive effects, but I would not reference that book as current evidence of any. Do the other two references (Davidsen or Morecroft & Sterman) do a better job of supporting the claim of very promising results? Since Mike and Paal have read these, I would be eager for them to say a little more about that.

But even if positive results are there, the thing Mike and Paal do not say much about is how very difficult it is to teach Systems Dynamics. All the books and articles I have read (e.g., Roberts, et. al.; Mandinach & Cline; Barowy, et. al.) say this. Students have trouble distinguishing stocks and flows. Every science teacher experiences this problem when he or she sees the difficulty students have distinguishing quantities versus rates of change. My own (graduate) students handle those fairly well, but have lots of trouble with time-delays and with what to include within and outside of the system boundary. Well, I'll dispense with the SD tech talk. Suffice to say, it may be a very worthwhile method of thinking and analyzing things but it's also a very complex method. Many teachers have tried it and given up. Is it wise to suggest such a complex methodology as a basic instructional strategy we should all use in our teaching?

Another difficulty I had while reading the article was whether they were suggesting SD as an instructional design strategy or as a teaching strategy. My difficulty was cleared up by the end, but it was at its worst in the middle of page 6 (pages according to my printout of the longer HTTP version) where they say, "we expect that most readers are more interested in improving the quality of instruction and learning than in bank accounts." Here and elsewhere (including in the title) I had an expectation that the authors were going to use SD as an ID strategy. After all, the economics (bank account) example is just as good if we were teaching a business or home accounting lesson, as is the predator-prey example if we are teaching ecology. I was lead to believe that later in the article Mike and Paal would show causal loop diagrams of the instructional design process, or of the factors affecting student learning in an SD-based learning environment. But it soon became clear that they were using SD as an instructional strategy. I don't have a problem with that. I'm just pointing out that for a while I was led to believe something different was in store.

Something the authors do not deal with much in the article, but which I am most interested in, is the two ends of the "simulation use continuum," both of which are possible with the SD methodology. On one end of the continuum you can use SD to create CAI simulations which students will learn from by using the simulation (perhaps in a directed mode, perhaps in a more discovery or research mode, but that's another continuum). On the other end of the continuum you can have students learn by using SD to do the modeling themselves. Mike and Paal's emphasis seems to be on the latter end of the continuum. But they did discuss how students might begin with a simple and already developed model, study it, modify it, and then move on to the more creative mode of developing new models. How to design that kind of learning sequence (when to use, when to create) is what I find most interesting.

Lest anyone misinterpret me, I am enthusiastic about the approach that Mike and Paal are suggesting. My suggestions to them (for publication purposes) would be to beef up the support for SD effectiveness (if they can find it), and remove the ambiguity about whether they are suggesting SD as a design strategy or an instructional strategy. My suggestions to others interested in simulation design is to pursue the very interesting question of designing along the "simulation use continuum" and what research needs to be done to give designers advice in that regard.

Steve Alessi
370 Lindquist Center
The University of Iowa
Iowa City, IA 52242

Phone: 319/335-5568
Fax: 319/335-5386
E-mail: steve-alessi@uiowa.edu