3 Mar 97.d
Rob Foshay

My basic argument is that:

(1) For products intended for professional use (i.e., by instructors and learners in settings where there are expectations for learning outcomes), software publishers should be required to accurately describe the intended use (or range of uses) of the product, the intended learning outcomes, and evaluation evidence supporting those claims.

(2) For products intended for general use (i.e., purchased by unsophisticated parents for their kids or for themselves), publishers should be held accountable for claims they make of what the software teaches and how well it does so.

(3) For products that don't exist yet (i.e., custom products), clients should at least require that their suppliers be certified under ISO 9000 ASQC/ANSI guideline Z1.11 (see my original paper for reference).

There are already examples of similar practices in other fields. To wit:

• The evaluation community has a pretty well-agreed upon set of technical and ethical standards for the development and use of tests. These had a major influence on the legal language in the U.S. Equal Employment Opportunities Act, and, as a result, there is now a substantial legal history of tests and testing practices which don't meet those standards being indefensible in court.
  
  
• In a highly publicized case, the International Reading Association evaluated the Hooked on Phonics audio tape series, and demanded that the Feds go after the company for its advertising claims that it taught reading. This happened, and the company stopped its advertising (I don't know if they are even still in business).
  
  
• The NEA allows software publishers to put NEA endorsement seals on the packaging and in the advertising of approved products.

And in other, less related fields:

• The American Dentistry Association does the same thing for toothpaste.
  
  
• There are strict ethical and regulatory controls on how drugs are advertised and marketed to health care professionals and to the public.
  
  
• When the U.S. Big Three auto makers decided they would conform to ISO 9000, they required all their manufacturing suppliers to do so too (just as ISO 9000 requires) (I know, I know, their version is called QS 9000). NASA did the same thing. That pretty much caused the majority of manufacturing entities in North American to get on the band wagon. Same thing happened in Europe when the EEC countries announced that government contracts would include a requirement for ISO 9000 certification from vendors.
  
  
• When Archer-Daniels-Midland (ADM) decided the demand for healthy foods would improve demand for their products, they created the Healthy Choice brand. The interesting thing is that the brand consists of nothing but a set of standards for nutritional composition, packaging, labeling, and advertising. I don't know if ADM actually makes and markets any of the products sold under that brand. I believe they don't make a number of them.

My point is that these are all examples of ways of approaching the "brand integrity" issue. Some are regulatory, some are market-based. Some work better than others. Some work in professional markets, and some work in general consumer markets. Some required considerable financial resources for the instigator to establish, and some are profit centers for the instigator.

What would it take for ID? Depends on the approach. For school software, if the largest-spending dozen school districts got together and announced that all future RFP's and purchasing contracts involving educational software would require that qualified suppliers be certified under Z1.11, I guarantee the whole industry would be toeing the line within 18 months, and it wouldn't cost the schools a dime (except possibly by removing from the market some of the cheapest, lowest-quality software).

You raised another point: textbook publishers don't go through all this, so it's not reasonable for education/training software publishers to. With this I disagree. The reason is that TBT makes claims that people learn from it; textbook publishers don't make such claims. That's either hubris on our part, or TBT really isn't "electronic publishing." (I would argue that failure to understand this difference is one of the reasons almost all TBT companies purchased or started by textbook publishers over the past 20 years have failed.)