(introductory text...)

William B. Rouse

Abstract

The design of information systems is considered in terms of the viability, acceptability, and validity of the information support provided. These issues are discussed in the context of several examples, including systems for bibliographic information retrieval, aircraft operations, maintenance information, sales transaction support, and design information. A variety of "lessons learned" that illustrate the impact of adopting a human-centred approach to designing information systems is summarized.

1. Introduction

Information is an essential ingredient in much that we do. We spend much time gathering, refining, and interpreting information. This process of digesting information has become increasingly complex as the store of information has become larger and more diverse.

The size of the information store makes it difficult to consume all relevant information. The increasing diversity of information sources, forms, and languages makes it difficult to identify and interpret all relevant information. Often indigestion and sometimes "information poisoning" result.

Information technology appears to provide the means whereby these problems can be overcome. Hypermedia, multimedia, natural language processing, expert systems, and CD-ROM are notable examples. Numerous commentators over the past 40 years projected that technologies such as these would soon help us to deal with the information explosion. Each of these commentators has, at best, been a bit too optimistic.

One could argue that the proclamations of success were premature because the cost and/or power of computer technology, as well as related technologies, did not evolve as quickly as anticipated. While this may be true, I believe that more subtle problems have hindered progress. Put simply, enabling technologies such as those noted above may be necessary, but they are not sufficient for success.

To support this assertion, consider the following two examples. In the late 1970s and early 1980s, we undertook an effort to put hard-copy procedural information on-line. It seemed intuitively obvious that problems associated with the growing size and number of technical manuals could be lessened, or perhaps eliminated by moving to computer-based information systems.

Two studies were performed in the context of aircraft operations manuals [5, 6]. Of particular interest here is the first study, where one condition involved putting on the computer display the exact same information, in the same format, as in the hard-copy manuals. Experimental results for this condition indicated the computer-based system was substantially inferior to the hard-copy presentation.

The problems appeared to be due to the inherently limited screen size and the distinct possibility of getting lost in the display hierarchy. Fortunately, means were devised for alleviating these problems and a derivative of the display system discussed by Rouse, Rouse, and Hammer [6] is being used in the Boeing 777 aircraft. Nevertheless, the "lesson learned" is clear- simply putting information on a computer does not necessarily make it more useful than presenting it in more traditional ways.

The second example concerns an effort in the mid-1980s to develop intelligent bibliographic information retrieval systems, primarily for use by engineers and scientists. Five studies were performed to understand how various computer aiding schemes affected users' abilities to retrieve information of value. The results of this series of studies are reviewed by Morehead and Rouse [3].

Three of the studies considered the impact of providing links among articles based on reference lists. As we expected, such links helped considerably. This led us to add further links based on citations of articles. In this way, an article was linked to both its ancestors and its descendants.

Much to our surprise, the citation links substantially degraded users' performance. Users tended to wander down citation paths long after they ceased to be productive. We modified the system to display the productivity metric of articles selected divided by articles viewed. In this way, the decrease in productivity of citation paths became evident to users and they abandoned citation paths much sooner. The "lesson learned" here is also clear - providing more links among information elements is not necessarily beneficial and may be detrimental.

Thus, the intuitively obvious benefits of enabling information technologies are not always realized. The straightforward reason is that intuition is not always right, as numerous lottery customers will attest. Rather than betting on technologies, users would be much better served if we first focused on the benefits sought, and then considered alternative means of providing these benefits.

2. Human-centred design

The types of problems noted earlier can be avoided, and the potential of enabling information technologies can be realized, by adopting a human-centred approach to designing information systems. Human-centred design is a process of assuring that the concerns, values, and perceptions of all stakeholders in a design effort are considered and balanced [X].

Stakeholders include users, customers, maintainers, investors, and so on. Further, the designers of information systems are stakeholders in these systems. While this paper necessarily focuses on users, were we to discuss the design, development, implementation, and servicing of an actual information system, we would consider all of the stakeholders.

Human-centred design can be viewed as a process for addressing and resolving the seven issues listed in figure 1. Four of these issues (i.e., evaluation, demonstration, verification, and testing) are well known to designers of information systems and are usually addressed in a reasonable manner. These four issues are not discussed within the confines of this paper. Interested readers will find a comprehensive treatment of these issues in Rouse [8].

The top three issues in the figure (i.e., viability, acceptability, and validity) are seldom addressed with sufficient rigour by designers of information systems. Human-centred design involves pursuing all of the issues in figure 1, starting at the top. Thus, the first question asked is "What matters?" while the last question asked is "Does it run?"

Rouse [8] discusses a four-phase methodology, as well as associated methods and tools, for pursuing the seven issues in figure 1. In this paper, discussion focuses on elaborating the nature of viability, acceptability, and validity. The use of these constructs is subsequently illustrated in the context of a few applications.

Viability is concerned with benefits and costs. Contrary to the apparent beliefs of many designers of information systems, the primary benefits to users seldom include having the opportunity to use an information system. Users typically use an information system to make better-informed decisions, solve problems, order products and services, save time, and so on.

Costs may include access charges; however, such costs are often paid by third parties. For most users, costs include the difficulty and time involved in learning to use and in using the system, as well as the difficulty and time associated with using the outputs of the system. Thus, for example, one of the costs of using conventional computer-based information retrieval systems is the difficulty and time of wading through the hundreds or thousands of abstracts obtained, as well as locating and obtaining source documents.

Viability® Are the Benefits of System Use Sufficiently Greater than its Costs?

Acceptability® Do Organizations/lndividuals Use the System?

Validity ® Does the System Solve the Problem?

Evaluation ® Does the System Meet Requirements?

Demonstration ® How Do Observers React to System?

Verification ® Is the System Put Together as Planned?

Testing ® Does the System Run, Compute, Etc.?

Figure 1 Human-centred design issues

Acceptability concerns the extent to which a way of doing things fits in with individual and organizational preferences and constraints. For instance, the hardware and software of an information system should be compatible with other hardware and software employed by users and their organizations. A more subtle need is for usage procedures for the information system to be compatible with usage procedures for other systems used by the same set of users. An example of preference-related acceptability concerns would-be users' desires for colourgraphic displays despite the fact that monochromatic alphanumeric displays would be less expensive and provide a valid means to meeting information needs.

Validity focuses on whether or not an information system solves the users' information-seeking problems. It is quite possible for a system to meet requirements - that is, pass evaluation with flying colours - but not provide valid support. For example, an information system might rapidly retrieve and display masses of information, much of which is irrelevant, the remainder of which is only marginally understandable by the class of users for which the system was designed. While one could blame this on the quality of the databases and argue that the information system satisfies its technical requirements, it is nevertheless a fact that the system does not provide a valid solution to users' problems. One might attempt to resolve this problem by adding artificially intelligent functionality that reads and translates all of the information retrieved to assure that what users get is relevant and understandable. This would not necessarily lessen validity problems if users were skeptical of the computer's ability to perform such filtering and translation.

Note that the discussions of human-centred design in this section have only paid passing attention to display formats? dialogue structures, and so on. While these issues are important, they are nor synonymous with the user-system interface within the human-centred design framework. Within this framework, the interface is "deeper" than the displays and keyboard. The interface includes all functionality whose goals are to enhance human abilities, overcome human limitations, and foster user acceptance [8].

Therefore, within human-centred design, one does not design an information system and then "add" a user-system interface. Instead, one begins with the user in terms of benefits, costs, etc., and progressively deepens the design. At some point, one translates the means to providing benefits into particular enabling technologies. Typically, the design of displays and input devices naturally evolves in this progression. In this way, human-centred design not only results in systems that are usable- it also produces systems that are useful.

3. Applications

In this section, three example applications are discussed: (1) maintenance information systems; (2) sales transaction systems; and (3) design information systems. The purpose of these illustrations is to show how human-centred design influences the nature of the products and systems that result.

3.1 Maintenance Information Systems

The application concerned the problem of transforming large, blueprint-size hard copy, often called C size, to small, computer-display-size images [2]. The context of interest was helicopter maintenance.

A very important element of human-centred design is initial emphasis on defining the true nature of the problem to be solved. From the point of view of the humans involved in this context, the problem of interest was helicopter maintenance, not reading blueprints. Thus, in terms of validity the primary concern was providing information to support maintenance activities rather than finding a way to access blueprints on a small display.

This realization led us to focus on the tasks to be done rather than on the nature of blueprints. It became clear that information is used in different ways depending on the nature of the task, i.e., problem solving vs. procedure execution. This conclusion led us to adopt Rasmussen's abstraction-aggregation hierarchy [4] as a means of organizing maintenance information.

The abstraction dimension included physical form, physical function, and generalized function depicted in terms of location diagrams, schematics, and block diagrams, respectively. The aggregation dimension included assembly, subsystem, and system-level representations. As a consequence of this approach to organizing information, it was no longer necessary to have large displays.

This system concept was evaluated in a series of five experiments. It was determined that the nature of the displays affected maintainers' activities. They performed at least as well using the new displays and overwhelmingly preferred the new displays. Further, it was determined that creation and updating of the display database would be easier with the new approach. Thus, both acceptability and viability were improved.

3.2 Sales Transaction Support

Computer-mediated sales are an increasingly prevalent approach to selling in retail stores, banks, airlines, and many other domains. Perhaps not surprisingly, there has been considerable interest in improving the user-system interface of such systems. Of particular concern, because of the high turnover among people performing such jobs, has been decreasing or possibly eliminating the need for any extensive training in the use of these systems.

We undertook two efforts in this area, one in the domain of retail sales and the other in passenger reservation systems. In both cases, we were asked to improve the usability of these systems by focusing on the user-system interface. We employed the human-centred design methodology to pursue these efforts.

In both cases, we focused initially on viability, acceptability, and validity for a period of 4-6 weeks. We discovered that usability problems, while important, were by no means the predominant concern. The benefit sought in both cases was increased sales and the cost was the time required to make sales.

It would have been quite possible to solve usability problems without enhancing viability - increasing benefits and/or decreasing costs. Focusing solely on usability would have probably increased individual user acceptance but not necessarily organizational acceptance. Finally, solving usability problems alone might have met requirements, but would not have been a valid solution to the right problem.

For both efforts, the initial focus on viability, acceptability, and validity led to an emphasis on sales support rather than solely on improved operability of computer terminals. While usability and the user-system interface still received much attention, it was given in the context of supporting the tasks that really mattered. The result was system designs that were substantially different from those originally envisioned.

3.3 Design Information Systems

The application under design information systems focused on access to and utilization of science and technology information in the context of designing aerospace systems. The motivation for this effort included a long-term interest in the value of information [7, 11], as well as a practical need to develop design information systems.

In keeping with the human-centred approach to design, we began by focusing on viability, acceptability, and validity. These issues were pursued using questionnaires, interviews, and observational techniques involving a large number of designers [9]. We found that very little science and technology information is accessed by formal means.

Why don't designers take advantage of science and technology information? One answer is that they perceive little benefit and great cost in accessing this type of information. They attach no benefit to using the information system per se.

They are concerned with making informed design decisions. They become informed by asking other people in their organization, a conclusion also reached by Allen [1]. Why do they rely on subjective opinions rather than the "hard" objective information provided by science and technology? A primary reason is that they find published research results to be applicable in general but not to their specific problems in particular. They want contextually based answers to their questions rather than generic simplifications. In other words, they question the validity of available science and technology information.

There are also acceptability problems. Almost all science and technology information is created, written, and published for consumption by scientists and technologists. Designers seldom have the specialized expertise, or the patience, to penetrate this information. They find the context and format of presentation totally unacceptable.

The essense of the designer's dilemma is depicted in figure 2. Each transformation in this diagram requires time and effort. Both time and effort increase as one moves to the right in this diagram. It is easy to see why a designer would not want 1,000 abstracts of research articles on human memory to answer a question concerning usability of radar modes. The cost of answering questions in this way far outweighs the benefits.

The above conclusions concerning designers' perceptions of viability, acceptability, and validity caused us to focus on designers' tasks and information needs rather than on the nature of science and technology information. Thus, rather than focusing on how to get designers to access and utilize science and technology information, we looked at the information requirements to support design decision-making. Such requirements should drive the way in which science and technology information is created, organized, formatted, and accessed if this information is intended to support design.

Figure 2 Alternative transformations in answering questions

After considering alternative representations, we concluded that information seeking in design could be represented as a process of asking questions and pursuing answers in the context of a "design space" including a set of archetypical tasks focused on attributes of the design artifact and characterized in terms of abstraction and aggregation [10]. Typical scenarios or trajectories in the design space were studied to determine information requirements in particular and support requirements in general. Using a structured analysis and design methodology [8] led to identification of hundreds of requirements and an appropriate conceptual architecture that would satisfy these requirements.

4. Lessons learned

The human-centred design issues of viability, acceptability, and validity have been discussed in the context of several examples, including systems for bibliographic information retrieval, aircraft operations, maintenance information, sales transaction support, and design information. In this section, the lessons learned from these efforts are summarized.

First and foremost, it is essential to recognize that information access and utilization are seldom ends in themselves. The benefit sought is successful task performance, not information seeking. Thus, primary tasks of interest do not include operating an information system.

Regarding primary tasks, the information requirements associated with these tasks would dictate information system design. The existing organization and format of information should not, to the extent possible, constrain the nature of an information system. It should also be noted that the ways in which information requirements are satisfied are likely to vary with tasks, despite the fact that the same information content may be required for two or more tasks.

Simply putting information on a computer display is not necessarily better, and may be worse, than using other media, unless appropriate aiding is provided to enable using the information in new ways. Similarly, additional information is not necessarily better, and may be worse, without appropriate aiding to enable using the new information.

People tend to interpret the validity of information in a very context-specific manner relative to their needs at the moment. People are also more likely to find information acceptable if its format and content make it easy to understand and interpret.

Finally, by focusing on the issues of viability, acceptability, and validity within the human-centred design framework, one is much more likely to solve the right problem and solve it in an acceptable way. The result is information systems that are both usable and useful.

5. Conclusions

The information explosion continues unabated. The promise of information technology has long been heralded as a means of containing, and perhaps counteracting, this explosion. This paper has argued and illustrated with many examples that the problem is not amenable to technology panaceas. Instead, success is much more likely if the concepts, principles, methods, and tools of human-centred design are used to determine contextually relevant information requirements, as well as synthesize support systems that satisfy these requirements.

References

1. Allen, T.J. (1977). Managing the Flow of Technology. Cambridge, Mass.: MIT Press.

2. Frey, P.R., W.B. Rouse, and R.D. Garris (1991). "Big Graphics and Little Screens: Designing Graphical Displays for Maintenance Tasks," IEEE Transactions on Systems, Man, and Cybernetics 21.

3. Morehead, D.R., and W.B. Rouse (1985). "Computer-aided Searching of Bibliographic Databases: Online Estimation of the Value of Information." Information Processing and Management 21: 387-399.

4. Rasmussen, J. (1986). Information Processing and Human-Machine Interaction: An Approach to Cognitive Engineering. New York: North Holland.

5. Rouse, S.H., and W.B. Rouse (1980). "Computer-based Manuals for Procedural Information." IEEE Transactions on Systems, Man, and Cybernetics 10: 506510.

6. Rouse, S.H., W.B. Rouse, and J.M. Hammer (1982). "Design and Evaluation of an Onboard Computer-based Flight Management System for Aircraft." IEEE Transactions on Systems, Man, and Cybernetics 12: 451-463.

7. Rouse, W.B. (1986). "On the Value of Information in System Design: A Framework for Understanding and Aiding Designers." Information Processing and Management 22: 217-228.

8. Rouse, W.B. (1991). Design for Success: A Human-centered Approach to Designing Successful Products and Systems. New York: Wiley.

9. Rouse, W.B., W.J. Cody, and K.R. Boff (1991). "The Human Factors of System Design: Understanding and Enhancing the Role of Human Factors Engineering." International Journal of Human Factors in Manufacturing 1: 87-104.

10. Rouse, W.B., W.J. Cody, K.R. Boff, and P.R. Frey (1990). "Information Systems for Supporting Design of Complex Human-Machine Systems." In: C.T. Leondes, ed. Control and Dynamic Systems. Orlando, Fl.: Academic Press, pp. 41-100.

11. Rouse, W.B., and S.H. Rouse (1984). "Human Information Seeking and Design of Information Systems." Information Processing and Management 20: 129-138.