Selection of training goals1

¹ Adapted from “An overview of applied research in medical education problems, principles and priorities”. Christine H. McGuire, WHO Report on the Workshop on the Needs for Research in Medical Education, Alexandria, March 1974.

1.21

Traditionally, this selection has been made by relying on the judgement of experts to determine what a neophyte in the profession ought to know and ought to be able to do. In the past we have relied almost exclusively on this method. As a result curricula are crammed with an ever-burgeoning amount of new and highly specialized knowledge which the student often perceives as irrelevant to his own goals and which, in fact, may be of little value to other than the super sub-specialist. Certainly expert opinion is an important source of information about the knowledge and skills which trainees should be able to demonstrate, but it is also possible to make this decision on the basis of scientific evidence about what competent health personnel need to know and need to be able to do in order to fulfil their responsibilities. A number of procedures have now been developed for collecting such data which provide an empirical basis for working out a behavioural description of the essential components of professional competence. This is of great assistance to faculties in setting goals and designing curricula. Three of these procedures are of special interest: the critical incident technique, task analysis and analysis of epidemiological data.

The critical incident technique

This method consists of collecting data about specific types of behaviour that characterize professional effectiveness and ineffectiveness and using these data to make an objective, empirical assessment of the essential performance requirements of the profession. This technique is an outgrowth of studies in aviation psychology made in the United States during the Second World War. In that programme it was found that in reporting the reasons for eliminating a trainee, pilot instructors and check pilots frequently offered such cliches and stereotypes as “lack of inherent flying ability”, “poor judgement” or “unsuitable temperament”. In an effort to determine the specific characteristics of personnel that contributed to success or failure, combat veterans were asked to report incidents observed by them that involved behaviour which was especially helpful or especially inadequate in accomplishing the assigned mission. This request concluded with the statement: “Describe the officer's action. What did he do?” The several thousand incidents submitted in response to this inquiry were analysed and categorized to provide a relatively objective and concrete description of the “critical requirements” of combat leadership.

To apply this method to the health professions, several thousand incidents describing observations of especially effective or ineffective colleague behaviour are collected from several hundred health workers representing various age groups, geographical areas, professional categories and specialty interests. For example, in a critical incident study of intern and resident performance (i.e. of the general practitioner) commissioned by the US National Board of Medical Examiners, the American Institute of Research collected over 3000 incidents from physicians across the country. The incidents submitted involved all areas of behaviour: practical, communication and intellectual skills. They identified, for example, such general requisites of competence as “Skill in gathering clinical information,” i.e. in taking a competent history and in performing an adequate physical examination, or “Skill in relating to the patient and in gaining his cooperation in a treatment plan”. In a similar study conducted by the University of Illinois Center for Educational Development of the critical performance requirements in orthopaedic surgery, over 1700 incidents were collected from more than 1000 orthopaedic surgeons representing various practice settings and sub-specialty interests. An empirical classification defining 94 critical performance requirements, grouped into nine major categories of competence, was derived from the incidents. This operational and prospective definition of the essential components of competence could then be used to determine the goals of specialty training, the design of programmes for their achievement and the criteria and methodology for their evaluation. If educational planning were regularly based on such operationally defined, empirically derived goals, educational programmes would look quite different.

Task analysis

1.22

A second method of determining the essential components of professional competence which should define educational objectives consists in detailed task analysis of what various categories of health personnel actually do, and in deriving from that list of tasks a statement of the knowledge and skills (what should be done, not merely what is done) which they must have to perform competently. Such a task analysis should be based on careful, systematic observations of the activities of a representative sample of various categories of staff or on the daily logs of a representative sample who report in minute detail the way in which they spend their working days over a specified period of time, or on some combination of these two approaches.

Wherever this method has been employed, the results have been most enlightening. For example, in a limited pilot study of paediatricians in a typical small US city, researchers found that all the physicians had different but consistent patterns for taking a history and performing a physical examination. Of the 481 patient visits observed, 222 were well children; an average of 10.2 minutes was spent with these children (range: 7.5 minutes to 13.6 minutes) in contrast with an average of 8.1 minutes spent with ill children (range: 7.4 minutes to 10 minutes). Of the 259 ill children, 104 (i.e. 40%) were diagnosed as having an infection of the upper respiratory tract, 15 had chronic illnesses and five had potentially dangerous diseases. For the total group of 481, optic fundi were examined only nine times and rectals were performed in only six cases; two physicians did not percuss the lung fields for any patient. The greatest amount of time was spent in discussion of nutrition and child development. The single most frequent topic on which advice was rendered in well-child care concerned toilet training. The authors of this study concluded, “Few aspects of well-child care appear to require the skill of a physician... the question is also raised as to whether current training programmes are aggravating the physician manpower shortage by overtraining in relation to community health needs.”¹

¹ Bergman, A., Probstfield, J. and Wedgewood, R. Performance analysis in pediatric practice: preliminary report. Journal of Medical Education, Vol. 42: 262 (1967).

This is a question that could apply to all members of health teams in every country; only task analysis or comparable empirical studies will give us the answer.

Epidemiological studies

One of the most interesting of the newer approaches to the use of such studies consists in combining three arbitrarily weighted factors - disease incidence, individual disability and social disruption - to define priorities in health care needs and, hence, in educational effort. As initially developed by Dr John W. Williamson², the three factors are computed as follows: disease incidence consists of a simple tabulation of the frequency of the disease (e.g. pneumonia) or other medical condition (e.g. pregnancy) in the target population. Individual disability involves a determination of the extent of patient disability or risk associated with a given medical condition; an Individual Disability Weight (IDW) is calculated for each condition from three elements: the average length of hospital stay, mortality rates and complication rates. Social disruption represents an estimate of the disruption that would be produced by a given disease or condition in the social group of which the patient is a member; it is based on such factors as cost of illness, age of patient and number of dependents, socioeconomic standing and the like. For each discharged patient a Total Priority Weight (TPW) is calculated combining these elements. This Total Priority Weight is then arbitrarily apportioned among patient diagnoses. Finally, a cumulative total for each diagnosis is calculated from the total patient sample. The resultant ranking represents a quantitative estimate of health care needs or priorities for the population at risk.

² Williamson, J. et al. Journal of the American Medical Association, Vol. 201: 938 (1967) and Vol. 204: 303 (1968).

It is clear that even with unlimited resources not all of these needs could be met in the present state of our knowledge. The next step therefore consists of determining what portion of total health care needs can be met, given our present understanding of disease and our present treatment possibilities. This portion indicates the target area for application of professional skills and helps to define educational priorities. The goals of education for health service staff can therefore be defined as encompassing those areas of health care needs that cause the greatest total preventable disability - i.e. those that cause the greatest total disruption that could be reduced or minimized by early diagnosis and appropriate intervention.

In his early studies using this method to review hospital practice in two large community hospitals in widely separated metropolitan areas in the United States, Dr Williamson found that pregnancy, including uncomplicated delivery, ranked first or second in priority in both hospitals, that cerebral vascular accidents ranked among the first five diagnostic categories in both hospitals and that fractures of the lower extremities ranked among the first five in one hospital. These particular conditions are mentioned because in certain educational institutions there is a general tendency to reduce the amount of clinical instruction for the general medical student in some of these areas. For example, instruction in orthopaedic surgery is often elective despite the fact that trauma in general accounts for a very significant proportion of total preventable disability.

While the study reported above was limited to hospital practice, the same method could easily be applied to any level of health practice. In addition, while the findings from such epidemiological studies and the particular weights to be assigned to such factors as individual disability and social disruption will, of course, vary markedly in different parts of the world, the approach is clearly applicable to any society for which health personnel are being trained.

In all parts of the world, use of such data will modify the goals and priorities of educational institutions and the emphases in curricula by focusing far greater attention on ambulatory medicine and on the more common causes of disability.

Implications of applied research on goals and priorities

It can be seen from the above that the means are now at hand for supplementing expert judgement with data derived from empirical studies to assist us in defining the roles and, hence, the skills required of students on completion of programmes. If such studies were carried out as a matter of course and if the findings were used to develop explicit educational objectives for the health professions, we should see revolutionary changes in the kinds of health professionals produced and in their training programmes. Furthermore, such changes would have a far greater impact on meeting health care needs than would simple expansion of educational facilities of the conventional type.

Here we should mention some simpler but also more rapid and less costly techniques which can be used to complement or replace other methods. These methods are not mutually exclusive:

- Interviews with members of the profession, who are asked to describe what, in the light of their experience, should be the functions and tasks of any member of the health team.

- Questionnaires, made up of either open-answer questions (what are the functions of...?) or closed-answer questions (which of the tasks listed below...?).

- Personal log-books kept by health professionals, describing the actual work carried out each day and recording the time spent on each activity.

- The simplest method consists of asking each of a group of colleagues to put himself in the shoes of a person needing care and to describe the functions and tasks that he would wish a given member of the health services to be able to perform. Comparison of the lists submitted will lead to rapid agreement on a common list of sufficiently high quality to provide a basis for a productive discussion on the relevance of the programme, for example.

1.24