Cover Image
close this bookEducational Handbook for Health Personnel (WHO - OMS, 1998, 392 p.)
close this folderChapter 2: Evaluation planning
View the document(introduction...)
View the documentWhat is evaluation?
View the documentContinuous evaluation formative and certifying evaluation
View the documentAims of student evaluation1
View the documentCommon methodology for student evaluation1
View the documentComparison of advantages and disadvantages of different types of test
View the documentEvaluation in education qualities of a measuring instrument
View the documentEvaluation is a matter for teamwork

Evaluation is a matter for teamwork

2.40

The planning of an evaluation system is obviously not simple. It is a serious matter, for the quality of health care will partly depend on it. It has been stressed many times, moreover, that it should be a group activity. We have stated in the preceding pages that evaluation must be planned jointly; that implementation of any evaluation programme is the responsibility of the teachers, in collaboration with students and the administration; that evaluation carried out jointly by a group of teachers is less likely to be erroneous than when carried out by one person; and, finally, that critical analysis of a test by colleagues is essential to its sound construction.

This work performed jointly by a group of teachers calls for a coordinating mechanism. The terms of reference of each group and group member must be defined explicitly and known to all. The institution's higher authorities must provide the working groups and their members with the powers corresponding to the task to be accomplished.

The diagram on page 2.44 shows one type of organization and meets the needs of a given institution. Other types of organization can be envisaged, according to existing structures and local traditions. Now construct the type of organization that will be needed by your institution.

It will obviously be best if you can discuss the following exercises (pp. 2.41 and 2.42) with some of your colleagues. To help you to complete these exercises, take them in the following stages:

1. Read the instructions on objectives 8 and 9 on page 2.02. Then study pages 2.02 to 2.16, If you are taking part in a training workshop, ask the facilitator for some explanations, if necessary.

2. Do the exercise on page 2.09:

- if possible, exchange your proposals with some of your colleagues.

3. For each decision, select the most appropriate “means” of obtaining the information you need to make your decision:

- make a list of these “means” (page 2.41);

- if possible, exchange your proposals with some of your colleagues;

- if you are taking part in a workshop, draw up a joint list of “means”.

4. Specify the type of human resources needed to prepare and use these means:

- read pages 2.17 to 2.19;

- if possible, exchange your proposals with some of your colleagues;

- draw a diagram (page 2.42) which includes the terms of reference for each component element (who does what);

- do the exercise on page 2.43, on the basis of your diagram.

5. If possible, discuss your diagram with a few colleagues to make sure it has every chance of being suitable for use and used in your institution.

EXERCISE

2.41

1. Draw up a list of the “means” which you think should be included in an evaluation system.

2. Show which of these are in practice already included in the evaluation of the educational programme in which you are involved.

Evaluation system

List of elements

Elements included


Yes

No




Do not change anything that works satisfactorily .... what is satisfactory to some, however, is not necessarily good enough for others. Teaching is a matter for teamwork.

EXERCISE

2.42

Taking the list of means you have drawn up on the previous page for an evaluation system, draw a diagram to show the type of organization (commissions, committees, boards, etc., with a description of their functions) which would seem desirable (in the establishment where you are working) for introducing (or improving) an evaluation system capable of providing the data needed to assure you that the training establishment in which you are working is functioning efficiently.

Diagram

Compare your diagram with the diagram on page 2.44.

EXERCISE

2.43

Describe the obstacles you are liable to encounter in applying the organizational plan you have imagined on the previous page and indicate tactics for overcoming each of these obstacles.

Obstacles

Tactics


Organizational diagram showing relationships between curriculum committee, evaluation committee and teaching units

2.44

EXERCISE

2.45

(Check your answers on p. 2.48.)

Question 1

The main role of evaluation is: ________________

Question 2

The purpose of evaluation is to make a value judgement concerning:

A. Students and programmes.
B. Students and teachers.
C. Programmes and teachers.
D. Students.
E. None of the above.

Question 3

Thorndike's “Law of Effect” is based on the fact that:

A. Students learn better when they are motivated.
B. Students learn better when they play an active role.
C. Students are receptive when they understand the educational objectives which have been defined.
D. Students tend to engage in activities which have success associated with their results.
E. Students work better if the teacher makes an impression on them.

Questions 4 to 8

Indicate to which of the following each question refers:

A. Formative evaluation
B. Certifying evaluation
C. Both
D. Neither

Question 4

Its main aim is to inform the student on his/her progress.

Question 5

Does not preserve anonymity.

Question 6

Enables the teacher to decide to replace one programme by another.

Question 7

Justifies the decision to let a student move up from the second to the third year.

Question 8

Permits rank-ordering of students.

Questions 9 to 16

For each of the aims of student evaluation (list numbered 9 to 16, p. 2.18), indicate whether the appropriate measuring instrument will be of the certifying evaluation type (C) or both certifying and formative evaluation (CF).

Question 17

The four steps of the process of student evaluation are as follows:

1. ___________________________________

2. ___________________________________

3. ___________________________________

4. ___________________________________

Question 18

All the following steps except one are essential in constructing any measuring instrument.

A. Precise definition of all aspects of the type of competence to be measured.

B. Obtaining reliability and validity indices for the proposed instrument.

C. Making sure that the type of instrument chosen corresponds to the type of competence to be measured.

D. Making sure, by an explicit description of the acceptable level of performance, that the use of the measuring instrument will ensure objectivity

E. Determination of the particular behaviour expected from individuals who have or have not acquired the specified competence.

Question 19

When evaluating communication skills (domain of interpersonal relationships), all the following steps should be taken except one:

A. Describe specific types of behaviour showing a given affective level.

B. Describe explicit types of behaviour showing the absence of a given affective level.

C. Observe students in real situations enabling them to manifest the types of behaviour envisaged.

D. Obtain the agreement of a group of experts on the relationship between explicit types of behaviour and the affective level envisaged.

E. Obtain the students' opinions on the way in which they would behave in specific situations.

Question 20

The essential variable to be considered in evaluating the results of teaching is:

A. The student's performance.
B. The opinion of the teacher and his colleagues.
C. The opinion of the student regarding his performance.
D. The satisfaction of the teacher and the students.
E. The teacher's performance.

Question 21

Which of the following is not suitable for measurement by written examinations of the “objective” type:

A. Ability to recall precise facts.
B. Ability to solve problems.
C. Ability to make decisions.
D. Ability to communicate with the patient.
E. Ability to interpret data.

Questions 22 and 23

If the following qualities can be attributed to an examination:

A = Validity

B = Objectivity

C = Reliability

D = Specificity

E = Relevance

Question 22

What quality is obtained if a group of experts agree on what constitute good answers to a test?

Question 23

What quality implies that a test consistently measures the same thing?

Question 24

The following factors, except one, generally affect the reliability of a test:

A. Its objectivity.
B. The mean discrimination index of the test questions.
C. The homogeneity of the test.
D. The relevance of the test questions.
E. The number of questions in the test.

Question 25

Which of the following test criteria is influenced by all the others?

A. Reliability.
B. Validity.
C. Objectivity.
D. Specificity.
E. Relevance.

Suggested answers for the exercise on pages 2.45 - 2.47.

Question

Suggested answer

If you did not find the correct answer, consult the following pages again

1

®

2.02, 2.06

2

E

2.08

3

D

2.13

4

A

2.15 to 2.19

5

A

2.15 to 2.19

6

C

2.15 to 2.19

7

B

2.15 to 2.19

8

C

2.15 to 2.19

9

CF

2.15 to 2.19

10

CF

2.15 to 2.19

11

CF

2.15 to 2.19

12

C

2.15 to 2.19

13

C

2.15 to 2.19

14

CF

2.15 to 2.19

15

CF

2.15 to 2.19

16

C

2.15 to 2.19

17

®

2.20

18

B

2.21

19

E

2.07, 2.21, 2.35

20

A

2.08, 2.21

21

D

2.27, 2.30, 2.31

22

B

2.27, 2.30, 2.31

23

C

2.33 to 2.35

24

D

2.36, 2.37

25

B

2.36, 2.37

Personal notes

2.48