The purpose of evaluation

Why evaluate?

Society, which finally pays the bill for nutrition education activities, has a right to know how resources have been used and the final impact of educational programmes. Evaluation of educational programmes are undertaken for several reasons: to judge how the nutrition education programmes are planned and executed, how the programme personnel have performed, and to increase the effectiveness of programme management and administration; to assess the utility of new programmes; and to satisfy programme sponsors (see Figure 1) (Oshaug, 1992; Rossi & Freeman, 1993; Oshaug et al., 1993). In all evaluation efforts it is very important that the purpose of the evaluation is clear from the beginning.

The functions of evaluation

Evaluation of nutrition education programmes includes not only collection of qualitative and quantitative data, but also their analysis and interpretation for the purpose of making judgement and decisions. In this context, evaluation is seen to have two main functions: formative and summative. Formative evaluation is used to improve and develop programme activities as they are carried out, and is therefore continuous. Summative evaluation measures the outcome of an activity or set of activities (Oshaug, 1992). It is also used to satisfy the accountability² requirements of programme sponsors. By providing feedback or involving people in evaluation activities, programme beneficiaries can be motivated about its usefulness. Furthermore, evaluation may have psychological or sociopolitical functions as it is used to increase the awareness of educational activities or promote public relations. Another function is to facilitate supervision. In an organisation responsible for a nutrition education programme, it is the responsibility of a manager to evaluate personnel and programme activities under her or his responsibility. This may be referred to as the administrative function of evaluation (see Figure 2) (Oshaug, Benbouzid & Guilbert, 1993).

² Rossi and Freeman (1993) discuss six common types of accountability studies in established programmes, directed at providing information about various aspects of programmes to stakeholders: 1. Coverage; 2. Service; 3. Impact; 4. Efficiency; 5. Fiscal; and 6. Legal accountability.

Figure 1: Reasons for evaluating nutrition education programmes

Figure 2: Functions of evaluation

Definition of evaluation

One dimension, and a recurrent question is, how to define evaluation. Guba and Lincoln (1989), agree that it is reasonable to begin with a definition of what we shall mean by the term evaluation. They proceed, however, by stating that definitions of evaluation are human mental constructions, whose correspondence to some “reality” is not and cannot be an issue. Therefore, according to their opinion, “there is no ‘right’ way to define evaluation, a way that, if it could be found, would put an end to argumentation about how evaluation is to proceed and what its purposes are”. Such statements are not very helpful to non-professional evaluators. They contribute to the confusion and protect the field as a playground for the “good guys”. Luckily several authors think a definition is important and can be provided.

In a recent and authoritative book, evaluation is defined in such a way that it comprises a whole programme cycle, from assessment of problems and needs to outcome or impact evaluation of social programmes (Rossi & Freeman, 1993). Here the definition of evaluation is:

Evaluation is the systematic application of social research procedures for assessing the conceptualisation, design, implementation, and utility of social intervention programmes.

This definition includes any type of information gathering from the very start of a situation analysis to the final outcome of social programmes. It is a very broad definition, and beyond many people’s understanding of evaluation. The assessment part of programme conceptualisation is a part of the evaluation. The authors’ specific reference to social research procedures is, however, limiting, and one may accuse them of being biased in their theoretical orientation, evaluation procedures, selection of methodology and analytical approach. Other writers also see evaluation as an integrated part of programme planning and management, whether it is a training/education programme, a specific nutrition intervention, development activities, or education of the public (McMahon, Barton & Piot, 1980; Romiszowsky, 1984; Oshaug, 1992; Oshaug et al., 1993). For community nutrition, evaluation has been defined as follows (Oshaug, 1992):

The evaluation of a programme is a systematic collection and delineation and use of information to judge the correctness of the situation analysis, critically assess the resources and strategies selected, to provide feedback on the process of implementation and to measure the effectiveness and the impact of an action programme.

This is also a broad definition, but it links the evaluation activities to a specific programme or activity. Here evaluation is seen as an essential management tool for all community nutrition activities, including nutrition education of the public. It includes a range of methodologies from medicine and social science to those specific to nutrition. All definitions stress the importance of planning the evaluation at the same time as the programme to be evaluated.