|Basic Guide to Evaluation for Development Workers (Oxfam, 1995, 96 p.)|
|4 The purpose and use of evaluation|
There are many different reasons why evaluations are carried out. Some good reasons are to measure progress and effectiveness; to look at costs and efficient use of resources; to find out if it is necessary to change the way things are being done; and to learn from what has happened in order to make plans for the future. These are all constructive reasons, within the control of those organising their work.
There are other, less good, reasons why evaluations are carried out. An evaluation might be demanded by a funding agency, who are wondering whether to go on supporting a project; or there might be a statutory requirement for evaluation from a government department. Evaluations are sometimes done as part of a research project as a way of testing out new techniques for gathering information; or sometimes because people who raise funds for an organisation need something to put in their publicity material. Evaluations may be done routinely, as a matter of policy at some high level of the organization, without anyone being very clear about why they are done. Evaluations done for some of these reasons are likely to be seen by those most directly involved in the work under scrutiny as being imposed from outside. The people who are taking part in the activities being evaluated are probably not fully informed in advance, and have little say in the matter of what is evaluated and how. They will seldom be told about the results of the evaluation.
Sometimes, expensive and disruptive evaluations have been carried out by funders when decisions have already been made about whether or not to continue financial support. Evaluations can be used as excuses for getting rid of certain staff, when what was needed was a management review. Where there is a suspicion of serious misconduct within an organisation, is better to do an audit than an evaluation. Evaluation can be misused, to cover up weaknesses in a programme by only focusing on what is 'good'. Conflicting groups within a project or organisation may demand an evaluation, when what is really needed is a facilitator to deal with the conflict, after which an evaluation should be a more constructive process.
If used well evaluation can contribute to development action, but it cannot solve every problem, or answer every question. Above all, evaluation cannot be a substitute for good management and firm decision making; it can only provide information to help in these processes.
Sometimes evaluations are expected to show clearly whether or not a project has been 'successful'. It is often very difficult to show clear evidence of success, because this may depend on so many factors, some of which are beyond the control of the project. It is often easier to show failure, and unfortunately, evaluations often concentrate on this. Another problem is that different groups of people will have different perceptions about what constitutes failure and success. The time-scale can also be a factor: a project may be deemed a failure at one stage, but several years later it may become clear that there were some positive effects. A failure may well spark off other activities that lead to positive change.
Evaluation is concerned with measurements. Evaluators will almost certainly want to know details of costs, and the quantities of resources that the project uses. Many of the inputs which a project uses (salaries, materials, tools, petrol) are measurable. Evaluators will also want to assess the activities that have taken place, and the changes that have resulted from project efforts, and want to know how many people are involved, and the products or services which the project generates. These may sometimes be very obviously measurable in terms of numbers: acreage cultivated, vaccinations carried out, springs protected. Whatever can be expressed in terms of numbers, amounts, and quantities are termed the quantitative aspects of evaluation.
Of course, not all the inputs or changes are measurable in this way. How do you measure advice given, or level of selfconfidence achieved? Evaluators must be very careful that they do not just concentrate on what can be easily counted, but must be sure they take account of the uncountable factors too: such things as beliefs and attitudes, level of knowledge or skill, behaviour, and motivation. These qualitative factors may be extremely important in determining whether projects are successful or not.
It is often what people think about a programme which is the really significant factor in their level of involvement and commitment to it, and therefore whether it achieves its objectives or not. In order to find out why a programme has followed a certain course, it is vital to look at the programme as a whole. It is not enough to know that a programme has succeeded or failed: it is even more important to know why. That means taking account of all the unmeasurable factors as well as the things that can be counted.
1 Evaluation can improve the management of programmes and projects and related activities, and point to the better usage of funds and other resources.
2 Evaluation can help people to learn from experience so as to improve the relevance, methods, and results of projects and programmes, for current and future work.
3 Evaluation can increase accountability:
· to donors: to meet their demands that resources are being used effectively, efficiently and for agreed objectives;
· of donors to the organisations they fund and work with;
· to the men and women in whose name these organisations are working.
4 Evaluation can provide information to enhance communications, within projects and organisations, and between different stakeholders, and also for advocacy work.
1 Evaluation is not decision-making, but can only provide
information for decision-makers.
2 Evaluation should not be used to protect managers from the need to face difficult decisions.
3 Evaluation should not be used for crisis management.