What Should Be Evaluated?

The main objective of an evaluation is to influence decisions to continue, change, expand or end a project or programme (Habicht 1999). How complex and precise the evaluation must be depends on who the decision-maker is and on what types of decisions will be taken as a consequence of the findings. Different decision-makers demand not only different types of information but also vary in their requirements of how informative and precise the findings must be. Hospital managers for example, may be interested in knowing what the quality and cost of their services is in order to decide what needs to be done to improve them. District managers, on the other hand, may need data on provision and utilisation of safe motherhood services to plan further amendments to the numbers and types of such services within their district. National agencies may require assessments of coverage or impact to justify further investments in their programme. And finally, international agencies may wish to make global comparisons in coverage and impact to understand global trends in maternal health, for advocacy or to justify continued funding.

It is not possible to design an evaluation strategy that will provide sufficiently valid and precise information for decisions to be taken at all the various levels of decision-making (facility, district, regional, national and international). Rather, evaluations need to be tailor-made to satisfy the specific needs of each level of decision-making. Unfortunately, evaluation efforts in Safe Motherhood have for many years searched for a universal blueprint of evaluation, largely focusing on what to measure (i.e. indicators) rather than on what to evaluate (i.e. who is the information for, what will it be used for). This has resulted in an over-emphasis on indicators that may be of little use in designing or managing effective health care strategies within countries. Only recently has there been a shift in emphasis from indicators of health to indicators of use of obstetric care but there is relatively little experience so far with the use and interpretation of such indicators.

Table 1. Example of indicators to evaluate safe motherhood programmes

One of the first questions to address when designing an evaluation is whether one is evaluating the performance of the intervention delivery or its impact on health or behaviour (Habicht 1999). One may evaluate the provision, quality or utilisation of services, their coverage and impact. Table 1 presents in a logical order the types of information that one may wish to gather on safe motherhood services. First, the services must be available and accessible to the target population, and of adequate quality. Second, the population must accept the services and use them. Third, if service provision, quality and utilisation are high this may result in an impact on health or behaviour. In addition, the recognition that not all women need specialist obstetric care to prevent maternal death has led to the search for indicators measuring the met (or unmet) need for obstetric care in a sub-group of the population. Examples of such indicators are described in detail below.