Cover Image
close this book Opportunities for Control of Dracunculiasis (1982)
close this folder Working Papers
View the document 1 Problem Assessment and Data Collection
View the document 2. Control Strategies
View the document 3 Program Monitoring and Evaluation

3 Program Monitoring and Evaluation

The contents of this section necessarily overlap with those of the previous two on problem assessment and surveillance and control strategies. On the one hand, monitoring efforts should be compatible with data gathered as a part of problem assessment--to document changes in incidence, disability, and case severity. On the other hand, there is a need to demonstrate the relationship between control efforts and these changes. Workshop participants therefore attempted to establish a minimally acceptable level of monitoring and evaluation and suggested additional steps that could be taken to increase confidence in findings.

The existence of only a few reports showing significant reduction of dracunculiasis as a result of interventions other than provision of safe water supplies points to the need to emphasize evaluation. Evaluation is needed to compare the effectiveness of different strategies and programs. Monitoring is understood to refer to frequent, routine, ongoing analysis of program activities; evaluation here refers to relatively infrequent, periodic, comprehensive reviews of progress toward program goals in training, education, surveillance, collection of baseline information, improvement of water source, and, ultimately, a reduction in dracunculiasis incidence.

Process Indicators To Monitor Program Coverage

Each operational component of the anti-dracunculiasis strategy should be monitored regularly as an interim measure of the program's effectiveness in order to permit corrective action to be taken promptly, when necessary. Such operational or process indices are important in this disease since the long incubation period limits the use of disease outcome (i.e., reduced incidence) as an Indicator of program effectiveness on a weekly or even monthly basis. Process indicators are expressed as a comparison of the number of selected activities actually carried out to the number of similar activities scheduled as a part of program operations.

The director of control activities and other members of the national steering committee or task force should agree early in the program on key control activities that will serve as process indicators. Agreement should also be reached about which activities should receive priority, so that monitoring efforts can continue throughout the life of a program, despite unforeseen cutbacks in resources. By relating these activities to an estimated operational timetable, attention can be directed to unexpected delays and alternative approaches can be considered. Process measures should be calculated by program management staff on a regular basis, either as part of routine activity reports or as a brief sample survey conducted in villages targeted for control activities. A periodic comprehensive program review, or evaluation, should be undertaken every 2-3 years to include:

- A thorough review of program operations

- Extent to which program objectives have been achieved

- Identification of problems

- Recommendations for improvement of strategies.

Based on the principle of technical cooperation among developing countries (TCDC), workers from programs in other countries should be invited to participate in these reviews.

The extent to which program goals are achieved can be determined by calculating the percentage of planned activities that are actually carried out. A list of these activities, and criteria for determining completion of each, follow:

1. Provision of protected water sources,

determined by presence of properly constructed wells or other sources of protected water, evidence of properly functioning equipment, and evidence of maintenance.

2. Health education.

Process measures can be based on reports from individuals responsible for health education, validated where possible through documentation of occurrence of health education activities in a sample of villages. Such documentation might include the type of activity (e.g., community organization efforts, small group discussion, contact with affected individuals, audiovisual programs, and placement of posters), the date held, and approximate attendance. The success of the program will be influenced by the extent to which educational activities are related to desired changes in dracunculiasis-related behavior.

3. Chemical disinfection of water sources,

evidenced by signature in book kept by village chief, schoolteacher, etc., signifying when chemicals were added, amount, and by whom. Availability of supply of chemical in village could also be a criterion, but only if feasibility study indicates someone in the village is capable of administering the chemical.

4. Treatment of patients.

Treatment of patent cases is part of patient care under the primary health care plan and should not be the major priority in a control program. Where treatment of cases is used to help enlist community participation, coordination with local health personnel will be important. The following proportions of all active dracunculiasis cases might be measured: those with bandaged active dracunculiasis lesions and those reporting controlled prophylactic expulsion of larvae from ulcers.

5. Training.

The program should include specific objectives relevant to the training of different types of health workers. A time frame and a means for assessing the extent to which training objectives are being met should be established. The extent to which needed training materials are produced, available, distributed, and used should be monitored.

Outcome Indicators To Measure Program Effectiveness

The effectiveness of dracunculiasis control activities can be assessed in two ways: (1) through program impact measures--by determining if the activities carried out had the desired effect (e.g., changed human behavior with regard to use of water for drinking); and (2) through epidemiologic measures--by determining whether the activities carried out reduced the incidence of dracunculiasis. Both categories of information are desirable for measuring the relative value of approaches to control. For example, persistence of dracunculiasis in villages where control efforts are judged to have been successful might indicate recent immigration of infected individuals to the area. (See questionnaire, Figure 9, regarding length of residence. Infected respondents with less than 1 year of residence could be imported cases.)

Program Impact Measures

Program impact measures indicate whether program activities led to their intended result or objective. They are often very useful in analyzing a pilot program, to provide information about which activities are more effective and therefore should be included as part of a wider program. In monitoring a regional or national program, impact measures might be calculated every 2 years in a sample of villages to determine the extent of:

1. Prevention of infected individuals from contaminating unsafe water sources

2. Use of a protected water source, if available

3. Use of unprotected water sources

4, Physical or chemical protection of all unsafe water sources

5. Household measures to filter (or possibly boil) drinking water

6. Appropriate measures to keep dracunculiasis ulcer(s) clean.

Where resources are limited, the above information should be collected by giving highest priority to item 1 and lowest to item 6.

Epidemiologic Measures

The most important epidemiologic measures are the number of cases, reported through existing reporting systems or detected by active surveillance, and the number of affected villages, obtained through active case detection (e.g., counting cases by village once the size of the population has been established or estimated). Where resources to provide comprehensive coverage are not possible at the outset, a sample of representative villages should be identified and followed over time. Eventually, however, information derived from comprehensive coverage will be necessary. The 1-year lag in appearance of patent infections makes more difficult the determination of effectiveness of interventions on outcome. Programs should consider classifying affected villages according to level of annual incidence (less preferably, according to prevalence or incidence during the peak transmission season): hyperendemicity = 20% or more; mesoendemicity = 5-19%; hypoendemicity = less than 5%. Because school-aged children (6-14 years old) and adults (15+ years) appear to have greater exposure to dracunculiasis than preschool children, they might be appropriate population subgroups for classification (see Figure 3).

Evaluation Of Socioeconomic Benefits And Program Costs

Socioeconomic effects of this disease are substantial and may be direct or indirect. For example, a decrease in average school attendance may result directly from disabilities suffered by school children or from the need for children to replace disabled parents in the field.

Although the evaluation of the social and economic benefits and costs of dracunculiasis control programs is optional, such evaluation can provide a sound basis for determining which intervention(s) can be implemented most effectively within a limited budget, and can provide useful information for Justifying previous expenditures and future requests for resources. Rural development and water supply agencies should be willing to assist in such data collection and interpretation, especially where control efforts are added to existing multipurpose programs.

Measuring Socioeconomic Change

Baseline data should be collected in endemic villages prior to the start of control activities. If possible, baseline data should also be collected in endemic villages not targeted for immediate program activities, so they may serve as controls. Alternatively, one could measure selected socioeconomic indicators in comparable affected and unaffected villages before and after control activities. The following indicators are suggested as measures of socioeconomic benefit resulting from reduction or elimination of dracunculiasis:

1. Differences in labor, as manifested by number of days worked (time), productivity per day or week (output: area harvested or planted, or amount of crop harvested or planted), or wages earned (pay). In addition to these individual measures, entire villages may be compared by their per capita production. Alternatively, in rural areas where seasonal agricultural work is the norm and "cash" for services or "output" terms may be less applicable, one might inquire about the number of days workers were restricted or disabled during a specified period.

2. Differences in school attendance, as manifested by either total number of days missed or days missed because of dracunculiasis only; or average daily attendance during the dracunculiasis season as compared to other times of the year, before and after control programs.

3. Possible differences in nutrition. Where the disease is highly prevalent in a precariously balanced subsistence economy, the presence or absence of dracunculiasis and its attendant secondary effects on the capacity of affected villagers to undertake farmwork may be manifested in demonstrable tertiary effects on the nutritional status of affected villages, especially in young children. Compare weight-for-height profiles to assess recent nutritional/caloric deficiency.

Programs in the Bendel State of Nigeria and the Dimbokro District of Ivory Coast have already afforded opportunities to measure changes in dracunculiasis incidence where safe drinking water sources are being provided. Such changes should be documented as carefully as resources permit and should be publicized when and where significant program efforts are being made. Important items that may be measured include changes (preferably compared with control villages) in:

1. Incidence of dracunculiasis

2. Productivity and labor

3. School attendance

4. Preschool nutrition

5. Direct costs of health care for dracunculiasis patients.

Costs of Control Programs

Information on costs will be extremely valuable to program administrators. Certain costs such as those for provision of wells and water systems, since they address a broad variety of health and socioeconomic issues, should be allocated among multiple programs. Similarly, health personnel already employed by state or national agencies might be used to conduct dracunculiasis control activities on a temporary or part-time basis. Thus the additional costs attributed to these workers may be less than proportional to the additional effort. Other costs that need to be calculated by control program administrators include:

1. Transportation

2. Special supplies, including pesticides, training materials, medical supplies, educational or promotional materials

3. Research costs, including operational studies to develop methodologies for measuring costs of dracunculiasis control programs

4. Training costs, including instructor compensation, trainee wages, materials, travel reimbursement, etc.

Determination Of Disease Elimination

There is limited experience concerning the measures needed to ensure that dracunculiasis has been eliminated from an area. However, at least two consecutive annual active case searches in recently affected villages during the peak transmission season are desirable. Although house-to-house searches are more costly, they should result in a higher degree of certitude. These active searches may be combined with other rural surveys.

Disease may be considered eliminated in a village if no new indigenous cases are discovered during two consecutive annual searches. Before a district or endemic area can be considered entirely free of dracunculiasis, however, one search must be made of all villages in the district or area, regardless of whether they have been known previously to harbor the infection. A case imported into an area that was previously disease-free should provoke a follow-up of that village for two consecutive years.