|Training Manual in Combatting Childhood Communicable Diseases Part I (Peace Corps, 1985, 579 pages)|
|Module 4: Health education|
|Session 22: Evaluation of health education projects|
TOTAL TIME: 2 hours, 30 minutes
By this session, participants have nearly completed all of the steps in planning a health education project. Still remaining is the important task of developing ways to evaluate the project.
Session 22 begins with a game that helps participants understand evaluation and how it is conducted. Later, the group forms working pairs (as in Session 20) to design specific evaluation criteria for their health education project plans.
· To define evaluation. (Steps 2,3)
· To explain how and when to evaluate a project. (Step 3)
· To develop and critique plans for evaluating a health education project. (Steps 4, 5)
Bridging the Gap Part IV
Helping Health Workers Learn. Chapter 9, pp. 12-22
- 14A Questions for Evaluating Community Participation (From Session 14)
- 20A Planning a Community Health Project (From Session 20)
- 20B Health Education Project Planning Worksheet (From Session 20)
- 22A Criteria for the Evaluation of Strategies
- 22B Evaluation Worksheet
Newsprint, markers, forms and handouts from previous sessions.
This session builds an information from Sessions 16-21. If these sessions have not been used, you will need to expand the time in this session and review the eight steps of the health education process in relation to evaluation.
If you chose to do the "Hollow Squares" activity during Session 20
you may be able to delete Steps 1 and 2 of this session.
Step 1 (15 min)
Ask the participants to select a game or contest with which they are familiar (i.e., baseball, soccer, basketball). Have them form three small groups and have one group represent members of the opposing teams, the second group the fans, and the third group the umpires/referees. Based on these three perspectives, ask each group to develop its evaluation criteria for deciding which team wins the game when no score is kept.
Tell them their criteria can range anywhere from how handsome the players are to how many times a player has committed a foul.
Inform them that they have 10 minutes to develop their list of evaluation criteria for presentation in the next step.
The purpose of this activity is to introduce the concept of evaluation in a non-threatening way. Its objective is to reinforce or raise the point that to determine the success or failure of a project (i.e., winning or losing a game), people need to have a clear understanding to base their decisions on (i.e., something measurable or that provides specific details of what qualities are found in a winning team).
The reason for having the participants develop evaluation criteria from the perspective of players, fans and umpires/referees, is to introduce the notion that evaluation should involve people with different roles or perspectives including:
- Active participants (i.e., team members)
(An alternative to basing this activity on a sports game is to
adapt it to evaluating who wins a beauty contest.)
Step 2 (20 min)
Processing The Game
Ask a representative from each group to read and list on newsprint the evaluation criteria they used in deciding which team won the game. After the groups have presented their criteria, discuss the following:
- What kinds of things do their criteria measure (physical qualities, ways the game is played)?
- Are each group's criteria basically the same?
- Do the criteria measure what is going on in the game (number of people getting to third base) as well as how this is happening (players are getting on base because the pitcher is walking them)?
- Would their criteria have been easier to develop or more realistic if: the group had worked together? it was determined and stated at the beginning what quantitative and qualitative factors usually affect or influence the outcome of a game?
When discussing criteria, the participants should understand that
objectives provide the criteria for evaluating the success or failure of
a program and how you are doing. Objectives must contain statements that are
measurable and observable and, prior to beginning a program (or game), what and
when you will evaluate needs to be determined. Without having some quantitative
or qualitative measures, this game in particular, and health programs in
general, become nebulous areas to evaluate. One final point is the need to
involve the participants in the program, and the community at large when
designing and evaluating the program.
Step 3 (45 min)
Determining How and What to Evaluate
Introduce this step by telling the participants that evaluation can address more than one aspect of a project. The terms "process" and "outcome" are used to emphasize the generic purposes of evaluative efforts. Process evaluation is aimed at improving a project and answers the question "Is our strategy working?". Outcome evaluation is aimed at providing information for a summary judgement of the project and answers the question, "Did we succeed?".
Distribute Handout 22A (Criteria for Evaluating Strategies) and tell the participants, after they have read it, that same of these criteria should be used for evaluating the strategies and related activities they select for their projects.
Ask the participants to spend the next 10 minutes in work pairs reviewing their Health Education Project Planning Worksheets and to think about the following questions:
- Are the objectives stated in terms of what will be measured and when?
- What information on the Project Planning Worksheet will help them determine criteria for evaluation?
- What is the relationship between monitoring and evaluation?
After 10 minutes, begin discussing the foregoing questions. Ask someone to state:
- the purpose of evaluation
- information they need to collect to do the evaluation and how they will collect it
- who should be involved
- when the evaluation should be done
Distribute Handout 22B (Evaluation Worksheet). Using the example from Handout 20B (Health Education Project Planning Worksheet), walk the group through the Evaluation Worksheet.
Participants should understand that evaluation can serve a number of useful purposes including:
- assessing the needs of the community prior to designing program objectives
Be sure participants understand the need to evaluate every part of a project (i.e., the strategies and related activities they have planned as the means to attain their objective) to be able to pinpoint strengths and weaknesses and make appropriate modifications. The evaluation criteria they should use when assessing strategies and related activities are adequacy, appropriateness, effectiveness and efficiency. These terms are defined in Handout 22A. Make sure the participants are comfortable with these terms.
By the end of this discussion be sure participants understand that:
- Monitoring is an integral part of the evaluation process. Through monitoring we gather ongoing information about project progress using pre-established benchmarks or milestones; we periodically ask the question, "Have we gotten as far as we had expected to get at this time?".
Step 4 (35 min)
Developing Evaluation Plans
Ask the group to form the same work pairs as in Session 20 and explain that their task for the next 30 minutes is to develop a plan for evaluating the health education projects they have elaborated an their Project Planning Worksheet. Because of the limited time, tell them to focus on only one of their strategies and related activities.
Tell them to reevaluate and, if necessary, modify their objectives and any other items they have listed on their Project Planning Worksheets. Ask them to attach the evaluation worksheet to the back of their Health Education Planning Worksheet when they have completed it, and to be prepared to describe their plans to the group in the next step.
While the group is working in pairs, walk around the room to
observe how the process is proceeding and assist any groups that seem to be
having problems. Also refer them to pp. 79 of Handout 20A (Planning a Community
Health Project) for further assistance or guidance in this
Step 5 (25 min)
Presenting and Discussing Evaluation Plans
Reconvene the large group and ask for two volunteers to briefly describe their evaluation plans. After each presentation, ask the rest of the group to offer comments on the plan and suggest ways to improve it.
When discussing each plan the following questions should be asked:
- Did you base the objective on information you collected on assessing the needs or knowledge, attitudes and practices of the community (i.e., do you have baseline information to evaluate the change your program is to create)?
- Are your strategies and related activities adequate, appropriate, effective, efficient?
- Are you evaluating how well the program is being done (i.e., process) as well as the outcome after the time specified in your objective?
- Are you including the appropriate people in your evaluation?
- Are you conducting the evaluation at the appropriate times?
- Will the results you think you might obtain be helpful to you in determining whether to continue the program as is, change it or terminate it?
Close this session by emphasizing the importance of basing evaluation on objectives and clearly planning from the beginning how and when it will be done.