Session 1: What do he need to know? How can we find out?

Purpose:

In this session, trainees review the basic uses of evaluation and they list questions about community nutrition projects that evaluation can help to answer. They also identify the information needed to answer these questions and the ways of collecting it.

Time: 2 hours

Materials:

- Newsprint and marking pens or chalkboard and chalk
- Handout - "Hypothetical Project Information"
Note: Information from an actual project may be used instead.
- Handout - "Quest ions About Community Nutrition Activities.

Preparation :

- If you are using newsprint, prepare the chart shown in step 7 below.
- Make copies of the Handout - " Hypothetical Project Information."
- Make copies of the Handout - "Questions About Community Nutrition Activities."

Steps:

Part 1 - What do we need to know?

1. Ask the questions below one at a time. Encourage participants to brainstorm their answers. Write key phrases from participants' answers on separate sheets of newsprint together with each of the questions.

- What do we mean by evaluation? or evaluating a project?
- Who is interested in evaluation?
- Why do we evaluate projects and activities?

2. Talk briefly about the generally accepted definitions of evaluation, distinguishing between ongoing evaluation (monitoring) and final project evaluation.

Explain the management cycle (planning, implementation and evaluation) showing how evaluation leads to new and revised plans, based on lessons learned and changing circumstances during one complete management cycle. During the life of a project we may complete many management cycles.

3. Display the following questions and ask participants to keep them in mind during the rest of the unit. These are the general questions that evaluation can help us answer.

- Did we do the things we planned to do? If not, why not?
- Did the target group respond the way we thought they would? If not, why not?
- Did the nutrition and health of our target group improve?

4. Divide the participants into small groups (4-6 persons each) and inform them that they have been chosen as Project Evaluation Teams for a model community nutrition project. Give each participant a set of objectives and a work plan either from a hypothetical or from an actual project. Ask them to review these project materials individually. (Hypothetical project information is included.)

5. Ask each Project Evaluation Team to make three lists of questions they will ask at different times in the project implementation, to make sure the project is on target. The first list should be the questions they will ask each month; the second list should be those they will ask after six months of project activity; the third list should be those questions they will ask at the end of the project to find out if the project has been successful.

6. When groups are finished, have the first group display its list of monthly questions; the second, its list for evaluation at six months; and the third, its list of questions to be asked at the end of the project. When they have finished, ask the remaining groups to add questions they would ask that were not mentioned.

The facilitator should congratulate the evaluation teams, and tell the group they will be referring back to these lists during the rest of the session.

7. Choose several of the evaluation questions listed in each category, and transfer them one at a time to the first column of a chart like the one that follows.

8. Fill in the second column of the chart by asking participants to be specific about the information they must have to answer each question.

Example:

9. When you have finished listing the information needed to answer three or four questions, pass out the Handout "Questions About Community Nutrition Activities." Review it with the participants. You may wish to make up some exercises to demonstrate how certain information and indicators might be compared and used to answer evaluation questions.

Part 2 - How can we collect the information we need?

10. Review with participants the general methods and tools managers use to collect the information they need to monitor and evaluate the activities and results of community nutrition projects. Include a description of:

- Baseline survey
- Interviews with workers, leaders, beneficiaries
- Client records
- Community reports (monthly, quarterly, semester, etc.)
- Supervision reports
- Mini-surveys
- Meetings/discussion groups
- Existing records

11. Discuss the factors that affect the information we decide to collect and the methods we use.

- What do we need to know? Only information that we must have to guide the project and evaluate results should be included.
- How much money and other resources do we have for the evaluation component of the project?
- Who can collect and compile the information we need? If community workers cannot read and write, or if there are only a few supervisors with minimal transport, the amount and quality of information we can collect will be limited.

12. Add two more columns to the chart so it looks like this:

Ask participants to help you complete the chart for the questions you worked on earlier. For each piece of information ask "How will you collect the information you need?" and "How often will you collect it?"

13. Individual Evaluation Plans - If there is sufficient time during the workshop, each trainee should be asked to use a chart, like the one above. Have the trainees identify the information and sources needed for their own projects. Trainers should be available at this time to assist the trainees.

HANDOUT

HYPOTHETICAL PROJECT INFORMATION

Nutrition Action in the Village of Ngamani

Background

In the village of Ngamani, the (community members, women's group, health committee, etc.) started a nutrition action project in June 1982. The village has about 500 families, with about 400 children under five years old.

The (women's group. committee. etc.) was helped by the (health worker. social development officer, agricultural extension worker) to understand more about the problem of malnutrition and what they could do to reduce the high level of malnutrition in the community. (She/He) helped the community to decide what kinds of activities they would try. Together they wrote a work plan and set targets for the first year.

They approached the (Ministry of _____, the Women's Bureau) with a letter telling about the nutrition problem in the community and what the group wanted to do to solve it. The (Ministry) promised $(_____) and a hand grain-grinding machine to help them get the project started.

The Ngamani village project has three components: regular growth monitoring and follow-up; nutrition and family planning education; and village production, sale and distribution of a weaning food supplement.

Project Objectives

By the end of the first year of the project, the (community. women's group. etc.) hopes to have accomplished the following:

1. Assessed (weighed/measured) the nutrition status of at least half of the 400 children under five in the community (three) times each during the first year.

2. Reached at least 100 families with each of their monthly educational activities.

3. Made and gave, or sold at a very reduced cost, weaning foods to each of the "high risk" children identified in growth monitoring. Each child will receive 2 kg of weaning supplement per month.

4. Made and sold about 50 kg of weaning food a month.

5. Increased the number of families practicing family planning.

6. Improved the condition of at least 75 percent of the "high risk. (malnourished) children they have helped.

Note: The project work plan is on the next page.

Work Plan - Year 1

HANDOUT

QUESTIONS ABOUT COMMUNITY NUTRITION ACTIVITIES

MONITORING AND EVALUATION	WHAT INFORMATION CAN WE USE TO ANSWER THIS QUESTION?
Growth Monitoring:
1) Has growth monitoring been carried out as planned in the community (i.e., at least once every _____ months )	- How many growth monitoring sessions have been carried out in this reporting period? Compare this number to the number planned.
2) Has growth monitoring reached most of the children 0-3(5) yrs in the community?	- What percentage of the total children 0-3(5) yrs. have been measured/weighed at least _____ times? Calculate:
Nutrition/Health Education:
1) Have group education sessions been carried out as planned?	- How many group education sessions have been carried out in this reporting period? Compare this number to your plan.
2) How many families have participated in nutrition education sessions?	- The average number of adults participating in group education sessions. Calculate:
3) If education is carried out in home visits, have home visits been carried out as planned?	- How may home visits were made during this period? Compare this to home visits planned.
Follow-up and Improvement of "High Risk" (Malnourished) Children:
1) How many of the "high risk" children found during growth monitoring have been treated according to your plan for follow-up of "high risk" cases?	- What percentage of "high risk" children identified have received: a. at least ___ home visits? b. growth monitoring at least ___ times per month? c. referral? d. weaning food supplements at least ___ times per month? (This depends on the plan for follow-up for each project.)
2) Have "high risk" children improved as a result of the project?	- What percentage of "high risk" children identified have improved, stayed the same or become worse during your project? Calculate:
	+100 (should equal)
Birth Spacing:
1) If the project supplies contraceptives, how many couples received contraceptives supplies from the project?	- How many of each contraceptive has the project distributed? To how many couples?
	- How many couple-years-of-protection can be attributed to the project?
2) Has the use of family planning methods increased	- Compare the number of family planning acceptors at the beginning of the project, to the number at different times during the project and at the end of the project.
Immunization:
1) Has the number of children with completed immunizations increased?	- Compare the number of children with complete doses of specific vaccines at the beginning and at the end of the project.
Breastfeeding/Weaning Practices:
1) What percentage of women are following the infant feeding guidelines of the project? For example: 0-5 months breastfeeding only, 5-24 months breastfeeding plus mixed solid diet.	- Compare percentage of infants 0-5 months who are breastfed only, to those given a bottle or bottle and breast.
	- Compare the percentage of children 5-24 months being fed according to the guidelines, to those being fed in other ways.
2) Has the project had any effect on the number of mothers breast and bottlefeeding?	- Compare at the beginning and at the end of the project, the percentage of infants 0-12 (24) months who are breastfed and not bottlefed.
	- Compare at the beginning and at the end of the project, the percentage of infants 0-12(24) months who are fed with a bottle.
Community Weaning Food Distribution:
1) How many families have benefited by the weaning food distribution (or sale)?	- Number of families receiving supplement for one or more children in each month. (Calculate percent of total number of families with children.)
2) What is the cost of the production and distribution of weaning food?	- Calculate: -Actual costs of raw materials, paid labor, equipment, transport and other costs =
	Total Expenditure
	- Total Expenditure minus Total Income from sales and donations =
	Net Cost or Profit
	- Net Cost or Profit divided by Number of Units produced (kilos, packages) =
	Net Cost Per Unit
3) Is the cost of the weaning food distribution to the community worth the benefit to "high risk" children?	- Calculate the number of "high risk" children receiving the weaning supplement who have shown improvement.
	- Divide the net cost of project to community by the number of "high risk" children improved = the Cost For Each Child Improved by the project.
	- What does the community think about the project? Are they willing to continue contributing their time and, perhaps, their funds to the project?