Evaluation measures the effects of work planned and carried out during the extension process. Evaluation helps extensionists and farmers answer questions like:
Which planned activities did we do? Not do?
How much did this cost?
What is the result of this work?
What are the benefits/ill-effects of this work?
What did we learn?
Are we closer to our goal?
Evaluation, like planning and work, must be done with community counterparts. In fact, evaluation is the most neglected part of the extension process. Two-way communication can only occur between scientists and farm families when field and community-level evaluations of extension work occur.
Evaluation is the closing link in effective two-way communication. Honest and open evaluation of community work gives small-scale farm families a real voice in the process of change and growth.
By using the planning tools illustrated earlier in this chapter as checklists, the detailed quantitative results of extension work can be measured. Progress toward material goals can thereby be assessed. There are "higher" goals than the material ones, however, which are harder to describe and harder to measure progress toward. These other goals include:
• increased two-way communication
• working onself out of a job, or increased competent independence of farmers
• ecological soundness of work
• cultural appropriateness of work
• moral or spiritual goals
• personal satisfaction or fun
• relationships with co-workers, farm families, host community
All of these goals have some relation to the overall goal of agricultural extension as described in this manual: helping small-scale farm families participate creatively in growth and change.
Expert advice may be needed to evaluate progress toward some of these "higher" goals, ecological soundness, for example. Very few village-level extensionists have the rural sociology or cultural anthropology resources to formally evaluate the cultural effects of work, but it is important to care. Philosophical, spiritual and relational issues often arise when extensionists and farm families are engaged in a process of change. Indeed, development is fraught with them. These issues cannot be evaluated in a normal sense, but they must animate the extensionist to stop and consider, to pause and reflect, and to open herself to change as it occurs. The most thrilling and challenging aspect of extension work consists in being at the center of change. Being at the center brings an enormous responsibility to others into focus. Evaluation on all levels, careful consideration in all that is done, is responsibility and care in action.
There are several types of evaluation which can be done:
Everyday observation which yields a subjective impression that can helpfully guide work. Such information is collected from casual conversation farm and home visits, meetings and individual discussions.
Informal studies which involve review and analysis of information obtained from records, reports, checklists, questionnaires, census data, etc. Done periodically, these studies should form a fairly objective basis for judging extension work. These may also include periodic reflection on the "higer" goals mentioned above.
Formal studies are more structured. They are often planned and carried out according to scientific rules to provide more objective information on the achievement of objectives or the effectiveness of methods used. These studies may include formal interviews, questionnaires, data collection and analysis, etc. Cost-benefit analysis is a good example of this (see Appendix), as are aspects of "Agricultural Survey" (see Chapter Two).
Certain basic principles underlie effective evaluation:
• Evaluation of extension work should be well planned and clearly defined as to what is to be evaluated in a program.
• Extension personnel and community people should take part in evaluation. Self-appraisal helps those who carry out a program and can be usefully combined with appraisal by an outside person.
• Everyday evlauation should be continuous and integrated with the program development process from its planning stage to the end.
• Reliable and effective devices should be used, and a representative sample chosen according to means available.
• Evaluation should be more concerned with the achievement of behavioral changed than with the number of participants, meetings, hours, items prepared, etc.
• Careful analysis and interpretation of findings should be considered when an evaluation study is being planned.
Since there are so many aspects to every extension activity, one must decide which are important to evaluate. Some aspects of extension activity are:
By deciding which to evaluate, it may be easier to determine which evaluation method to use.
It is very important in evaluation to make conservative judgments and conclusions. No matter how scientific the device used or how objective the criteria for measuring, each evaluation made by an extensionist is only one person's (or several persons') point of view. There are no absolutes in the process of change, only probabilities. Critical awareness and openness to various points of view is essential to the proper interpretation of evaluation results.
Most important of all is the awareness that community members and farmers are not to be victimized by evaluation results. It is often the case that extension project failure is attributed to farmer ignorance or non-cooperation, but it is rarely true. As it is an observed tendency in evaluation of extension work, extensionists are cautioned to be wary of such conclusions. In actual fact, assigning the cause of any result to one specific factor is often misleading or short-sighted.
By spending time and energy measuring the effect of extension work at various levels, extensionsists close the final link in the chain of two way communication. They bring farm families into direct contact with scientists and other agents of change. As the catalysts at the center of the process of agricultural change, extensionists are charged with an immense responsibility to others which they can translate into action by means of sensitive evaluation.
Mike had helped a group of farmers develop an inland valley swamp for rice production in the dry season. They had cleared the land, constructed the dikes, bunds and paddies, planted the rice, and performed all the production tasks. Now it was time to evaluate the results of all this work.
Mike was filled with uncertainty as he rode into the village near the rice farm. He had been away for several months working elsewhere. He was very eager to see what had happened since his last visit. As he entered the village, he saw a familiar face. It was Mami, the master farmer's wife. She saw him, but ducked inside quickly. What did that mean? Mike stopped his motorcycle and parked it. All of a sudden people came from each house smiling and running. Most of the farmers and their families surrounded him and welcomed him joyously. Mike was overwhelmed and confused. One of the farmers said, "You've given us so much! We are so grateful!" They all walked together down to the village farm.
Mike wondered if he would be able to do the careful yield sample analysis on each farmer's acreage that he had hoped to do before harvest. He wondered if this was the time to do any of the specific evaluations he had listed and planned before he came... Everyone stopped at the edge of the farm with Mike and gazed out. There was a sea of ripening rice blowing in the wind, surrounded by fruiting vegetable gardens along the edge. Mike was very happy.
After a while, the crowd dispersed. Families went home to cook, farmers tended their plots, and Mike was able to inspect details with the master farmer. Mike began to ask a specific set of questions about the growing season. He wrote down the answers. Mike then explained what a yield sample was, why it was important to do, and how to do it. He and the master farmer tried one and agreed to do it for each farmer during the next visit. Then Mike looked at the dam, dikes, irrigation channels and gateways. He troubleshot sample plots for diseases, insects, iron toxicity, drainage or irrigation problems, uneven growth, etc. He recorded each of these things in his notebook.
He asked all the farmers to gather briefly under the mango tree where they had rested so often during land preparation and production work together. Mike asked the farmers a series of prepared questions which were designed to help the farmers review what they had done. He especially focused his questions on the concrete results of their own hard work, on how each land preparation or production task resulted in something specific that helped the rice and gardens grow. They engaged in a review of the season's events in this way, where Mike learned of specific problems encountered during the season and the farmers could see cause and effect at work.
By the time evening fell and the rice meal was ready to eat, Mike and the farmers had talked for a long time about what had occurred. Mike left with a very good feeling as to the production and other physical results of the community's project. He helped them see their ability to solve ag problems more clearly.
He felt the very warm relationship they shared. He was disturbed however, by the farmers' attribution of success to him personally. He also wondered about the ecological effects of their irrigation system and double-cropping methods. He saw that the farmers had ambitiously cleared the footpath to the farm so trucks could take out the rice. Unfortunately, this also made it too dangerous for the old woman and blind man who lived near the farm to work as they had before. They had to move. The road also brought the chiefdom tax collector, and the health inspector to the village for their respective duties. The tiny village of Bambaia was transformed now. Mike felt very small going home.
1. Use the planning tools in each subchapter of the manual as checklists with which to evaluate work progress and the accomplishment of specific work tasks.
2. Factors to consider in extension evaluation:
a. People involved:
-project or community leaders
- outsider resources
- community people
Does everyone know their role, the purpose of evaluation, etc. Can everyone do what is required?
-Is it too early to tell if a project or activity works?
-Is this the best time to evaluate?
c. Evaluating tools and methods:
-Are they appropriate and applicable?
d. Finances and other costs:
-Is this the least costly way to evaluate effectively?
3. Questions related to each aspect of work to be evaluated:
Aspect of extension activity
What is the situation before and after the activity?
How was this activity planned?
Who did what, when?
How was this work done?
What are the concrete quantitative results of this work?
What process did everyone engage in?
What might be the long-term effects?
Did the work meet stated goals or objectives?
Was the specific problem solved?
What new problems arose?
4. Planning and Conducting an Evaluation Study
These steps are essential in planning an evaluation study: (1) select a problem demanding investigation; (2) clearly formulate what is to be measured; (3) determine the kind of information needed; (4) decide how to collect the information; and (5) plan how the information is to be used.
The following points will serve as guidelines when planning and conducting evaluation of an extension program.
A. Determine what personnel and financial resources are available and needed for making an investigation
-what cooperation is needed from outside resources as to personnel and money?
-what time is required for carrying out a study?
B. Select and define a part of a program to study
-what objectives or content of the extension program are to be evaluated?
-which phase of this program will be evaluated, i.e. program action, teaching methods, results, etc.
-will an evaluation of a certain program or aspect of a program be useful for the future?
-will it be necessary to collect data for the evaluation or is some information already available from other sources?
C. Define and clarify the objectives of the study
-What evidence is needed to determine that the educational program is reaching its goals in terms of (a) number of accomplishments, or (b) changed behaviour of the people? Which are the most important indicators or changed behaviour?
D. Decide on now to collect information and what devices are needed
-what kinds of devices are to be used questionnaires, observation forms, etc.?
-if a questionnaire is to be used, what questions should be asked and how should they be phrased?
-set up record forms and prepare instructions for using them,
-pre-test devices and revise if necessary.
E. Plan tabulation
-Type of tabulation methods and tables needed to discover relationships
-Persons and equipment needed for the tabulation.
F. Determine samples
-Define population to be sampled.
-Determine what sample is needed and feasible.
-Prepare instructions for sampling.
G. Prepare for the collection of information.
-What persons are available to collect information?
-Select interviewers or collectors of information.
-Determine what training they need in interviewing, recording, etc.?
H. Collect information in the field
-Provide necessary supervision.
I. Edit and tabulate data collected.
J. Interpret facts and prepare a report
-What are the important findings of the study?
-Do different reports need to be written, e.g. for professional and for lay people?
K. Plan for use of the findings
-What implications might the findings have for future work?
5. Another format for the questions you may ask to evaluate a project:
(US AID Kenya Ag Ext. Manual)
Evaluating the Project
Don't stop yet-evaluate! Planning never ends, so, each time a project or step of the program is completed, the participants should look back over what has been done to be sure that things are going as they should. This is called evaluation and is an on-going, continuous process-just like planning. You must evaluate past efforts to plan for changes.
Develop a means for evaluation when defining the goal and writing up a Plan for Action. Keep in mind your community survey and any responses from questionnaires and statistics you might have collected as possible sources of information for evaluation
Following each step or activity, ask questions such as:
• How well did we do?
• Did the plans work?
• Why did we succeed? or
• Why did we fail?
• What should we be doing now?
• What do we do next?
• If we made mistakes, can we keep from making them again?
Encourage the community members to begin to evaluate the project shortly after its initiation. Are people using the latrines that have been installed? Are they keeping up their vegetable gardens and eating the harvest? Are the children really going to school? Did the group for whom you intended your activities come?
After each phase of the project is over, you must follow up to determine how successful it has been. At the end, ask yourself all of these questions again. Did you get the job done? What can be done to make your efforts more successful?
Possible kinds of measurements you might use to evaluate your project, if planned from the beginning, are:
1. Quality or amount
a. How many persons were reached?
b. How many posters, pamphlets, home visits were made?
2. Quality - What do the people think?
a. the leaders?
b. the participants, villagers?
c. other health workers?
d. the pupils?
3. Changes in knowledge shown by:
a. discussion among farmers
b. answers to questions posed by extension agent
4. Changes in attitude
a. Community support for the program.
b. Requests for further cooperation by government agencies.
c. Less opposition by groups in the village who had previously been against the project.
d. Public opinion poll
5. Changes in behavior, such as:
a. Increase in visits to the ag station or worker
b. Improved habits and conditions noted on farms
c. Increase in production
d. Increase in the sale of milk, meat, vegetables or other good foods
e. Increased need for post-harvest support
f. Increased interest in marketing
6. Changes in community life:
a. Improved nutrition
b. Food surpluses
c. Signs of economic prosperity (new houses, material goods)
d. Improvement in health as shown in individual cases
In the case of evaluating extension work, you will find it difficult to measure the results. The mere number of lessons or demonstrations and the ability of the people to repeat them are surely not the only measure. Behavior change is the goal, yet these changes are not easily evaluated immediately since they may occur slowly over a long period of time.
As always, throughout your work with a community, it will be necessary to record your observations. This is a form of written record which you've already done during your community investigation. You should discuss the importance of record keeping with counterparts.
Evaluating the progress of complex activities (such as extension) is never simple, but it can be made easier by clearly defining the project's objectives early and relating your evaluation plan directly to those objectives. With careful planning, evaluative data will help to assure that the project is better managed, and that those who support the work, and particularly members of the community, will feel confident in the progress being made.