|Improving agricultural extension. A reference manual. (1997)|
|Part II - Improving extension programmes and processes|
|Chapter 11 - Evaluating extension programmes|
The Perspective of Learners
We have briefly considered methods for programme evaluation at the macro level. Let us now consider the evaluation of learning at the micro level from the perspective of the two most important stakeholders, the extension educators and the farmers or programme volunteers as learners in specific educational events. When learning is evaluated, there are many questions to answer. Central of course is how learners experience the learning process and what they actually learn (the outcome of learning), their knowledge, attitudes, skills, and aspirations, and their behaviour change. An evaluation also can focus on the extension educator as learner and the content, processes, and resources that are used. Because learning is always a social phenomenon, an evaluation can focus on the social environment, organizational context, and the relevance of language, culture, and sometimes public policy to learning. These underlying cultural assumptions often explain resistance to learning, as well as the way learning either reproduces existing racial, gender, and economic power relationships or challenges these relationships. Not all evaluations include all of these questions. Educators tend to focus on questions that serve their own perspectives. Learners, including farmers, likewise may be interested in questions that serve their perspectives. Often adult learners are eager to reflect critically on their past and present learning contexts in order to overcome socially constructed deterrents to their learning.
When farmers consider evaluation of their own learning, they may ask themselves a broad range of questions. For example, have they, as learners,
· Gained knowledge or problem-solving skills that are useful to them?
· Increased their hopes and aspirations regarding the future?
· Learned how to learn better or gain access to more knowledge?
· Changed their assumptions, habits of mind, priorities?
· Gained confidence in taking leadership and presenting their ideas?
· Increased their commitment to experiment or take direct action?
The Perspective of Extension Educators
When extension educators consider evaluation of learning, they usually want to know how the learners perceive the process of learning, especially how they, as educators, have been helpful to the learning process. Educators ask themselves - and ask the learners to indicate - whether they have,
· Negotiated expectations and objectives?
· Introduced a variety of useful methods and materials?
· Encouraged the use of examples to illustrate concepts or practice?
· Given step-by-step instructions?
· Summarized the material presented?
· Related theory to practice?
· Showed concern about learners as human beings?
· Promoted discussion and learner interaction?
· Encouraged silent learners to participate?
· Used understandable vocabulary?
· Respected racial, ethnic, and gender differences and their unique contributions to learning?
· Appreciated learning handicaps and disabilities?
· Helped learners reflect critically on how they learn?
· Appreciated local knowledge of learners and made use of it?
· Helped learners learn from each other during learning activities?
Asking learners to reflect on educator behaviours also encourages critical reflection on their own learning.
The purpose of listing all of these questions from both the learners' and educators' perspectives is not that one should ask them all, but to stimulate discussion about which are essential items for a specific evaluation effort. The methods that follow can be used to gather evidence of learning through relationships.
Documentation of Local and External Knowledge. A basic principle of adult education is that learning should begin with prior knowledge. How can we appreciate what has been learned if educators do not know what participants already know? For centuries, farmers have passed on their indigenous or local knowledge. Extension educators can assist farmers in documenting this knowledge either in written, photographic, audio, or video forms. This is essential, since evaluation of new learning should be compared with what farmers already know. Also, farmer acknowledgment of the limitations of their local knowledge can form the basis for collaborative inquiry and the linkage of external knowledge with their local knowledge.
Rating Scales and Checklists. Educators can use these in checking the performance of learners. Learners can use these to check the performance of educators. They can be administered in group or field settings and can be easily revised (Worden & Neumaier, 1987). Learners can use them to judge their own performance, current knowledge, or educational expectations. Rating scales and checklists are not very useful in measuring attitudes or consequences of performance.
Feedback Committees. When residential training for extension staff and farmers takes place, participants can elect a feedback committee to provide evaluation observations to the leadership during the event. The feedback committee should be open to any complaints participants may have about the event, ranging from relevance of content, adequacy of facilities, or effectiveness of leadership, to the involvement of learners in discussion or activities. The committee can bring items to the attention of the instructor or to the group as a whole through written or oral form (Apps, 1991).
Group Discussion Assessment. This method can be incorporated into an ongoing group or meeting. It is relatively efficient in terms of costs and time use. The discussion is usually focussed on several open-ended questions, including those listed above. Groups can create their own questions and then make recommendations for changes.
Peer Review Panels. Farmers can become involved in evaluating one another's work through peer review panels. Panels can be taught to use standards and rating scales. Their evaluative judgements can be made with or without identification of reviewers. When peer review panels are used, it is important to establish a positive climate of constructive criticism.
End-of-Event Analysis. This can be done in several ways. The most frequently used is an evaluation form that is administered at the end of workshop sessions. Another way is to have these forms distributed, collected, and summarized by a feedback committee. They can then report these findings and conduct a discussion on the overall strengths of the workshop or training event. After discussing what should not be changed, they can then discuss what specific modifications should be made (Apps, 1991).
Testimonials and Stories. Testimonials and stories can provide subjective records of educational experiences and activities from the perspective of the learners. They are a form of results data and can qualitatively describe the nature and process of educational change. These stories also can be easily understood by others outside the programme as illustrations of types of outcomes and can lead to ideas for future programming. Disadvantages of this method include social desirability bias, nongeneralizability beyond the person giving the testimony, and difficulty in determining what happened as a result of the programme versus other influences on the person. Stories can be either written by the learners or created as a result of an audio-taped interview.
Pretests, Posttests, and Quizzes. In spite of the negative attitudes associated with tests and quizzes, they can be useful for diagnosing learner proficiencies, documenting prior knowledge, projecting learning achievements, and understanding learner attitudes. Repeating the quiz at the end of a learning event can document change that also can demonstrate to learners themselves that they have learned (Jacobs & Chase, 1992).