Cover Image
close this bookParticipatory Impact Monitoring - PIM Booklet 2: NGO-Based Impact Monitoring (GTZ, 1996, 38 p.)
close this folder2. NGO-based impact monitoring
View the document2.1 Advantages and obstacles
View the document2.2 Steps in introducing and carrying out NGO-based monitoring of socio-cultural impacts
View the document2.3 Joint reflection workshops
View the document2.4 Facilitating the PIM process

2.3 Joint reflection workshops

Formulation of guiding questions

Workshop Step 1: What has changed?
Workshop Step 2: What have people learned?
Workshop Step 3: What action must be taken?
Workshop Step 4: How can we improve our impact monitoring?
Post-Workshop Step (5): What conclusions can we draw for our work?
Concluding remarks

PIM comprises different autonomous monitoring schemes of various actors (self-help group, NGO, FA or other organizations/groups) in a single project. The individual actors observe the area that interests them most.

Advantages:

- it is not necessary for everyone to gather all the data, and the amount of data stays manageable for each actor;

- the facts are seen from different perspectives;

- more information is available for joint decision-making.

These different perspectives complement each other, and can portray the project reality more completely and realistically than a single actor could. To this end, the observations of those involved must be communicated and discussed regularly. The Joint Reflection Workshops fulfil this important need.

They provide a forum for exchanging and evaluating information. The various actors also hold a mirror up to each other, enabling them to compare the way they see themselves with the way others see them.

The Joint Reflection Workshops are held regularly together with the self-help groups. They will be organized by the NGO to reflect on the progress of the project, though not so much in the sense of the planned "project outputs" as in the sense of impact assessment. These workshops should be arranged in a manner and an environment that the self-help group is familiar with. The participants are the parties involved (for ex.):

- self-help group
- NGO field staff
- NGO office staff


Planned project

Procedure at the joint reflection workshop (4 guiding questions):

Step 1: Compare observations: "What has changed?" Step 2: Analyze socio-cultural impacts: "What have people learned?" Step 3: Take decisions: "What action must be taken?" Step 4: Improve monitoring (if necessary): "How can we improve our impact monitoring?"

After the workshop:

Step 5: Evaluate internally: "What conclusions can we draw for our work?" If necessary take decisions on changes.

The frequency will depend on the degree of familiarity between the NGO and the group, and on the importance of the project. At first it might be every three months, in the long run at least once a year.

Reflection should not be limited to these rare workshops; it is also customary, of course, in routine project work and at meetings. But as described in the introduction, it is useful to have different opportunities for reflection, with different depths and frequencies.

Normally, the day-to-day problems of a project tend to dominate meetings between field workers and the self-help group. However, at these monitoring workshops the participants should explicitly try to look back to the start of their activities, comparing it with where they stand now. In a process involving prolonged activity this is an opportunity to stop for a moment for profound reflection.

It is advisable to have an independent facilitator who has the confidence of all sides.
Formulation of guiding questions

The field staff and the group will have to discuss four guiding questions. These questions must be discussed with the group by adequate methods (i.e. not simple questions and answers).

The results obtained by asking these guiding questions will lead to an analysis of the project context. Emphasis should be given to socio-cultural impacts. The discussion may induce decisions concerning the project's activities, or at least pave the way for decisions which have to be taken by superiors. If necessary, the monitoring system will be revised.

The questions must be open; the NGO should not influence the replies. Although NGO staff and field workers might have their answers to the questions, it is still important that they should first ask the group, by appropriate methods, and only afterwards encourage discussion by introducing their own observations (if necessary).

Workshop Step 1: What has changed?

The monitoring workshops start with (Step 1) the general guiding question "What has charged?".

This is to compare the results of the group-based monitoring system with the the results of the NGO-based monitoring system.


Comparison of observations

The general question "What has charged?" leads to some deeper questions. Some questions refer to change:

What/Who has changed?

(This question is meant to introduce the report on the group's findings, which are a result of the group-based monitoring system.)

What has caused the change?
(the individual members, the NGO, or other factors?)
How has it changed?
How has this change affected you?
What other change(s) has/have occurred as a result?

Clearly, these questions cannot be limited to socio-cultural impact; they include everything that is important to the group. The discussion should nevertheless focus on socio-cultural impact.

Workshop Step 2: What have people learned?

When analyzing the socio-cultural impact, the NGO personnel not only have to refer to the indicators formulated in their monitoring system. They should try to grasp the various learning processes as a whole, and this is perhaps more feasible by asking open questions than by a strict comparison of isolated indicators. NGO members should be aware of the discussion needs of the group and use the opportunity of the joint reflection workshop for an open dialogue with the group.

What have people learned?

- Have the members of the group taken on new responsibilities?
- How far have the group's internal and external relationships changed?
- How far has the internal structure of the group changed?
- What new activities have been started by the group (or by members of the group)?
- What similar activities have other groups (or individuals) started?


Figure

Workshop Step 3: What action must be taken?

The next step is decision-making. The analysis of the findings will be aimed at achieving unequivocal results here. In keeping with the importance and frequency of the Joint Reflection Workshop, the decisions taken here tend to be of a strategic nature. That is to say, they indicate the basic direction and provide a framework for the solution. Operational decisions should be taken subsequently, at other meetings.

What action must be taken?

- What should the members of the group do?
- What should the project team do?
- What should the other people involved do?
- Who else should be brought in?

Workshop Step 4: How tan we improve our impact monitoring?

The last step in the workshop is fairly general: if important issues have previously been neglected the monitoring system must be revised. In such cases it is useful for all those involved to agree at the workshop that these issues should be included in the monitoring system. Alternatively, each organization may take a decision at its own evaluation meeting after the workshop.

Potential improvements in PIM

- Which criteria and indicators should be improved?
- Which criteria and indicators are no longer necessary?
- How could the observation and assessment system be improved?
- How did you feel in our reflection workshop?

The case studies from the PIM field phase showed that many expectations/fears and their indicators which had been identified in the first meetings tended to be of short-term interest. By revising the monitoring system periodically, aspects which are of long-term interest are automatically sifted - and thus relevant indicators for the sustainability of the project come to the fore.

Post-Workshop Step 5: What conclusions can we draw for our work?


Figure

After the monitoring workshop, the NGO should internally evaluate the results of the NGO-based impact monitoring. This reflection should go beyond the management of the actual project: instead, it should refer to more fundamental questions which often relate to the development of your own organization. If changes are necessary, decisions should be taken immediately.

The following questions should be discussed in the NGO after the workshop:

With regard to project management:

- What have we achieved? How have we achieved it? Who has assisted us? What has helped us to achieve this?

- What conclusions can we draw from the comparison between the group's observations and our own?

- Does anything need to be changed in our activities?

With regard to PIM:

- Which socio-cultural indicators need to be taken into account in our own regular monitoring and evaluation?

- How well did the original proposal work?
Has it been modified by the beneficiaries' opinions?

- Should a questionnaire be circulated at regular intervals to document the specific activities of the projects, beneficiary groups and the context?

With regard to your own NGO:

What can we learn from this project that should also be considered in other projects?

- Should we introduce new internal rules to improve our cooperation?

Concluding remarks

For the field staff, it will be important to learn about people's perception of their work. The participation of an external facilitator will further reinforce this. The feedback is useful for the field worker's self-assessment. Joint analysis of observed changes by the people's group and the field staff will increase the appreciation of successes which were previously concealed or rated as "merely subjective" impressions.