Cover Image
close this bookParticipatory Impact Monitoring - PIM Booklet 2: NGO-Based Impact Monitoring (GTZ, 1996, 38 p.)
close this folder2. NGO-based impact monitoring
View the document2.1 Advantages and obstacles
View the document2.2 Steps in introducing and carrying out NGO-based monitoring of socio-cultural impacts
View the document2.3 Joint reflection workshops
View the document2.4 Facilitating the PIM process

2.2 Steps in introducing and carrying out NGO-based monitoring of socio-cultural impacts

Preliminary Step: What do we know about the context?
Step by Step Step l: What should be watched?
Step 2: How can it be watched?
Step 3: Who should watch?
Step 4: How can the results be documented?

Which information and for whom? When and how?

Steps 5 to 7: What did we observe?
Why? What should be done?

As mentioned above, the procedure for NGO-based monitoring of socio-cultural impacts is similar to that of group-based impact monitoring, which was described in Booklet 1. The following description is therefore merely a brief outline.

Preliminary Step: What do we know about the context?

Certain essential information about the situational context should be available before PIM is introduced. It is then easier to adapt PIM to specific needs and integrate it in a given context.

Apart from this, if possible, you should as a rule try to use participatory methods for situation analysis which also serve for planning, monitoring and evaluation, such as

- PRA: Participatory Rapid Appraisal
- PAR: Participatory Action Research
- SWOT: Strengths, Weaknesses, Opportunities, Threats
- PALM: Participatory Learning Methods
- GRAAP: See, Reflect, Act (with the help of pictures)

These methods are based on ideas similar to PIM. You should make use of these to permit a realistic assessment of how people see their situation, their problems and needs.

STEP BY STEP

Steps in introducing PIM

1. What should be watched?

expectations and fears of the staff members with regard to socio-cultural changes

2. How can it be watched?

concrete examples of how these changes


can be observed (indicators)

3. Who should watch?

elected staff members who are directly


involved in the respective project

4. How can the results

records, tables, graphs, descriptions charts, be documented?

Steps in carrying out PIM


5. What did we observe?

reports at the beginning of staff meetings

6. Why do we have these

assessment and analysis by the staff results?

7. What should we do?

immediate decision (or preparation for a decision) at the meeting (= adjustment of plan)

Step 1: What should be watched?


Expectations and fears

The project team like the self-help group should make a note of some of the expectations and fears concerning the self-help project. As far as possible, these should relate to the socio-cultural impacts: skills and learning processes in the group. In this context the following questions are helpful:

- Based on your experience, what socio-cultural changes do you expect or fear from this project?

- What socio-cultural impacts resulted from similar projects?

- Which impacts were barely registered by conventional monitoring instruments?

- On the basis of which factors were they clearly recognizable?

- Which socio-cultural indicators should we be aware of in order to manage this project?

In this way, some of the hypotheses on the future development will be identified. It makes sense to discuss these hypotheses and indicators with selected resource persons from the group.

This procedure may lead to a result which is identical to the objectives of the NGO, or to the overall goal, project goal and expected results of the project. If they are fully congruent, so much the better. Whereas in conventional monitoring we rely on the formal information of the planning documents, PIM relies more on an informal assessment by the field workers and NGO staff.

In the case of the store of the Housewives' Committees in Caracoles, Bolivia, staff members had a number of expectations and fears. The prioritized aspects included the following:

EXPECTATIONS

that prices would be lowered and better quality offered than in other shops
that the Housewives' Committees would take over responsibilities
that a higher level of integration of the three Caracoles cooperatives would be achieved
that an opportunity would be created for women to participate in the cooperatives

FEARS /DOUBTS

Should they sell for cash only or give credit?
that they would not be able to recover the credits
that the cooperatives might not pay the store

that the women might not be able to administer the store themselves

Step 2: How can it be watched?

"Monitoring" and "indicators" are often quite abstract terms. Those responsible for keeping a watch should be encouraged to report very simply, on the basis of their experience, how they can tell that people have learned and changed. The procedure for mobilizing the practitioners experience should be similar to group-based impact monitoring. Concrete examples of how the social environment changes can be presented by each team member.

The reasons for these changes can then be analyzed and the most vivid examples chosen for illustrative purposes. In accordance with what was said concerning group-based monitoring, there is no fixed procedure for deriving indicators from these examples. It is not a problem if no measurable or scalable indicators can be found, because in NGO-based impact monitoring descriptive examples can be observed and documented as well.

The expectations and fears to be observed may be chosen in Step 1, or definitively decided upon here in Step 2.


Figure

In view of the work burdens project teams are faced with, it is not necessary initially to introduce more than three to five indicators for a project. The necessary number of indicators, however, will depend on the complexity of the organization and its activities and hence its monitoring system.

PIM is easy to link to a conventional monitoring system. If the NGO has one, it must determine how far the socio-cultural indicators should be integrated into it.

In Caracoles, the NGO staff selected the following indicators for observation:

EXPECTATIONS/FEARS

that prices would be lowered and better quality offered
than in other shops that they would not be able to recover the credits
Should they sell for cash only or give credit?
that the cooperatives might not pay the store
that a higher level of integration of the three Caracoles cooperatives would be achieved
that the Housewives' Committees would take over responsibilities

INDICATORS

the prices of 20 staples in the shop are below prices in other shops nearby
credits given each month do not exceed the cash payments received
the cooperative leaders held monthly meetings to analyze results
the committees were able to check the discount efficiently

INDICATORS

(derived from expectations or fears)
the prices of 20 staples in the shop are below prices in other shops nearby
credits given each month do not exceed the cash payments received
the cooperative leaders held monthly meetings to analyze results
the committees were able to check the discount efficiently

OBSERVATION METHODS

list the prices of each staple monthly for the shop and for 5 other shops nearby separately; reply YES or NO and note down what was observed and what comments were made.
list credits granted and payments received; reply YES or NO and note down what was observed and what comments were made

reply YES or NO and note down what was observed and what comments were made

price lists and prices charged are checked monthly by the committees; reply YES or NO and note down what was observed and what comments were made

Step 3: Who should watch?

The socio-cultural impacts chosen as indicators are frequently those which have already been observed by the NGO personnel. It is therefore best for the field workers and other NGO staff members to observe the selected changes themselves. One or two people should be chosen to be responsible for observation.

However, it is also useful to confirm one's own views by cooperating with other people or organizations who know the project environment: a teacher, a priest, or staff members of other NGOs, or any insider concerning the group's internal structures. Whether this is appropriate will depend very much on the specific conditions.

While it is not the purpose of NGO-based PIM to employ members of self-help groups as observers, they need not be excluded. It is first and foremost the field staffs' view which is of interest here. The views of the self-help group and NGO will be compared later in the joint reflection workshops this is part of the learning process!


Figure

Step 4: How can the results be documented?
As explained for group-based impact monitoring, a record must be kept of the impact observed. If it is done in the same way as for conventional monitoring, it will be recorded in a kind of logbook.

In the Caracoles project, monitoring forms were developed for noting down the prices observed in the various shops each month. For other indicators, there are questions which have to be answered with yes or no, with a blank space for remarks and comments.

Graphs and charts are also helpful for visualizing quantitative indicators. Indicator no. 2, about credits granted and payments received, was drawn as a bar chart.


Figure

Which information and for whom? When and how?

The NGO has to set priorities in information flow in accordance with the decision-making structures. This means that not all information has to flow to the NGO director or even to the funding agency, only summaries from time to time, or to report outstanding successes or conspicuous failures.

These rules for information flow should be worked out jointly by the entire NGO team. They should also decide what kind of monitoring information is regarded as sensitive or confidential, and establish clear rules as to how it should be handled: who must be excluded from the information flow?

In addition, a decision must be taken on how information should be fed back to the group. In Caracoles, the women were informed monthly. A minimum would be reports at the joint reflection workshop.


Figure

Important: All those involved must accept that although each organization wants to know what the others are doing behind the scenes, every actor is entitled to confidential treatment of his inside information. Not everything has to be analyzed: some secrets are best left under wraps!

Steps 5 to 7: What did we observe? Why? What should we do?

The steps to answer these questions are now basically similar to the steps described in Booklet I (Group-based Impact Monitoring).

Also, an NGO might already have a (conventional) monitoring system. Socio-cultural impacts might be monitored in a similar way. PIM is compatible with general management rules.

As mentioned in Section 1.2, the depths and periodicities of reflection and decision-making are different in every organization: you should match your PIM rules to your project and organization structure.


Figure