Cover Image
close this bookParticipatory Impact Monitoring - PIM Booklet 4: The Concept of Participatory Impact Monitoring (GTZ, 1996, 36 p.)
View the document(introduction...)
View the documentAcknowledgments
View the document1. Problem
View the document2. Objectives of PIM
Open this folder and view contents3. Situation analysis
Open this folder and view contents4. The Structure of PIM

(introduction...)

Eberhard Gohl / Dorsi German

Acknowledgments

FAKT

FAKT the Association for Appropriate Technologies is a church-based non-prot'it consultancy founded in 1986 in Germany. FAKT is engaged in technology transfer and looks for socially and environmentally compatible solutions together with partners from the South. FAKT works in interdisciplinary teams with engineers. economists and social scientists and is specialized in the fields of food technology, water supply supply sanitation. energy supply. hospital technology. vocational training small business promotion. organizational development and participatory project management. FAKT services include planning and accompanying technical projects. evaluations and studies, training programmes. seminars and Question-and-Answer-Service.

GTZ

Deutsche Gesellschaft fhnische Zusammenarbeit (GTZ) GmbH

The government-owned GTZ operates in the field of technical cooperation. 2200 German experts are working together with partners from about 100 countries of Africa. Asia and Latin America in projects covering practically every sector of agriculture. forestry. economic development. social services and institutional and material infrastructure The GTZ is commissioned to do this work both by the Government of the Federal Republic of Germany and by other government ant semi-government authorities.

GATE

GATE (German Appropriate Technology Exchange), a programme of' the Deutsche Gesellschaft fhnische Zusammenarbeit (GTZ) GmbH. acts as a centre for the dissemination and promotion of appropriate technologies for developing countries.

The Information and Advisory Service on Appropriate Technology (ISAT), was established as a section of GATE. in 1991. It has at its disposal the full range of services the GTZ offers, and is financed through the German Ministry for Economic Cooperation and Development. ISAT specializes in a large variety of appropriate technologies for developing countries, whereby the technologies that ISAT focusses on principally include those which

help satisfy basic needs'
make efficient and environmentaily sound use of locally available resources.
mobilize existing skills and promote self-help
extend user's scope for action and promote independent action.

ISAT offers its services ¿question-and-answer service. various types of publications and titans) to German development co-operation as well as organizations and individuals in developing countries. Requests for literature and documentation on specific areas of appropriate technology can be addressed to ISAT.

Deutches Zentrum fwicklungstechnologien GATE in: Deutsche Gesellschaft fhnische Zusammenabeirt (GTZ) GmbH
P0. Box :5 1 80
D-65726 Eschborn
Federal Republic of Germany
Telephon: (061 96) 79-0
Telex; 41 523-0 gtz d
Fax (061 96)79 73 52

A Publication of Deutsches Zentrum fwicklungstechnologien - GATE
A Division of the Deutsche Gesellschaft fhnische Zusammenarbeit
(GTZ) GmbH

The authors:

Eberhard Gohl, economist and sociologist, spent a few years in Turkey, Peru and Bolivia. For eight years, he worked mainly with FAKT, DSE-ZEL and GTZ as a consultant for project management and organisation development. At present, he works as Controller in the German Protestant Churches' funding NGO "Bread for the World".

Dorsi Germann, sociologist and graphic artist, spent four years working in a community development project in Senegal. For the last fourteen years, she has been a consultant on adult education, appropriate technologies, technics of communication and visualization, project management, monitoring and evaluation, organizational development and participatory methods in Africa, Asia and Latin America, mainly working for GTZ and FAKT.

Die Deutsche Bibliothek - CIP-Einheitsaufnahme

Participatory impact monitoring: a publication of Deutsches
Zentrum fwicklungstechnologien - GATE, a division of
the Deutsche Gesellschaft fhnische Zusammenarbeit
(GTZ) GmbH. - Braunschweig; Wiesbaden: Vieweg.

ISBN 3-528-02086-5
NE: Deutsches Zentrum fwicklungstechnologien <Eschborn>

Booklet 4. The concept of participatory impact monitoring
/ Eberhard Gohl/Dorsi Germann. - 1996
NE: Gohl, Eberhard

The author´s opinion does not necessarily represent the view of the publisher.

All rights reserved
Deutsche Gesellschaft fhnische Zusammenarbeit (GTZ)
GmbH, Eschborn 1996
Published by Friedr. Vieweg & Sohn Verlagsgesellschaft mbH,
Braunschweig/Wiesbaden

Vieweg is a subsidiary company of the Bertelsmann Professional Information.

Printed in Germany by Lengericher
Handelsdruckerei, Lengerich
ISBN 3-528-02086-5


Problem

1. Problem

"What's the point of development projects?"

We know very little about the impact of development projects, and even less about the social and cultural effects than about the frequently modest technical and economic successes.

Yet, critical inquiries regarding this aspect are arising more and more often. The funding agencies have to increasingly demonstrate successes to their financial backers, who are more often than not small donors or tax payers/voters. This pressure to be successful is passed on to the development organizations (NGO) in the South, which means that their staff have to increasingly prove the impact of their work. And the self-help groups or grassroots organizations for their part? albeit for the time being only those with greater awareness, start asking whether the NGO really needs so much money for their promotion or whether it would not make more sense for the funding agencies to support the self-help groups directly.

The funding agencies are adjusting to the new requirements. More and more often, impact analyses are drawn up and published. Yet, the limits of such analyses quickly become evident:

- The studies are conducted ex-post and barely affect the future of the project reviewed.

- The impact analyses are prepared by external persons and only include the standpoints of the relevant NGO and self-help groups to a limited extent.

- The studies are commissioned by a funding agency; therefore, at the most, they influence only future decisions at this level. They do not trigger any learning processes in the NGO or self-help groups who are actually responsible for implementing the project.

- The information that is published serves as justification and is not intended to promote learning. One is afraid of the public criticism that may arise if problems and errors become known.

The point is to find new solutions. To this end, a new concept called Participatory Impact Monitoring (PIM) was developed.

2. Objectives of PIM

The following task was defined for the PIM team at the beginning of the project:

The objective of PIM is to improve the realization of projects by

- orienting the project along the socio-cultural impact;

- promoting autonomous activities of the people;

- improving the flexibility of and interaction between the development organization/NGO and self-help group.

GATE (German Appropriate Technology Exchange, the unit within GTZ responsible for adapted technology) did not commission the study in connection with large technical projects, but aimed at small projects conducted within the framework of self-help promotion and appropriate technology.

Such projects are primarily considered learning situations that are designed to trigger learning processes. Thus it becomes evident that if one wants a regulatory instrument for development projects that is based on the joint perception of impacts, one needs a concept which is not limited to being a regulatory instrument of the relief organization but can also be generally used as a management tool by all NGO and self-help groups.

However, PIM should be compatible with the goal-oriented project planning (ZOPP), the official planning procedure of GTZ, because many of the abovementioned projects are part of larger programs that were planned with ZOPP.


Situation analysis

3.1 Management concepts for development projects

ZOPP is a self-contained, logical planning procedure from which a comprehensive project management system can be derived. ZOPP is a further development of the Logical Framework, which was developed for US AID. It seems suitable for all types of planning.

When we plan the construction of a plane, all the marginal conditions are usually fixed and known: we can plan the course of proceeding quite precisely and only have to change our plans if something unforeseeable occurs, for example if a certain material does not meet our expectations.

A game of chess, a football game, or a development project cannot be planned entirely because the different participants react to one another. The marginal conditions are known to a limited extent only, and we expect to encounter new situations continuously. We cannot plan the course of events exactly unless we can exert great influence on the other actors. Instead we have to think over our manner of proceeding one step at a time.

It seems that ZOPP is a planning procedure more suitable for the tasks mentioned first, i.e. solving the problem on the basis of defined targets and conditions. This may be sensible in development policy on a macro level where many marginal conditions change relatively little and only infrequently.

Naturally, the contrasting representation given above is a simplification. In reality, the data used in connection with ZOPP have to be questioned frequently, and the management can be and is improved by the frequency and depth of the monitoring. But the premise remains that the framework is relatively constant and that we can foresee and influence it.

Moreover, we have to deal with other problems in development projects, since they usually involve several actors who observe each other carefully and also react spontaneously to one another. For example, bustling activity and changes can be triggered in a self-help group by the mere fact that a funding agency has become aware of its existence. A situation analysis as such stimulates reflection and wishes. A decision to support a group can lead to power struggles. The claim that development aid plays a subsidiary role is not true, because the overall conception that a self-help group has of itself changes when a funding agency enters the scene: the self-help group has to take a position with respect to the funding agency under changed conditions.


Planing

In self-help promotion, perhaps in development policy in general, we need new planning and management concepts that are based on the premise that everything is in a state of flux. We need such concepts at all levels: for the funding agencies as well as for the development and self-help groups. What is needed are concepts that take into consideration the continuous changes in the positions of all actors, i.e. concepts that are systematic and process oriented.

3.2 The project reality in self-help promotion



From the expected continuity to the expected change
From insisting on agreements to promoting the skills to solve problems
From checks to acceptance of errors
From evaluation to monitoring
From the factual level to the social level
From examining objectively verifiable impacts to perceiving the subjectively important changes
From determining exact information to perceiving trends
From formal to informal structures
From project consensus to interaction between autonomous project actors

In the colourful diversity of this world there are no magic recipes for correct project management. New concepts are continuously being tested, adapted and combined with other concepts in all areas of development cooperation. As in other fields, the management paradigms are always changing. A few trends which are already being implemented in self-help promotion and many other development projects will be discussed below.

From the expected continuity to the expected change

The only thing that is permanent is continuous change


Deviating from a laid-out plan

Deviating from a laid-out plan is normal. This is known by anyone who is familiar with the realities of projects at the grassroots level. Yet many funding agencies are attuned to continuity. Changes disrupt administrative processes and give rise to additional work. Development projects are supposed to change many things, but not themselves if possible.

Plans have to be revised not only because surrounding factors change, but also because the individuals involved in the project develop further- and that is, after all, an essential goal of development policy!

From insisting on agreements to promoting the skills to solve problems

Not only the project activities, but also the agreements and even the organizations themselves, including funding agencies, have to have room to develop. This calls for continuous adaptation and development - not only of the methods but also of ways of thinking and attitudes:

Man: Not merely carrying out, but analyzing and solving problems independently.

Power: Not forced, but voluntary cooperation.

Success criteria: Not purely technical and economic, but in harmony with socio-cultural aspects.

If we want to promote the ability to solve problems, i.e. the ability for creating innovations, and the efficiency of the people and organizations of the South, we cannot persist in adhering to old plans and agreements, but have to acknowledge changes, even if we do not like them. We must neither demand obedience nor fan the flames of rebellion; rather, we have to help the people concerned take action, so that they become actively involved and can identify with the results.

This cannot be achieved through administrative or formal measures alone. It also requires apositive, partner-oriented attitude that motivates responsible action. Then the quality of the project is not defined by the degree to which the plans were fulfilled, i.e. the largest possible congruence between planned and actual, but by the results that were actually attained.

From checks to acceptance of errors is the enemy of creativity

It is not without reason that many NGO in the South shrink from external evaluations: their experience with know-all evaluators has not been good. Nevertheless, evaluations always have two goals: control and reflection. Project evaluations should - especially if we view projects as learning processes - emphasize the second goal.

In the reality of projects there is hardly ever an unequivocal right or wrong. The future is so uncertain, the co-actors so flexible, reality so complex, perception so limited, and so on that we can only make imperfect decisions - indeed our decisions as individuals are even more imperfect than those taken in agreement with other actors. Yet, it is normal to make decisions that later turn out to have been wrong.

Errors are an important part of a learning process. It is no mistake to make errors in a learning process, if one is willing to learn from one's mistakes.' But this calls for a positive learning environment which permits self-realization of the individual and the group within the context of personality development, team and organization development.


Objectives

From evaluation to monitoring

He who comes too late is punished by life

A classic differentiation between monitoring and evaluating states that monitoring is a process which systematically and critically observes the events connected to a project in order to control the activities and to adapt them to the conditions. Monitoring does not examine the planning. In contrast to this, evaluation involves an extensive analysis of the course of the project with the aim of adapting the strategy and planning to the conditions, i.e. evaluation does examine the planning.

But this distinction is ideal-typical. In practice the borders cannot be clearly demarcated: there are different actors and decision-making levels in an organization or group, and the periodicity and depth of reflection - irrespective of monitoring or evaluation - will extend through several levels.

Many development organizations/NGO link their evaluation of the impact of a project to the infrequent external evaluations carried out by funding agencies. Others conduct evaluations more frequently on their own, for example once a year. If, however, we want to become aware of and promote learning processes by means of such regular reflective measures, we need a more regular rhythm.

From the factual level to the social level

Development projects are implemented with people

The logical reasoning of projects as a rule is that a project needs a concrete problem to be called into life. In other words, the project would then be defined as the solution to the problem, in which certain inputs would logically lead to determined outputs. In this way, projects are reduced to a technical economic entity, and measurable, tangible success stands in the foreground.

The socio-cultural component, however, can only be found in superior goals and marginal conditions, if it still fits into the prevalent logical system. And yet, this socio-cultural level is decisive: learning processes, changing attitudes and behavior- that is the development process we are striving towards, in the North and in the South.

From examining objectively verifiable impacts to perceiving the subjectively important changes

The subjectively important changes have an impact.

Project applications often contain complicated, long-term and general objectives - the higher in the financing hierarchy, the more abstract they are. For the participants at the grass-roots level these general goals become less and less comprehensible or manageable. In that form they are far too abstract and not suitable as attainable objectives.

It is generally known that the further a problem lies in the far future and the less it affects a person's daily reality, the less the awareness of a problem. The more direct the effect of a problem, the more we are personally concerned about such problems. In this case, we are much more sensitive in our perception of signals of change and perceive the changes much earlier, albeit intuitively without being able to give logical reasons or proof.

The observations of all actors are defined by subjective perception and are often coincidental. Goal-oriented action, however, requires continuous, systematic and comprehensible detection of signals of change. Which changes are relevant with respect to activities is perceived most frequently and explicitly by those actors who are most affected by such changes. For us subjective evaluation is an important selection instrument, since it is easier and more effective than the external application of objectively verifiable indicators, which function mechanistically.

If we rely on the sensitiveness of the group, we will obtain finer and earlier reactions as well as adaptations to changes that- contrary to administrative decisions - give rise to early countermeasures.


Information

From determining exact information to perceiving trends

Relying on intuition saves time.

Only phenomena that are based on absolutely precise information and lead to definite conclusions by means of Aristotelian logic are considered to be scientifically proven.

Yet, in actual fact, our world is so complex that we will never be able to definitely determine all factors and their interaction in our dynamic environment. In our daily lives we have to filter out the information that is important to us. Formal criteria help us make our decisions, but we are even more guided by our experience and intuition. We have to use our limited perception to recognize patterns and interpret them. In this way, we can obtain quite meaningful results and decisions with little effort.

As far as development policy is concerned, we would be unable to act if we were to conduct projects primarily on the basis of exact information and scientific conclusions. If we wanted to do justice to the complexity of the circumstances, we would have to collect piles of data - a practical limitation of many formalized systemic approaches. Then, however, the impression would be given that the protagonists, who are unskilled in formal logic, were not capable of helping themselves.

Monitoring systems that lay claim to a high scientific quality and feature a complicated language act as deterrents. Users fear that their observations will be unscientific and incorrect - or, to put it more succinctly, they are simply afraid of being incompetent. Thus, they hamper such systems.

In projects where self-help plays a significant part, we need simple monitoring systems and indicators that are easy to manage. Even though such indicators may not always be exact, they will illustrate essential trends quickly and plausibly. With such monitoring systems we will be able to manage self help projects effectively.

From formal to the informal structures

Not the formal, but the informal structures and communication are important

We have to revise our thinking considerably both with respect to processes and changes as well as structures. If we know the project reality at the grassroots level, we know that informal communication is normal and that the informal structures are the decisive ones.


Figure

This point is best illustrated by comparing an iceberg to the highly formalized structures of the established organizations: although about 6/7 of the volume is hidden from view under the surface, it is of substantial importance for the colossus. The visible 1/7 is a structure that developed over a long period of time, i.e. an expression of the internal arrangement and the history of the organization. In young organizations, however, the formal structure frequently is merely put on, and it does not fit the internal processes. Thus, it is dysfunctional and only of slight significance.

Naturally project structures have to be formalized as agreements between different actors. But we don't need palaces, we need tents. We don't need organizations that are characterized by mechanistic-bureaucratic structures, because with time these lose their ability to perceive and to react properly to changes in the environment. We need flexible and innovative organizations, because they will be more efficient in connection with dynamic environments.


Different environments

From project consensus to interaction between autonomous project actors

Every actor has "his" project.

The self-help group has "its" project; the staff of a development organization/NGO have "their" project; the desk officer of a funding agency has "his" project; the consultant has "her" project. And quite frequently they all believe it is one and the same project. Within the framework of ZOPP, "the" project is planned on the basis of agreement, i.e. there is "the" project team, and "the project" sometimes has many employees and vehicles at its disposal.

Actually a project is a bundle of measures designed to solve a specific problem. In the field of development cooperation, the word ,,project" has a more extensive meaning - frequently it is characterized by the illusion that there is a common feature. The mutual project often is pure fiction - at the most there is a point of overlapping, a smallest common denominator, between the individual viewpoints and projects of the different actors.

This is so because each actor defines the problem he is confronted with differently, perceives it differently, and, thus, also defines his bundle of solutions differently. Each of these projects is different, and the cooperation between the actors is limited to the area where these different definitions overlap. Furthermore, each of these projects is planned, conducted, monitored, and evaluated by the respective actors - albeit in coordination with the others - and the responsibility is borne by each actor or by each group alone.

(introduction...)

The rules for applying PIM were already explained in Booklet I (for self help groups) and Booklet 2 (for development organizations/NGO), practical examples were provided in Booklet 3. Therefore, the mode of operation of PIM will not be described in this booklet. Instead we want to point out certain characteristics from theoretical and practical standpoints.

4.1 Combining the different approaches to solutions

Interaction of the project actors
Valuation of the informal structures
Perceiving trends
Emphasizing the subjectively important changes
Determining the socio-cultural impact
Periodical reflection as monitoring
Accompanying learning processes

PIM tries to take up the approaches described above and to combine them in a new way. The following features are of particular importance in connection with PIM:

Interaction between the project actors

PIM involves several autonomous strings: the self-help groups observe what is important for the members of their group, it collects data and makes decisions. The development organization/NGO does the same on its own at first. The principle can be expanded at will: also the funding agency could proceed in this way and other groups. A federation of cooperatives could have separate monitoring systems for federation leaders and for the employed experts. The self-help group can differentiate by social groups or partial projects.

Each group of actors covers its areas of interest; thus, it collects only a limited amount of data. If the groups were to regularly exchange their perceptions and interpretations, much broader coverage could be attained. The information would not be summed up mechanistically, but would form an overall picture due to the interaction between the project actors. A systemic mode of observation is not achieved by accumulating data, but only through cooperation between the actors.


Objectives

This concept of autonomous actors, whose areas of interest overlap at certain points, is a brief description of the meaning of participation within the framework of PIM. Participation is not understood to mean participating in a project managed by an outside party; rather participation is the responsible execution of one's own project in cooperation with other actors.

The project data and autonomous monitoring systems of the individual actors are discussed regularly and continuously at Joint Reflection Workshops. This enables the observations of all those involved to be utilized and considered with regard to the joint project goals.

The more congruent the aims and expectations of the individual actors are, and the more they are in agreement with the overall project goals, the more smoothly and efficiently PIM will function.

If the aims and perceptions of the individual actors differ, or if they deviate from the overall project goals, PIM acts as an early warning system, because discussion and the process of reaching agreement are disrupted.

PIM can have a corrective function if the actors are sensitive to such signals and are in a favour of change. In the regular Joint Reflection Workshops, aims and strategies can be continuously reviewed and jointly redefined.

If there is little willingness to accept change, conflict and division may occur. In such cases, concepts from the field of organizational development and conflict management often also have to be applied.


Graphic

Valuation of the informal structures

PIM is a monitoring system that is not based on planning, but works independently of it. That is provoking to many people active in the field, because there seems to be no orientation. But in actual fact PIM is goal-oriented.

Since planning in practice involves so many errors and, in addition, often is nothing but a formal compromise between the different project actors, another mode of action had to be found. The objectives of the people concerned were deliberately separated from the formal context of planning by querying the individual objectives of the people within the framework of the project: "What are your expectations (fears)?" or "What should (not) happen?" In this way, an attempt was made to determine the motivation (or motivation killers) relevant to taking action. If the formal planning was good, the informally expressed expectations and fears will not show any deviations.

The factual realization of the goals is accompanied by both hindering and supportive social processes. However, these often escape. In connection with PIM, they are included in the monitoring and, hence, in the management system, so that not only formal PIM activities but also informal group processes can be controlled, that is to say controlled not from outside but from the group itself. In the field phase of PIM this was an important aspect for all groups. Self-observation also led to restructuring.

The significance of the informal structures is also enhanced by the existence of the different strings where autonomous monitoring occurs: the monitoring is not conducted for outside parties who have unclear demands, but for the use of the group carrying out the monitoring. The observation criteria, indicators, and reporting only have to be suitable for the respective actors, so that they can make decisions.

Permitting or even enhancing the status of the informal aspects is not only helpful for the self-help groups, it also helps the staff of NGO. Even the funding agencies may leave the level of formal logic and agreements and monitor such other expectations/fears as leadership structure, participation of women, or social climate in the project environment.


Perceiving trends

Perceiving trends

The more intangible the goals, the less exact the information that can be obtained. The aim of PIM is to encourage the actors to form hypotheses about their perceptions. Inaccurate observations are permissible. No formal indicators are expected. Even if the information is not accurate, it is first assessed within a group, verified or disputed, and, if necessary, supported by additional perceptions. The information can also be exchanged and compared with other project actors. Thus, the group serves as a filter and corrective instance.

PIM is by no means devoid of logic. It contains elements of both formal logic and networked logic ("rock logic" versus "water logic" according to Edward de Bono). Within every PIM system, the relationship between expectations/fears, indicators and describability is questioned in formal-logical terms. However, there exist relations of networked logic between individual expectations and the different strands of PIM, which are defined by relevance of action, interests and subjective perception. Due to this, PIM - in view of its simplicity - is better able to do justice to a complex reality than a complicated but one-sided monitoring system.

In this way, new trends can be perceived at a relatively early stage, not only by individuals, who can be wrong, but they are analyzed collectively, and they can be quickly integrated in the group's decisions.

Emphasizing the subjectively important changes

PIM wants a solution that is subjectively the best for the actors. Contrary to conventional monitoring systems, PIM is not only oriented towards abstract project goals. These abstract long-term goals are frequently irrelevant with respect to the actual activities and problems of the actors. In view of the fact that PIM builds upon the fears and expectations of the members of the group with respect to the achievement of goals in the long-term, it concentrates on goals that can be achieved quickly and, thus, the activities have an immediate relevance.

This aspect of the subjective relevance of activities makes PIM attractive to the actors. In contrast to collecting data in order to attain an objective goal, the monitoring is regarded as being meaningful.

By the way, the special feature of PIM is not that it works on the basis of subjective information. Perception and evaluation are always subjective. Rather, the special feature seems to be that this kind of information is explicitly permitted. Thus there is no pressure to prove anything to an outside party. Nonetheless, during the field phase, there were always attempts to make the perceptions verifiable "intersubjectively" (= objectively) within the group during the internal discussions of a group of actors.


Project

Determining the socio-cultural impact

PIM is designed to monitor the impact on the project environment. Conventional planning - like monitoring and evaluation - concentrates on technical and economic effects. PIM wants to focus on the socio-cultural impact.

Yet, this seems to involve some contradictions: on the one hand, the expectations and fears are focussed on goals to be achieved in the short-term - and these usually are technical and economic; on the other hand, PIM considers the actors to be autonomous. So how can one tell them what they are to concentrate on?

For development organizations/NGO and funding agencies the following applies: Since PIM usually is supposed to supplement a conventional monitoring and evaluation system, it is not difficult to systematically obtain socio-cultural indicators.

The monitoring of the socio-cultural impact without any external specifications also worked quite well with self-help groups, as a matter of fact, much better than expected by the PIM team. The mechanisms:

- Many expectations and fears of people are connected to their abilities:

"that we can manage it properly",
"that the storehouse is refilled regularly",
"that the bookkeeping works",
"that we have more time for our families",
"that all members participate", and so on

- The people concerned often take technical or economic indicators as indicators for socio-cultural changes:

"sugar content of the palm juice" = "that the members do not dilute the palm juice delivered with water any more";
"the amount of quinua sold" = "the members' diets are healthier";
"status of loans" = "that the members' payment habits have improved".
What these expectations express in general is that "the members are acting more responsibly".

- Changes in the indicators are also very informative. On the one hand, they show to what extent the perception of problems is changing (changes in an indicator) or whether a problem is no longer considered important or has been solved (omission of an indicator). On the other hand, it shows whether these are short-term problems or a long-term process: the short-term problems disappear after a while; the long-term problems remain part of the monitoring system. Thus, the impact is also determined through the system's self-regulation.

- The application of PIM itself triggered many learning processes for persons and, in particular, in self-help groups, which were then observed by the staff of the development organizations/NGO (see Booklet 3)

Let us make one more comment with respect to impact: First, PIM does not make a strict differentiation between changes, effects and impact. Rather it tries to identify subjectively important changes at the beginning. Only in a second step, does it determine how these changes are related to the activities of project actors and, thus, are effects. Thirdly, the permanence and range of the changes is determined by regular monitoring (see Section 4.2.2). Due to this self-cleaning mechanism, effects and permanence are filtered out automatically.

Periodical reflection as monitoring

In simplified terms, one could say that the project cycle is divided into situation analysis, planning, monitoring, and evaluation. The participatory approaches to project management always focus on one of these areas, most frequently on situation analysis, planning, and evaluation. Concepts such as PIM, which purposely concentrate on monitoring, are rare.

Even though PIM does not differentiate strictly between evaluation and monitoring, the term "monitoring" is purposely used in the name. The term "monitoring" is better suited to underscore the frequency and continuity of the perception, reflection and, if necessary, decision-making, which accompany the continuous activities.


Applying PIM

PIM can be applied in every phase of the project cycle, since it is not based on formal specifications or plans. Thus, PIM is a tool for process-oriented project-cycle management. As shown by the examples, PIM also improves the management of the entire organization.

The key element is regular observation and reflection - at different intervals and to a different depth at the individual levels - within the group of project actors and between the groups. Vice versa, it can be concluded that the message of PIM essentially is as follows: People, open your eyes, think things over, sit down together and talk to each other!

Accompanying learning processes

Projects are learning situations, and learning situations are characterized by the intention with which the learners change. Indeed, every human action is a form of changing oneself if it involves reflection.

Learning in learning situations should make people more capable of acting with respect to things and other people. It should help people develop special skills and a general ability to solve problems. In order to help the learners develop this ability to act, the feedback in learning situations has to be controlled. The learner needs room and has to be able to count on informative and non-punitive feedback; he must be able to make mistakes and improve by learning from them.

PIM also wants to fulfil these educational requirements. As described above and in Booklet 3, PIM accompanies people and organizations in their processes of change. In this respect, it closely resembles the concept of organization development.

It would be wrong to expect linearity in these learning situations, instead we expect irregular changes. We have to avoid using planning - knowingly or unknowingly - as an instrument of power. That obstructs any autonomous changes.


Graphic

In this way, PIM does justice to its objectives, and it achieves more than a conventional monitoring system: PIM not only monitors the socio-cultural impact in the project environment through the cooperation of the parties involved, it also stimulates the parties, changes the behavior and attitudes of people and organizations, initiates and reinforces new learning. PIM makes an impact.


Applicability of victories

4.2 The applicability of PIM

4.2.1 Invitation to handle PIM with fun and creativity

The simpler a concept, the more influence it can have on the manner of thinking and acting, the easier it can be adapted to the specific conditions. PIM consists of very simple building blocks (like Lego or Metaplan) that - once their use has been understood - can be adapted easily to different situations and contexts. For this reason, PIM is highly suitable for use in learning situations as well as in situations that change quickly or develop unforeseeably. The individual elements of PIM can be rearranged, replaced or expanded as easily as Lego building blocks or Metaplan cards. Thus, PIM is eo ipso process-oriented.

PIM uses very simple language and relatively many pictures. It tries to integrate itself in the respective reality, to respect and take up local languages and customs, so that it can be used in different contexts. The application of PIM should be considered meaningful, and it should be possible to implement PIM without difficulties or stress. PIM is not supposed to scare off people, and it should be fun to use. It is not supposed to correspond to the concept of hard work; rather, it should be possible to adapt it to cultures with a more easy-going understanding of work.

It is very easy to start PIM: It already works with just a few expectations and indicators. Subsequently, PIM can be expanded piece by piece; it can grow with the users.

Every PIM system, no matter how small it may be, is coherent and independent. It monitors its own expectations, uses its own indicators, and results in its own analyses and decisions. If one part of the monitoring system is given up, other parts can be continued independently. Up until now we have group-based impact monitoring and NGO-based impact monitoring. Yet, other PIM systems can be integrated easily, not only for funding agencies, but also, for example, for supervisory boards of the development organizations/NGO or other committees of the self-help groups.

PIM can be combined with other concepts, i.e. not only with such that feature similar premises with respect to process-conformity and participation, but also with completely different concepts such as ZOPP. Thus, PIM can accompany and support the gradual discontinuance of an old concept.


Monitoring steps

4.2.2 The monitoring steps

Group-based impact monitoring can be integrated without any difficulties in the regular meetings of groups; thus, only relatively little additional time will be needed. Since the monitoring is supposed to be carried out by the members of the group themselves, the individual steps can be defined freely. The informal aspects should be enhanced and must not go under as a result of premature formalization.

In the same way, NGO-based impact monitoring can be integrated in team meetings. If these are not held regularly, PIM provides an occasion. As a rule, it should only be used as a supplement to conventional monitoring of the target structure, resources and frame of action. It is possible that indicators for socio-cultural changes are already being obtained with the conventional monitoring; otherwise one merely has to add the socio-cultural indicators obtained through the PIM concept.

However, in those cases where conventional monitoring is not being conducted by the NGO, but the organization is interested in having such a system, PIM can also be used as a basic building block for a monitoring system. Due to its loose concept, it can allay people's fears and illustrate the feasibility of monitoring. Then it can grow slowly together with the NGO.

The steps and elements of PIM are described and commented in detail in the booklets 1 and 2.

Step by step

Steps in introducing PIM

1. What should be watched?

expectations and fears of the staff members with regard to socio-cultural changes

2. How can it be watched?

concrete examples of how these changes can be observed (indicators)

3. Who should watch?

elected staff members who are directly involved in the respective project

4. How can the results

records, tables, graphs, charts, be documented? descriptions

Steps in carrying out PIM


5. What did we observe?

reports at the beginning of staff meetings

6. Why do we have these

assessment and analysis by the staff results?

7. What should we do?

immediate decision (or preparation for a decision) at the meeting (= adjustment of plan)

Steps in introducing PIM

1. Expectations and fears: at the beginning, the group determines which changes are most important for it:

- What are our expectations?
- What are our fears?

Three to five important aspects are selected from the statements.

The introduction is based on the assumption that these expectations and fears are a significant motivation for the group's members to participate in the activities or in the self-help group. Constant observation of these subjectively important effects or changes leads to reflection and decision-making, and it is a means of self-controlling the project and, if applicable, the organization.


Expectations &fears

Naturally, the expectations and fears mentioned above are only the tip of the iceberg. Therefore, the concept provides that

- they be corrected and refined continuously;
- open questions investigate the actual, unforeseen changes;
- other significant expected or feared changes be included in the monitoring;
- a differentiation can be made between different subgroups (e.g. women, youth) that have different expectations and personal obiectives;
- group meetings begin by asking about any observed changes, so that the group can take regulative decisions.

An important objection to the approach of querying the group's expectations is that it does not help define the "real" impact. Yet, what is the "real" impact? The further away the recipient of the monitoring report, the more abstract the concept of impact becomes. However, the significance of PIM is that it creates awareness of the fact that there are different actors with different points of view. A theoretical explanation of the terms "impact" and "monitoring" cannot be provided in an introductory meeting of a self-help group, and this is not necessary. The course of the process is marked by the corrective measures, e.g. whereas indicators for short-term problems will disappear after a few months, the significant problems will remain on the agenda for a longer period of time.


Indicators

2. Indicators: the group gives examples - as concrete as possible - of its expectations and fears. Simple indicators are formed on the basis of these examples.

Anyone who criticizes the scientific quality and systematics of this type of monitoring has to accept that it is primarily concerned with learning. And learning always means starting at a simple level and moving on to more complex aspects: the indicators are formulated and developed further by the group.

During the field phase, many indicators had not been derived from the expectations or fears by means of formal logic. Possibly some indicators were chosen on the basis of other considerations, e.g. because they are observed by the people anyway or because they contain other expectations. By analyzing the perceptions repeatedly, the meaningfulness and quality of the indicators are examined. In a process of trial and error, the group can work towards better indicators.

Another striking factor was that relatively many measurable indicators were selected. Then the relationship with learning processes and behavioural changes is not obvious to outside persons and needs to be explained. Yet, as mentioned above, the relationship made sense to the group.

In addition, a relatively high percentage of the indicators required "Yes" or "No" as an answer; supplementary comments then described the background. At first sight such indicators may seem unsatisfactory, but they can be applied most easily by an unskilled group. Moreover, the comments, in which these indicators were made more precise, proved very informative.


Observers

3. Observers: the group designates observers ("watchers") for the indicators/examples and defines how they are to report on these (e.g. on posters, at meetings).

The observers - formally - can be designated very quickly. Whether they then perform their tasks satisfactorily cannot be influenced from outside. Observers are expected to assume a high degree of responsibility and to work independently.

During the field phase, members holding senior positions were often chosen as observers. Thus, the group had influential observers who were obligated to assume more responsibility and provide transparency. At the same time, the creation of a parallel structure of power was avoided.

In many cases, however, it may be meaningful to confer the monitoring tasks on other persons from the group. Not only does this lead to more selfcontrol within the group, it also tends to stimulate learning processes, responsibility, and commitment among the other members.


Documentation

4. Documentation: the indicators and their form, analyses and decisions are documented continuously.

This step is very important because with the documentation the organizations write their own history. It is a laborious task, but not excessively time consuming. Many self-help groups and development organizations/NGO have already made it a habit to keep records of meetings. During the field phase, the groups started keeping monitoring booklets or documentation.

Nonetheless, this is a weak point in practice, because it requires individual persons to sit down and complete the records regularly. Then the documentation can also be used for outside support.


Steps in implementing PIM

Steps in implementing PIM

5. Monitoring report: A short report is given at the beginning of every group meeting - prior to setting down the agenda: a. How have the indicators changed? (This can lead to corrections and refinements of the indicators used until then). b. What other important factors have changed? (This can help to decide whether additional indicators ought to be observed in future).

The manner of proceeding within PIM involves repeatedly monitoring specified criteria at regular intervals. In this respect it differs somewhat from many processes of self-evaluation and so on, where the situation analysis is based on open questions. Thus, PIM induces more regularity and, hence, shows the development of the group and its situation much more clearly.

Since PIM expects only limited continuity, however, there are inquiries with respect to additional changes. It remains unanswered at first what influences these changes, i.e. whether the changes are direct effects of the activities of one of the project actors. This is still of secondary importance in this step.

6. Analysis: if necessary, the group analyzes the origin of changes, for example on the basis of the following questions: a. What did the persons involved (we ourselves/other project actors/external actors) contribute to the changes? b. Which other effects/conclusions result from this?

Now the actual analysis of the effects is conducted. It may be unsatisfactory in formal-logical terms, but the actors are encouraged to reflect on their responsibility.

A documentation of this effect analysis need not be drawn up. However, it can be prepared at any time, if necessary.

7. Taking decisions: after the analysis, the group defines its agenda and I takes decisions. l

PIM - i.e. regular analysis of changes - should form the basis for the decisions of the individual project actors. Thus, the decisions are based on factual reasons, and the members are enabled to participate responsibly. The leadership of the organization becomes more transparent and democratic.


Figure

Joint reflection workshops

When testing PIM this part was sometimes underrated. The joint reflection workshops are an important part of impact monitoring, because the individual monitoring systems are brought together here, i.e. the individual actors are confronted with the standpoint of the other parties involved in the project.

This comparison of perceptions from different viewpoints underscores the project reality, provides more complex information on the changes, and gives rise to a deeper understanding of the impact of the project. PIM becomes a systemic approach without a formal-logical superstructure through the dynamics of the actors involved, since they continuously rearrange their positions with respect to one another.

Furthermore, this kind of exchange also affects the emotional level: the understanding of the project actors for one another is improved, and communication between them will be easier in future. By having a mirror held up to him, every actor can compare his self-image with the image of the other - and presumably can learn from this experience.

Facilitation of the PIM process

PIM cannot be implemented overnight. The monitoring system needs to be introduced and needs to be accompanied at all levels until it functions automatically. For this purpose facilitators are needed, who support the group in its efforts to conduct an autonomous opinion-forming and decision-making process.

This is even more necessary in joint reflection workshops: for example, if the NGO does not lead the workshop well and slips into an attitude of self justification, it risks blocking mutual learning.

4.3 Prerequisites for and limits of PIM

PIM cannot be the solution to all problems

PIM, first of all, should not be considered a set of instructions for activities, but a concept: it provides a simplified representation of a more complex process; it is limited to a manageable number of dynamic elements; it permits a planned and methodical manner of proceeding; it has to be reviewed and adapted to local conditions; it serves as an introduction to a learning process; it contains a temporary truth.

PIM is hardly formalized. When it is applied in practice to projects, PIM has to be arranged in many respects. When PIM has been defined and adapted to the respective conditions, it can be used as an instrument. Its value as a tool is also limited: The knowledge of how to use a tool is an art!

PIM is not suitable for every project. It was developed for self-help projects where there are many processes and very few structures, where actors constantly react to one another, and where change is expected continuously.

PIM is not neutral with respect to interests. PIM is participatory. It affects power structures, since it works towards giving more decision-making power to previously disadvantaged people. Since PIM brings to light internal conflicts, changes decision-making mechanisms, and limits the influence of those previously in power, it necessarily provokes resistance.

Participation is desirable, but not at any price. It should not be overlooked that participation is practiced differently in different cultures. It can be dangerous to force - in a rush of well-meant missionary zeal - ill-prepared organizations or their members to accept and implement a western-style participatory leadership model. Not every organization can handle the transparency induced by PIM. PIM can work to break up authoritarian structures, but only if many members of the group want such changes and are willing to take responsibility.

The self-help groups should exhibit the following characteristics:

Internal participation, responsibility and decision-making power should be divided among several members to a certain degree.

Continuity of the actors (slight fluctuation), internal consolidation.

The task that is to be managed must be considered an important joint concern.

Prior to introducing PIM, the cultural background of the self-help group should be examined carefully to determine whether the following preconditions are met:

Participation of women to a certain degree.
Extensive ethnic and cultural tolerance.
A certain degree of literacy.

The projects must have the following characteristics:

Flexibility of those involved in the project, especially the NGO and the funding agency.

Continuous and trusting relationship between self-help group and NGO.

Mutual wish of the self-help group and NGO to control the project jointly by means of monitoring.

When it is applied regularly, PIM demands a relatively high degree of discipline from all parties involved. In particular the NGO feel the increase in the required amount of work when they did not plan to accompany projects that were initiated. Roughly speaking about 10% of the working time should in general be reserved for joint reflection, and the PIM concept can provide suitable guidance for this.

PIM also requires certain attitudes:

Mistakes happen! The fear of mistakes has to be reduced to make possible a certain degree of self-critical behavior and, thus, learning.

All cooperation partners (for ex. self-help groups, NGOs, funding agencies) accept one another as competent partners who are willing to learn in their respective fields. Only then can autonomous monitoring and mutual comparison lead to a meaningful exchange, from which all project actors can benefit and learn.

PIM calls for confidence in the ability of the partners and, thus, willingness to cooperate. This means that there must be no impatience or paternalistic behavior. It means that all parties involved must be willing to listen to one another, to let go of their own concepts in order to be receptive for the ideas of others and to be cooperative.


Willing to cooperate

Although PIM is simple, it does require a certain amount of effort to teach its contents. The first organizations in which PIM was introduced did not understand the concept entirely. Here the introduction was accompanied by more intensive support. In one organization, however, where only a basic introduction to PIM was given, the PIM concept was quickly implemented. The successful communication of the concept of PIM still needs to be clarified.

PIM is a young concept which needs to be developed further. Right from its beginning it has been concieved together with partners from all over the world - in a process-orientated way.

We need the cooperation of other practitioners and thinkers to test and to improve PIM.

If you are implementing and testing PIM in your project area we would be very interested to hear from you. Write and tell us about your experience with PIM.

We are planning to organize more regular and more efficient exchanges, if a substantial number of practitioners continue with the development and adaptation of PIM.

Please write to

FAKT or GTZ - GATE (ISAT)

Association for Appropriate Technologies

German Appropriate Technology Exchange

Gheidestra 43

Postfach 5180

D - 70184 Stuttgart, Germany

D - 65726 Eschborn, Germany

Thank You

Participatory Impact Monitoring (PIM) is a concept for guiding self-help projects in development cooperation. The actors involved carry out the monitoring themselves. Because PIM assumes that these actors are autonomous, it has several strands or "strings" - the monitoring systems of the self: help groups and the development organizations are separate. The strings are periodically compared: the actors reflect on their observations and assessments, adapt their planning accordingly and deepen their dialogue with one another.

PIM was developed as an alternative to conventional planning, monitoring and evaluation procedures. It does not presuppose the availability of good planning documents. nor does it postpone reflection to a late evaluation.

The main purpose of PIM is to document socio-cultural impacts. By doing so it initiates and reinforces learning processes, and complements more technically or economically oriented monitoring. Al the same time it is compatible with many other monitoring concepts.

PIM was designed in a joint study by workers in development cooperation from the Philippines, India. Bolivia, Argentina and Germany, and tested in 1993/94.

Deutsches Zentrum fwicklungstechnologien

ISBN 3-528-02086-5