Cover Image
close this bookContribution of People's Participation: Evidence from 121 Rural Water Supply Projects (World Bank, 1995)
View the document(introduction...)
View the documentEnvironmentally sustainable development series
View the documentForeword
View the documentAcknowledgments
View the documentExecutive summary
View the documentChapter 1 - Introduction
View the documentChapter 2 - The concept of participation
View the documentChapter 3 - Research methodology and project descriptions
View the documentChapter 4 - Role of beneficiary participation in project effectiveness
View the documentChapter 5 - Factors affecting beneficiary participation
View the documentChapter 6 - Translating lessons into design features
View the documentChapter 7 - Conclusions and recommendations
Open this folder and view contentsAppendixes
View the documentNotes
View the documentBibliography

Chapter 4 - Role of beneficiary participation in project effectiveness

Project-specific evidence from around the world suggests that participation of users in decisionmaking produces more effective, and more sustainable, projects. There is surprisingly little quantitative, statistical evidence, however, that addresses the following questions:

· Is beneficiary participation an important independent contributor to project outcomes?
· What is the path of influence of beneficiary participation on project outcomes?

This chapter answers these questions by using the data on project outcomes, participation, and other factors described in chapter 3. (The pertinent part of the model under analysis is diagrammed in figure 4.1.) In short, the chapter concludes that the evidence suggests a strong, causal relationship between greater participation and improved project outcomes.

The following section describes the measurement of overall performance and of overall participation; it also reports the basic results for the bivariate relationship between performance and participation. The next section shows that the performance-to-participation relationship is strong even after controlling statistically (through multivariate regressions) for other determinants of project success. The third section shows that the relationship between participation and performance holds across several different subjective and objective indicators of project success; moreover, the relationship holds true for outcomes not directly related to water, such as community empowerment. The fourth section examines the sensitivity of the results to the fact that the data are subjective and therefore may be distorted by the "halo effect" (that is, evaluators, because they know the hypothesis being tested, may unwittingly attribute participation to successful projects, and vice versa).

Following these sections, which establish a strong, robust statistical association between participation and performance, are the two that present the evidence that this relationship is causal. The fifth section examines the path of influence of participation throughout the stages of the project to show how participation affects the proximate determinants of project success. The next part goes beyond the statistical evidence to show, using narratives of specific projects, how participation (or the lack thereof) directly influences project outcomes. Finally, the concluding section draws policy lessons from the evidence about participation and performance.

Participation and Project Effectiveness

Overall project effectiveness (OPE) is a global measure of project performance, which was coded from the project evaluation reports. Although there are many dimensions of project success, they are generally highly correlated with one another, and this overall measure appears to capture project effectiveness well.6 Measures of other dimensions and aspects of project performance are discussed in later sections.

Participation is defined here as the extent to which beneficiaries were involved in information sharing, consultation, decision making, and initiating action in the program activities. Participation was scored on a 1-7 point scale, with 1 being little or no participation and 7 indicating high levels of decisionmaking by the community. In addition to overall participation, participation at three stages of the project cycle-design, construction, and operations & maintenance (O&M)-was also included; these analyses appear below, in the discussion of participation at various stages of project implementation. Participation by women was included as a separate variable. Overall beneficiary participation, from both men and women, correlated highly with the other measures and therefore will be used as the main measure of participation.7

Figure 4.2 reports the cross-tabulation between project effectiveness and beneficiary participation for the 121 projects.8 There was only one project that was highly effective and yet low in participation, and no highly participatory projects were low in effectiveness. Only 3 percent (1 of 37) of projects with low participation were highly effective, in contrast to 31 percent (18 of 58) with medium participation and 81 percent (21 of 26) with high participation. Simply put, effective projects also tended to be participatory.9

A second test of the relationship of beneficiary participation to project performance is a bivariate linear regression (ordinary least squares) of OPE and participation.10 Table 4.1, column 1, reports the partial correlation coefficient of overall beneficiary participation (OBP) on OPE. The estimate of 0.62 implies that the impact of increasing participation from a low level to a high level will improve project performance from between 1.0 to 2.5 points (on a 7-point scale). A 4-point increase in participation is associated with, on average, a 1.2 point increase in project performance. The t-statistic tests whether this estimate is consistent with the hypothesis that the association is zero. The value of the t-statistic at 10.6 implies that the estimate (0.62) is statistically different from zero at essentially any significance level.11 Thus the statistic implies a significant relationship between beneficiary participation and overall project effectiveness.


Figure 4.1. Model for testing the relationship of participation to performance outcomes

Participation and Other Performance Determinants

Evidence of a bivariate association is just the first step in establishing causality. Before drawing any inferences from the evidence of a strong bivariate association between participation and project effectiveness, it must be ascertained whether the association is due to the mutual association of performance and participation with some other variable (such as quality of management) that may be responsible for performance and participation. There may be a tendency for "all good things to go together" so that, for instance, projects with strong management are both more effective and more participatory. The second step toward establishing causality therefore is to test for the robustness of the participation variable in the presence of other important nonparticipatory determinants of project outcomes. In order to do so, a series of multivariate regressions was performed, adding a wide variety of factors that affect project outcomes directly (that is, not through participation) as well as both directly and potentially indirectly (that is, factors that directly affect performance but which may also have an effect on participation).12

Figure 4.2. Cross-tabulation of overall project effectiveness with beneficiary participation by number of projects


Overall project effectiveness


Low

Medium

High

Total

Low

21

6 0


27

Medium

15

34

5

54(45%)

High

1

18

21

40(33%)

Total

37(31%)

58(48%)

26

121(100%)

Overall beneficiary participation

Note: Percentages of the total number of projects are given in parentheses in the Total row and column.

Participation after Controlling for Direct Determinants

Based on the model charted in figure 3.1 and elaborated in figure 4.1, a set of seven project characteristics and exogenous factors, which were hypothesized to directly affect project outcomes, were added to the estimated equation for project effectiveness. These were (1) per capita GNP, (2) total cost, (3) project complexity, (4) adequacy of facilities, (5) difficulties in recruiting and retaining staff, (6) specified objectives and targets, and (7) availability of parts and repair technicians. Since the chapter is focused on participation, the results for these variables are discussed separately in box 4.1 (see also appendixes 1A and 1B).

The coefficient of overall beneficiary participation is reported in the first row, second column, of table 4.1. A bivariate correlation is likely to exaggerate the effect of any single "causal" variable with the seven direct determinants. Hence the magnitude of the estimated coefficient falls roughly in half (from 0.62 to 0.28), as one would expect from adding controls. The drop suggests that much of the bivariate association of participation to project effectiveness is attributable to the mutual association with the direct determinants of project success.13 This conclusion is not surprising, since it is quite likely that one of the determinants of participation is the beneficiaries' expectation of project success. For example, one of the direct determinants of project success (see box 4.1) is the availability of spare parts. If clients know that spare parts are not available, they are unlikely to devote much effort to organizing for participation; the reverse holds, as well.

Even though the partial correlation of participation to project effectiveness is lower after controlling for direct determinants, the estimated impact is still large and strongly significant. The estimated impact implies that an increase of 1.0 point in participation is associated, all else constant, with about a 0.3 point increase in effectiveness. The t-statistic for the coefficient, 5.3, is still strongly significant. The conclusion, therefore, is that beneficiary participation remains a significant determinant of project effectiveness even after controlling for the effects of the seven direct factors.

Table 4.1. Overall project effectiveness as a function of overall beneficiary participation

Model

Regression 1

Regression 2

Regression 3

Overall project

0.62d

0.28d

0.24

effectiveness

(10.6)

(5,3)

(3.7)


n=121

n=77

n=68

Note: Reported are the partial correlations and t-statistics (in parentheses) on OBP from three linear regressions, with OPE as the dependent variable, including different sets of independent variables: (1) OBP alone (bivariate), (2) OBP plus seven direct determinants, and (3) OBP plus seven direct and eleven direct/indirect determinants.

Significance levels are indicated thus a = significant at 0.05; b = significant at 0.01, c = significant at 0.001; d = significant at 0.0001. Numbers of projects for which the regressions were performed are indicated under the t-statistics.

Participation after Controlling for Both Direct and Direct and Indirect Determinants

Project performance is also determined by the range of factors that exert their influence either directly or through their effect on participation of clients. Factors that could have operated either way were classified as "direct/indirect"; these consisted of eleven variables, organized into five subsets. The major distinction of these variables is that part of their effect on project performance may arise from increasing participation, or that their effect on performance may be direct but affected nonetheless by participation. In either case the links are indirect. These variables were added to the model and a third set of multivariate regressions was run. The five subsets of variables included in the regression model were the following:

1. Technology (appropriateness of technology)

2. External agents (support of host government and interagency understanding)

3. Client characteristics (average number of users per system and competition from other sources)

4. Exogenous climate (conduciveness of political climate, conduciveness of economic context, conduciveness of the social and cultural context, and conduciveness of the geological and environmental context)

5. Management (skill and motivation of staff and quality of management).

The results for these variables are discussed in box 4.1.

Box 4.1. Impact of nonparticipation factors on outcomes for water projects, the environment, and equality outcomes

Model 2: direct factors

Availability of parts and the presence of repair technicians emerged as the most significant determinant of nearly all performance outcome measures in model 2.

Clarity of objectives and targets emerged as a significant determinant of overall project effectiveness, while adequacy of facilities and equipment turned out to be only somewhat important. Difficulty in recruiting and retaining staff was an important determinant of other more narrowly focused project outcomes.

As can be seen from appendix 1.A, the model had lower explanatory power for the percentage of target population reached. Compared to the other variables, this variable had the lowest intercoder reliability.7,8.

Two other outcome measures were measured, which were qualitatively different from the water system outcomes: environmental effects and equality of access. Beneficiary participation emerged as the only important determinant of these outcomes in this model. The next set of variables employed have greater explanatory power for these outcomes, however.

The relationship between beneficiary participation and project outcomes remained strong regardless of variations in per capita GNP, project complexity, and total cost. A couple of cautionary notes are in order. The projects included in the study were primarily from the poorest countries, so the range of per capita GNP was narrow and at the low end of the income scale. Similarly, complexity measured the number of different organizations and activities to be coordinated, but these projects generally registered only low or medium complexity; only eleven projects (9 percent) were rated high in complexity. To some extent total cost can be taken as a proxy for project size. The findings seemed to indicate that total project size, as measured by total cost, neither has a bearing on project outcomes nor affects the importance of participation.

Model 3: direct/indirect factors

Availability of spare parts and presence of repair technicians remained an important input determinant. Other significant factors were appropriateness of technology and quality of management. Participation, however, remained significant even in the presence of these factors. Depending upon the specific output being considered, adequacy of facilities, difficulties in retaining staff, and clarity of objectives were important. (See appendix 1.B.)

It is equally important to note that per capita GNP and total cost of project, a proxy for project size, and project complexity did not determine project performance.

For measures of environmental effects and equality of access to facilities, beneficiary participation was the sole critical determinant in this model.

Results for the model that includes all variables are in the first row, third column of table 4.1. Controlling for this additional set of inputs reduces the estimated partial impact of participation only slightly, from 0.28 to 0.24. The effect is still strongly significant statistically. The multivariate results establish that, controlling for the effects of eighteen direct and indirect input variables, overall beneficiary participation remains a large and statistically significant determinant of overall project effectiveness.

Participation and Other Project Outcomes

A strong association between overall project effectiveness and participation raises two questions. First, to what extent is the association a result of the subjective nature of the assessment, both of those writing the project reports on which the data are based and of the coders who mapped the project evaluations into numbers? Second, what are the effects of participation on other aspects of project outcomes, including those not directly measured by effectiveness of water system projects, such as increasing community empowerment?

Participation and Overall Project Outcomes

Three additional measures of overall project outcome were selected (based on data quality and availability) for testing the basic model (figure 4.1). These are the percentage of water systems in good condition, the overall economic benefits, and the percentage of the target population reached. It is likely that these outcomes, because they measure more objectively verifiable quantitative achievements, are less subject to coder and reporter bias than is the evaluation of overall project effectiveness.

Table 4.2. Water project performance as a function of overall beneficiary participation

Model

Regression 1

Regression 2

Regression 3

Percentage of water

0.54d

0,30b

0.29a

system in good

(6.4)

(3.1)

(2.4)

condition

n=98

n=64

n=60

Overall economic

0.53d

0.27d

0.26'

benefits

(10.3)

(4.1)

(3.6)


n=120

n=77

n=68

Percentage of the

0.29d

0.17a

0.25b

target population

(5.3)

(1.9)

(2.5)

actually reached

n=118

n=76

n=68

Note: Reported are the partial correlations and t-statistics (in parentheses) on OBP from three linear regressions, with OPE as the dependent variable, including different sets of independent variables: (1) OBP alone (bivariate), (2) OBP plus seven direct determinants, and (3) OBP plus seven direct and eleven direct/indirect determinants.

Significance levels are indicated thus: a = significant at 0.05; b = significant at 0.01; c = significant at 0.001; d = significant at 0.0001. Numbers of projects for which the regressions were performed are indicated under the t-statistics.

The results of using these outcome variables as the dependent variable are reported in table 4.2. Bivariate results are in column 1 and the results that include the set of direct and direct plus direct/indirect determinants are in columns 2 and 3. Overall beneficiary participation was found to be strongly significant statistically for all three outcomes. Moreover, the estimated magnitude of the participation effect is roughly the same as for overall project effectiveness.14 For instance, for percentage of water supply in good condition, the estimated bivariate impact is 0.54, dropping to 0.29 (t-statistic of 2.4) with the inclusion of all direct and indirect determinants of project performance, as expected since there were no controls in the bivariate relationship. For the overall economic benefits and the fraction of the target population reached, the results are similar, with bivariate (and multivariate) partial correlations of 0.53 (0.26) and 0.29 (0.25), respectively. This strong similarity of results-all estimates from column three are between 0.25 and 0.29-from both overall subjective and physical indicators of project performance suggests that the halo effect of coders' subjective ascription of project effectiveness to participatory projects (or vice versa) does not appear to account for the positive correlation of participation with overall project effectiveness.

Participation and Other Aspects of Water System Outcomes

Another water-system performance outcome, equality of access, was recorded.15 Equality of access measured the degree to which people had access to improved facilities, or the degree to which anyone was denied access for political reasons or for nonpayment of fees. This measure was included because of its relevance to the important goal of reaching the poor. Beneficiary participation was a significant determinant of equality of access (see row 2, column 1, of table 4.3). The result is much less strong in the bivariate case, however, and less strong overall than with other project outcomes.16

Table 4.3. Relationship of participation to environmental benefits and equality of access

Model

Regression 1

Regression 2

Regression 3

Environmental

0.21d

0.23b

0.23a

benefits

(3.9)

(2.8)

(2.3)


n=115

n=74

n=67

Equality of access

0.23d

0.26a

0.17


(3.9)

(2.8)

(2.3)


n=115

n=74

n=67

Note: Reported are the partial correlations and t-statistics (in parentheses) on OBP from three linear regressions, with OPE as the dependent variable, including different sets of independent variables: (1) OBP alone (bivariate), (2) OBP plus seven direct determinants, and (3) OBP plus seven direct and eleven direct/indirect determinants.

Significance levels are indicated thus: a = significant at 0.05; b = significant at 0.01; c = significant at 0.001; d = significant at 0.0001. Numbers of projects for which the regressions were performed are indicated under the t-statistics.

The relative weakness of the results for equality of access is not surprising, because greater participation could engender either greater or less access. Collective action works to the extent that a group of people can formulate and implement rules and regulations regarding entry and exit in using a common property resource. These rules and regulations are critical to preventing free-rider problems. Participation in decisionmaking is a means to formulate rules, and to achieve consensus on them and on their implementation. Hence high levels of participation may mean that access to water systems is successfully limited to those who agree to be responsible caretakers and owners. For example, in Honduras those who did not participate in construction, but who later wanted to be part of a piped-water system, had to pay higher connection fees. Most community groups do not deny access to water because of inability to pay fees, however. The world over, community groups have developed different criteria, dictated by cultural and social norms as well as by economic realities, to ensure access by the poor. In Tanzania, for example, even in poor villages user fees are based on a sliding scale, with widows and others identified as destitute being exempt from user fees. In other countries, the poor are allowed to pay with their labor rather than in cash.

Environmental benefits were also measured. The adjusted it-square for environmental effects is low. Compared to large infrastructure such as dams or river basin management, rural drinking water projects have little dramatic impact on the environment. Nevertheless, the long-term environmental effects from waterlogging or from localized reforestation efforts can be significant. Results from the study show that other determinants-the geological and hydrological context, and the overall political context, which encourages environmental consciousness through legislation and education-are more significant for environmental benefits, although beneficiary participation remains important. The model has low explanatory power for this outcome. Again, participation might or might not lead to greater environmental benefits, depending on where the environmental costs fall. One possibility is that local communities might be expected to be less sensitive than other decisionmakers to nonlocal environmental costs.

Participation and Capacity Building Outcomes

Participation has been conceptualized in this study as a means to achieving project effectiveness, efficiency, equity, and empowerment. The limited number of project cases and the poor quality of the data did not allow testing of the efficiency hypothesis; this section focuses on capacity building and empowerment. These two outcomes are treated as variables, yet they may well be more properly classified as proximate determinants.

Participation in decisionmaking is an important capacity building process. As people participate in making new decisions and solving problems, learning takes place. This learning is internalized, because it is accomplished experientially rather than by rote. It therefore leads to changes in attitude, behavior, confidence, and leadership.

At the individual level are three social actors of particular importance, namely, women, men, and leaders. The question of women's empowerment and participation is important, but the issue is too complex to treat here; to highlight the findings and to do them justice, the subject will be covered fully in a separate paper. Findings on overall beneficiary participation and community empowerment follow.

Empowerment is a result of participation in decisionmaking. An empowered person is one who can take initiative, exert leadership, display confidence, solve new problems, mobilize resources, and undertake new actions. Empowerment, it is hypothesized, is an important outcome of high levels of participation involving control over decision making for a range of activities. Hence empowerment is a leading indicator of successful capacity building at the individual and institutional levels.

The second important indicator of capacity building or empowerment is specific to water and sanitation. Are the ability and skills of individuals enhanced in carrying out specific management and technical tasks related to water and sanitation?

The third indicator is organizational. Decentralized programs require strong local organizations. When local organizations get the opportunity to manage resources and support development, they can become stronger (Esman and Uphoff 1984; Uphoff 1986). Participation in decisionmaking is hypothesized to strengthen the capacity of local organizations to carry out activities. Local organizations can be a few people working on water committees, or a village council, or larger, more formal organizations.

Finally, much has been written and said about the effect of participatory development on local leaders. The general belief is that local leaders resist participatory decisionmaking because it leads to empowerment, which changes the power balance and jeopardizes their power base. The often accompanying belief is that local leaders should be bypassed because they either resist broad-based decisionmaking or capture a disproportionate share of resources. On the other hand, when local leaders get involved they can be effective in mobilizing communities and accessing resources. Although the process may enable community groups to become more powerful, it can also allow leaders to emerge with more power and respect. For this reason a variable on the net effect of participation on local leaders was included in the model.

Table 4.4 reports the coefficients of participation for three models, with t-statistics in brackets. The results confirm the hypothesis that beneficiary participation is deeply embedded in all four capacity building or empowerment indicators, since community empowerment and skills related to water-system tasks are determined by beneficiary participation even after controlling for all eighteen direct and indirect input variables.

Beneficiary participation is even more positively related to the other two outcomes not specific to water projects: strengthened local organizations and effect on local leaders. Clearly, participatory decisionmaking does lead to strengthened local organizations, as evidenced in several ways: a greater sense of pride and identity with the village council or water committee or group; undertaking of new activities, from latrine building to brick and cloth making; greater political savvy and ability to negotiate with (or pressure) political leaders and outside agencies to change rules and regulations, or to mobilize additional resources.

Evidence also establishes that the net effect of participation on local leaders is positive, not negative. An increasing number of projects work through local leaders, win their support, and then reach out to others in a community, including the poor. Often people prefer that their leaders make the decisions and negotiate with the outside world. The status of such leaders is enhanced when they successfully bring resources to the community. Many leaders, once they are satisfied that they are not being bypassed, become either personally involved in stimulating collective action or supportive of others, thus allowing the emergence of new leaders.

Table 4.4. Outcomes not specific to water systems, as a function of overall beneficiary participation

Model

Regression 1

Regression 2

Regression 3

Community

0.77d

0.59d

0.55d

empowerment

(15.6)

(8.2)

(6.7)


n=121

n=77

n=68

Water supply task

0.77d

0.70d

0.63d

capacity building

(14.9)

(8.3)

(6.3)


n=121

n=77

n=68

Extent local

0.99d

1.01d

0.98d

organizations

(17.1)

(10.0)

(8.1)

strengthened

n=109

n=71

n=63

Net effect on

0.24d

0.26b

0.26b

local leaders

(46)

(3.0)

(2.4)


n=106

n=70

n=64

Note: Reported are the partial correlations and t-statistics (in parentheses) on OBP from three linear regressions, with OPE as the dependent variable, including different sets of independent variables: (1) OBP alone (bivariate), (2) OBP plus seven direct determinants, and (3) OBP plus seven direct and eleven direct/indirect determinants.

Significance levels are indicated thus: a = significant at 0.05; b = significant at 0.01; c = significant at 0.001; d = significant at 0.0001.

Numbers of projects for which the regressions were performed are indicated under the t-statistics.

How Important Is the "Halo Effect"?

The case review method uses systematic content analysis, which is based on subjective evaluation by coders of facts and findings reported in documents. It assumes that each coder will be able to subjectively judge and quantify the diverse phenomena. There are two ways in which this coding could go wrong. Either the codings could be subject to a large degree of random error or the coders could be influenced in their assessments of the outcomes by their assessments of participation.

The first problem, random measurement error, is not a major concern for two reasons. First, all of the variables used in the analysis show a high degree of intercoder reliability, in that the correlations of the same variable across projects for the two different coders are quite high.18 Second, a measurement error, if there were one, would lead to an underestimation of the coefficients.19 In other words, if the results are strong and statistically significant in the presence of measurement error, they would only be stronger and more significant if that error were absent. Hence the possible objection that the measured variables could be contaminated by a large degree of measurement error is moot as a criticism of the present results.

The much more serious concern is that subjective evaluations are susceptible to two types of nonrandom measurement error. First, if coders rate both outcomes and inputs, their ratings of project outputs may be influenced by their knowledge of system inputs. This is more likely to happen if the coders know the hypotheses being tested. Second, if project inputs and selected project outputs tend to be positive, that is, good things in their own right, the subjective evaluations of overall project effectiveness-by the original reporters as well as by the coders- may be upwardly biased. This is known as the halo effect.

Two types of analyses were conducted to check for the pervasiveness of the halo effect. The first, already discussed elsewhere in this chapter, is the use of objective measures of project outcome, such as the percentage of the water system in good condition. The reporting and coding of the outputs of such objective measures should reduce the halo effect, although it still may be the case that the degree of participation is overstated when project outcomes are good.

The second technique for examining the halo effect is to take advantage of the fact that two coders coded both outcomes and inputs. If it were the case that a higher assessment of outcomes led to a higher assessment of participation, then the "within"-coder results should be stronger that the "across"-coder results. That is, if X's assessment of OBP is higher when X gives OPE a higher rating, then the regression of X's OPE on OBP should have an upward bias relative to the regression of OPE coded by X on OBP coded by Y (whose assessment of OBP is not influenced by X's assessment of OPE).20 Table 4.5 shows the results of exactly this test. The first row reproduces the results from table 4.1 of OPE on OBP, both bivariate and multivariate (for the most inclusive model). The second and third rows regress X's (or Y's) values for OPE on Y's (X's) values for the independent variables. There is no significant drop in the estimated coefficients. The results are the roughly the same, which is consistent with no halo effect.21

Yet resolving the issue of coder halo effect does not negate the possibility that the writers of the project evaluations themselves built the halo effect into the reports that were subsequently coded. Any halo effect operating on the writers of evaluation reports is likely to be weak, however, since participation was not a focus of those reports and the evaluators did not know that their reports would be used as data for a study on participation.

Table 4.5. Tests for intercoder "halo effects" in coding project effectiveness and participation

Test

Bivariate relationship between
OBP and OPE

Multivariate relationship
(all inputs) to OPE

Mean values: (X+Y)/2

0.62d
(10.6)

0.24c
(3 7)

Coder X outputs on coder Y inputs

0.62d
(10.3)

0.26a
(2.1)

Coder Y outputs on coder X inputs

0.57d
(9.3)

0.26a
(2.7)

Note: Reported are the partial correlations and t-statistics. (in parentheses) on OBP from three linear regressions, with OPE as the dependent variable, including different sets of independent variables: (1) OBP alone (bivariate), (2) OBP plus seven direct determinants, and (3) OBP plus seven direct and eleven direct/indirect determinants.

Significance levels are indicated thus: a = significant at 0.05; b = significant at 0.01; c = significant at 0.001; d = significant at 0.0001.

Proximate Determinants and Participation

The foregoing sections established the existence of a strong, robust association between the level of beneficiary participation and project performance, but even this evidence falls short of proving that more participation causes better project performance. This section and the following one move beyond statistical association, to make a case implying causation and to give meaning to the statistics by highlighting specific project experiences. The present section does so by showing that participation is related to success in each of the stages of a project, revealing the mechanisms whereby participation causes success. The section on project experiences uses case studies to illustrate patterns and processes used in linking participation to improved project effectiveness.22

The model diagrammed in figure 4.1 identifies a series of proximate determinants of project outcomes, which are the stages of a project (design, implementation, construction, and maintenance). It is hypothesized that beneficiary participation affects final outcomes by affecting outcomes at each of these stages. Multivariate regressions were performed to test these hypothesized relationships between proximate determinants and outcomes.

Success in the stages of a project, or proximate determinants, leads to successful project outcomes. For example, better "quality of design" should lead to a more effective project (although better design quality, as an outcome itself, is not an independent cause of a better outcome). If this is true, the correlation coefficients between the set of proximate determinants and outcomes should be high and significant. Table 4.6 reports these bivariate correlations. Looking down column 1, the correlations between overall project effectiveness and the quality of each of the project stages is very high. The same holds true for the other measures of project success, although, not surprisingly, the correlation is less strong for the environmental and equality outcomes.

Institutional Inputs

To explore the path through which beneficiary participation effects outcomes, the proximate determinants were divided into two broad categories: institutional outputs concern the design and implementation of a project, and physical outputs concern the construction and maintenance of water systems. Few would dispute that project outputs are influenced by overall quality of project design and by quality of project implementation. Early involvement of project beneficiaries and other stakeholders not only improves project design but also begins the process of local ownership of projects, which is critical for achieving sustainability. While the importance of getting project design right has long been understood, only recently has attention shifted to ensuring quality of implementation to achieve high-quality project outputs. The two proximate inputs examined here, then, are overall project design and overall quality of implementation.

Table 4.6. Bivariate correlations between outcomes and proximate determinants


Outcome variable

Proximate determinant

Overall project effectiveness

Percentage of water system in good condition

Objective value of benefits

Percentage of target population reached

Environmental effects

Equality of access

Quality of design

0.81d

0.66d

0.70d

0,43d

0.34c

0.32'


(n=121)

(n=98)

(n=120)

(n=118)

(n=115)

(n=112)

Quality of implementation

0.92d

0.75d

0.78d

0.50d

0.38d

0.39d


(n=121)

(n=98)

(n=120)

(n=118)

(n=115)

(n=112)

Quality of construction

0.61d

0.59d

0.55d

0.53d

0.26b

0.29'


(n=121)

(n=98)

(n=120)

(n=118)

(n=115)

(n=112)

Quality of O&M

0.90d

0.83d

0.77d

0.42d

0.30'

0.31'


(n=121)

(n=98)

(n=120)

(n=118)

(n=115)

(n=112)

Quality of maintenance

0.85d

0.89d

0.74d

0.40d

0.35d

0.29b

after one year

(n=117)

(n=98)

(n=116)

(n=114)

(n=111)

(n=108)

Quality of maintenance

0.81d

0.87d

0.72d

0.47d

0.37c

0.33b

after five years

(n=74)

(n=70)

(n=73)

(n=71)

(n=70)

(n=67)

Note: Reported are the partial correlations and t-statistics. (in parentheses) on OBP from three linear regressions, with OPE as the dependent variable,, including different sets of independent variables: (1) OBP alone (bivariate), (2) OBP plus seven direct determinants, and (3) OBP plus seven direct and eleven direct/indirect determinants.

Significance levels are indicated thus: a = significant at 0.05; b = significant at 0.01; c = significant at 0.001; d = significant at 0.0001. The number of projects under consideration are in parentheses.

Quality of design. 4.7 lists the coefficient on participation in each cell, with t-statistics in parentheses, for the bivariate and for two different multivariate regressions. For quality of design, beneficiary participation is significant only in the bivariate model. Given the general belief that early inputs from stakeholders in a project, including beneficiaries, are important in creating a design that fits the needs of project clients, the lack of a significant relationship across the board is puzzling.

But three issues must be kept in mind when interpreting these findings. First, statistically speaking, the lack of significance at the 5 percent level is not the only standard. Low precision can lead to statistical insignificance even when the qualitative relationship is substantial. Moreover, the coefficient itself is not much lower for design (0.16) than for implementation (0.21), which was significant at the 0.01 level. Second, clients include people at the community level-the ultimate project beneficiaries and stakeholders- agency staff, and others who will be directly affected by the project. The staff of public sector agencies, through which most large rural water supply projects have been implemented, were included in the measure used in this study of beneficiaries, but other project stakeholders were not.

Table 4.7. Impact of overall beneficiary participation on the institutional proximate determinants of project performance

Proximate determinant

Regression 1

Regression 2

Regression 3

Quality of design

0.46d

0.12

0.16


(6 9)

(1.3)

(1.3)


n=118

n=76

n=68

Quality of

0.53d

0.17b

0.21b

implementation

(9.3)

(2.7)

(2.7)


n=121

n=77

n=68

Note: Reported are the partial correlations and t-statistics. (in parentheses) on OBP from three linear regressions, with OPE as the dependent variable, including different sets of independent valuables: (1) OBP alone (bivariate), (2) OBP plus seven direct determinants, and (3) OBP plus seven direct and eleven direct/indirect determinants.

Significance levels are indicated thus: a = significant at 0.05; b = significant at 0.01; c = significant at 0.001; d = significant at 0.0001. Numbers of projects for which the regressions were performed are indicated under the t-statistics.

Third, there are two ways of making projects responsive to beneficiary demand: beneficiaries can be involved indirectly or directly. This measure captures only direct involvement, although indirect involvement, as evaluated by other techniques, is considered in the study's overall findings. Indirect involvement includes market surveys, beneficiary assessment, contingent valuation methods to assess willingness to pay, sociological and anthropological studies, and other consultation techniques which produce information on the needs, preferences, capacity, and social and political organization of beneficiaries and intermediary organizations. Indirect information can also be drawn from lessons learned from other projects attempting to provide services to the poor, and then fed into the design process.

Indirect involvement of beneficiaries in design was rated low in participation in the study; the majority of projects did not involve beneficiaries in design at all. Only six projects actually involved beneficiaries directly in decisionmaking for design, so the spread of projects in the highly participatory category was very low. Direct beneficiary participation means beneficiaries work together with project authorities in evolving the design of the project. This kind of involvement is rare, except through the involvement of NGOs and through pilot implementation activities during the process of project formulation; again, lessons from these experiences are fed into the design of the project.

What the results suggest, then, is that the critical factors in macroproject design may be consultative processes and broad stakeholder involvement rather than direct and intensive involvement of large numbers of beneficiaries. Besides participation, the factors of importance in the model were the quality of attention paid to availability of spare parts and repair technicians (significant beyond the 0.0001 level); the number of users per water system, as specified by design criteria (a negative relationship significant at the 0.06 level);23 and the overall complexity of the project (a negative relationship significant at the 0.08 level).24

Quality of implementation. Beneficiary participation is significantly related to quality of implementation in the bivariate model and in both multivariate equations even after controlling for the effects of other inputs, which means, of course, that it is a significant determinant of the quality of implementation. In fact, OBP is the critical determinant of overall quality of implementation. The implications of these findings are important for project design and management processes.

Two other input variables of significance were (1) clarity of objectives and targets and (2) availability of spare parts and repair technicians. When the design of individual subprojects is evolving through the process of implementation and all the details of implementation vary from site to site, it is particularly important to be clear on overriding objectives and targets. Strategies can then be adapted to the local context but still achieve defined objectives and targets.

Physical Outputs and Participation

Many factors affect construction and operation and maintenance, or intermediate physical outputs. These two components are treated separately because, in the past, the water sector has equated participation with free labor and materials or with handing over O&M to communities after construction had been completed. In both scenarios, although communities were asked to "do something," they were not part of the decisionmaking process. Neither approach treats participation as a process.

Effectiveness of construction. The contribution of beneficiary participation to construction effectiveness is significant only in the bivariate model (table 4.8). It is not significant in either of the multivariate models. Thus, when controlling for the effects of other inputs, beneficiary participation no longer remains a significant determinant of construction quality, although it is marginally significant in the limited multivariate results (column 2, p-level of 0.075); also, as discussed elsewhere, the full multivariate results may understate the true total effect.

In the multivariate model that included all direct and indirect inputs, three inputs did emerge as significant for the quality of construction. They were project complexity (significant at the 0.01 level), conduciveness of economic context (significant at the at 0.003 level), and average number of users per system (significant at the 0.02 level). Project complexity once again emerged as a significant negative determinant (-0.30), as did number of users per system (-0.30). Effectiveness of construction was the most severely affected by the overall economic context (0.59). The worse the economic context, the worse the quality of construction. This finding makes sense. When there is high inflation or dependence on highly taxed imports, or when economic conditions are poor and markets are not functioning effectively, the quality of physical construction is negatively affected.

Although communities actually led the construction for some projects, historically the norm has been that communities provide unskilled labor and participate in the manner dictated by the construction contractors. Generally, the contractor is accountable not to the communities but to external, governmental or nongovernmental agents. Contractors therefore have had little incentive to interact with people in local communities, whom they see as fonts of free housing, construction assistance, and unpaid labor, not as decisionmakers who will certify that construction has been satisfactorily completed. Project experience also establishes that even when construction is led by communities, timely technical assistance is helpful in assessing quality of construction.

Effectiveness of O&M. Three different measures of project operations and maintenance were coded from the project reports: effectiveness of O&M, maintenance after one year, and maintenance after five years. Since the results for all three were similar (as expected, because they are highly correlated), they will be discussed together. Beneficiary participation is a significant bivariate correlate of O&M with all three variables.

Table 4.8. Impact of overall beneficiary participation on the physical outputs related to project performance

Output

Regression 1

Regression 2

Regression 3

Effectiveness

0.30d

0.18

0.11

of construction

(4.6)

(1.8)

(0.9)


n=120

n=77

n=68

Effectiveness

0.49d

0.14a

0.11

of O&M

(7 4)

(2.0)

(1.1)


n=121

n=77

n=68

Maintenance

0.43d

0.16a

0.18

after one year

(6.6)

(2.0)

(1.8)


n=117

n=75

n=66

Maintenance

0.46d

0.09

0.25

after five years

(4.9)

(0.7)

(1.5)


n=74

n=49

n=45

Note: Reported are the partial correlations and t-statistics (in parentheses) on OBP from three linear regressions, with OPE as the dependent variable, including different sets of independent variables: (1) OBP alone (bivariate), (2) OBP plus seven direct determinants, and (3) OBP plus seven direct and eleven direct/indirect determinants.

Significance levels are indicated thus: a = significant at 0.05; b = significant at 0.01; c = significant at 0.001; d = significant at 0.0001. Numbers of projects for which the regressions were performed are indicated under the t-statistics.

Controlling for the seven direct determinants of project success again lowers the estimated impact. But for two of the three measures, participation remains a significant determinant of success in O&M. As with the outcomes for water system conditions, effective O&M depends heavily on the availability of spare parts and on the presence of repair technicians.

When all input variables are included in the equations for maintenance, however, the estimates of the effect of participation drops to modestly insignificant levels. The estimates range from 0.11 (for effectiveness of O&M) to 0.25 (for maintenance after five years). In the full multivariate model, the availability of parts remains as the only significant determinant of maintenance success. But here again arise the difficulties of interpreting the results of a "kitchen-sink torture test." Including eighteen variables both reduces the available sample of projects and decreases the degrees of freedom, either of which can be expected to lower the precision of the estimates of the participation effect, independent of the effect of the estimate. What is surprising, then, is that so many of the results survived the torture of testing with a large number of covariates, not that some of them failed (in a statistical sense) to survive.

Examining the impact of participation at various stages of the project is the final step in tracing out the effect of participation in general. This examination is particularly interesting, given that programs reluctant to change their way of doing business typically tagged on beneficiary participation at the end of the project (the "handing over" syndrome) or limited participation to the construction phase and to contributions of free labor and local materials. The data suggest that neither approach has the desired effect on any intermediate outcome, institutional or physical.

To test for the lack of efficacy of piecemeal participation or of participation by command, a series of multivariate regressions were conducted matching participation during a particular stage of the cycle with the particular outcome, rather than with overall participation. For example, table 4.9 reports multivariate regression testing for all models for the effect of participation in design (with quality of design); participation in construction (on quality of construction); and participation in O&M (on quality of O&M).

The results establish that participation in one or another stage of a project does not by itself positively affect proximate determinants of outcomes. The coefficient values for the bivariate and multivariate models are lower than when tested for overall beneficiary participation across stages. The findings confirm that participation is a process whose benefits cannot be fully realized when limited to particular stages of implementation. For maximum benefits, beneficiary participation needs to be viewed as a long-term process that necessitates involvement of users from the beginning of a project to its end.

Table 4.9. Proximate determinants of water project outcomes as a function of beneficiary participation in different stages

Project stage

Proximate determinant

Bivariate model 1

Limited multivariate model 2

Full multivariate model 3

Beneficiary participation

Quality of design

0.26d

0.12

0.14

in design


(3.6)

(1.7)

(1.5)



n=113

n=72

n=64

Beneficiary participation

Effectiveness of

0.20'

0.13

0.13

in construction

construction

(3.3)

(1.7)

(1.6)



n=117

n=76

n=67

Beneficiary participation

Effectiveness of O&M

0.39d

0.0#

0.05

in O&M


(6.5)

(14)

(0.7)



n=119

n=77

n=68

Note: Reported are the partial correlations and t-statistics. (in parentheses) on OBP from three linear regressions, with OPE as the dependent variable, including different sets of independent variables: (1) OBP alone (bivariate), (2) OBP plus seven direct determinants, and (3) OBP plus seven direct and eleven direct/indirect determinants.

Significance levels are indicated thus: a = significant at 0.05; b = significant at 0.01, c = significant at 0.001; d = significant at 0.0001. Numbers of projects for which the regressions were performed are indicated under the t-statistics.

Project Experience with Beneficiary Participation

The results insofar reported in the chapter establish statistically the positive relationships between project outcomes and participation. Yet the statistics, which are themselves derived from project experiences, do not tell the whole story. To untangle causal linkages, and to understand the processes and problems in managing rural water supply programs, the project experiences have to be examined in depth.

This section looks at project experiences over the last two decades and highlights three key findings:

· Beneficiary participation leads to project effectiveness, but the forms of participation vary from context to context.

· To achieve sustainable operation and maintenance requires beneficiary participation, spare parts, and repair technicians. Training technicians and distributing spare parts are easier tasks than achieving participation in decisionmaking, which requires simultaneously that beneficiaries be interested and involved in O&M and that agencies give up control.

· For maximum benefits, participation has to be viewed as a process that involves beneficiaries in decisionmaking from the beginning to the end of a project. A command-and-control approach to participation does not work.

Forms of Participation Vary

The statistical findings establish that highly effective projects also tended to be highly participatory. Some of the key characteristics of the twenty most effective projects have been summarized in appendix 2. Sixteen of the twenty most effective projects also scored the highest in overall participation. The forms of participation during the various stages of the projects and the mechanisms used to foster participation varied widely in the different economic, cultural, and political contexts, however. Boxes 4.2, 4.3, and 4.4 illustrate three very different projects, all of which were highly effective and participatory. Box 4.2 highlights a project in Paraguay, which achieved a sustainable rural water supply through institutional and fiscal reform, combined with community organization activities based on community demand. Communities invest in the system and legally own the water systems.

Box 4.3 describes the Aguthi water supply project in Kenya and the involvement of a parastatal organization, which enjoyed a greater degree of fiscal autonomy than is typical of government departments. The Aguthi community initiated the project but turned over responsibility for operation and maintenance to the parastatal agency.

Box 4.4 focuses on the Azad Jammu and Kashmir (AJK) province of Pakistan. In the last fifteen years community action, supported by the local government, has resulted in construction of 1,200 kilometers of roads and in 40 percent greater access to safe water. Local leaders and the rich elite in AJK play leading roles in negotiating with government authorities, in planning systems, and in setting tariffs. Almost every community has a functioning system and keeps monthly financial records; many expand the water system on their own (S. Khan 1992; Minnatullah 1993).

Sustainable O&M: Participation, Spare Parts, and Agency Control

Beneficiary participation, availability of spare parts and repair technicians, and appropriateness of technology were significant determinants of overall project effectiveness. Beneficiary participation in decisionmaking was also important in achieving effective operation and maintenance, but such participation implies that agencies must let go of decisionmaking at the microlevel. Agencies, especially government engineering departments, have found this particularly difficult to accomplish, as the study results reflect. Of the twenty most effective projects in this study, only three were implemented primarily by government engineering and water works departments.

Of course, beneficiary participation cannot correct for inappropriate technologies or lack of spare parts and repair technicians. Yet even if spare parts and technicians are available, they will not be used, in most situations, unless the community has an interest and stake in keeping systems in good order.

These findings are the problems that have plagued the rural water supply sector. By the end of the 1970s there was increasing recognition that greater access to safe facilities was not the only issue; facilities had to be maintained to ensure their functioning after construction, as well. Government, it was also clear, could not sustain central maintenance systems. These insights led in the 1980s to the development of different types of decentralized maintenance systems, ranging from privatized to government-managed two- or three-tier systems.

The success of the these variations depended on creating responsibility for simple repairs at the community level, through "community participation," and on making spare parts easily available. As a decade of experience attests, technology simplifications and distribution of spare parts in a sector dominated by engineers was the easier of the two tasks, and much attention was devoted to it.

Nevertheless, in the absence of meaningful community participation by local people and local responsibility and incentive for undertaking repairs, the presence of appropriate technology and spare parts did not solve the O&M problems. Three cases illustrate the difficulties with government-led maintenance systems.

India: three-tier maintenance systems. The three-tier maintenance system was developed in India in the 1970s in an attempt to create sustainable O&M for village-level water systems. The central government, which had been largely unsuccessful in achieving this goal, saw decentralization as the key to long-term sustainability of water systems.

In the three-tier system, responsibility for maintenance is decentralized and tasks are assigned to one of three levels. The community is the first tier, which is responsible for basic and preventive maintenance. The second tier, at the subdistrict or ward level, is responsible for systems in several villages; a community calls in the second tier when minor breakdowns occur. Finally, the third tier, at the level of the district government, oversees the second tier and is responsible for major repairs.

Box 4.2. Paraguay: institutional and fiscal reforms

The $12.5 million, World Bank-funded rural water supply project in Paraguay had the objective to "promote community commitment to, and participation in, the project" to help achieve long-term sustainability. To achieve this goal, the project employed a number of strategies based on capacity building and establishing clearly defined, legally binding responsibilities between the executing agency and each community.

The capacity building component focused primarily on the executing agency, the National Service of Environmental Sanitation (SENASA), a subdivision of the Ministry of Health. SENASA was a relatively new, untried organization and needed extensive support in a number of areas. The project provided capacity building in such areas as finance, information systems, community organization, tariffs, and design and construction standards. Building capacity did take time: the project took four years longer to implement than had been originally estimated. By sticking to a participatory approach, however, and taking the time to develop SENASA's institutional capabilities- instead of bringing in an outside organization to fulfill short-term construction goals-the project helped to create a stronger overall institutional structure and increased the likelihood of achieving long-term sustainability.

The responsibilities component addressed the sub project negotiations between SENASA and each community. Before SENASA contracted with a junta for the construction of the water system, the community had to fulfill the following legally binding steps:

1. Junta formation. The community had to follow project guidelines for forming a Water and Sanitation Committee (junta), which is duly recognized as a legal entity by the Government of Paraguay.

2. Project description/agreement. The community and SENASA had to negotiate and sign a project agreement, which included a detailed description of each project component and its quantities and costs. (The contract also lists all project plans and documents.)

3. Users'contribution. The junta had to agree to make a cash downpayment of 5 percent as a condition for starting construction; provide cash, labor, equipment, materials, land, or a combination thereof, equivalent to ten percent of the project costs; and take out a loan from SENASA, to be paid back at market interest rates in not more than ten years.

4. Revenue covenant. Each junta had to set tariffs for water service at a level sufficient to obtain revenues to cover routine O&M, debt service on the SENASA loan to the community, and major repairs and replacements (at an amount to be determined by SENASA and the junta).

In fact, the project exceeded expectations: communities contributed 21 percent of total construction costs (6 percent over original estimates), and the project serves almost 20,000 people more than originally estimated. Operation and maintenance is satisfactory, and the majority of systems provide adequate service. The juntas are well-motivated, manage systems satisfactorily, meet most financial commitments, and have little trouble collecting revenues.

Source: World Bank (1987).

The three-tier system looked simple and straightforward on paper. Unfortunately, when put into practice, three-tier maintenance turned out to be much more complex than designers and program implementers had envisioned. Attempts to implement the system countrywide in several countries in Asia and Africa proved that many districts do not have the institutional capacity to effectively execute their responsibilities. An entire infrastructure (logistical, administrative, financial, and technical) is necessary at the district level to make the system work. In order to have any success in creating this infrastructure, central governments have to hand over financial and administrative control to the districts, which they have been reluctant to do.

The most ironic aspect of the three-tier system is that when districts are capable of executing their responsibilities effectively, communities and the second tier become dependent on the third tier and renege on their own maintenance responsibilities, which reintroduces the problems engendered by lack of community involvement-that is, sustainability of the water system is jeopardized. These problems stem in part from the design of the three-tier system, which focused more on structure than on the incentives that would be necessary to get communities to undertake repairs. Users have no meaningful role in the decisionmaking process and no technical, financial, or organizational control. In fact, in this system community inaction is rewarded: others eventually do the work for the community, at unaffordable cost to the agency.

In addition, the three-tier maintenance system is part of a bureaucratic structure that leaves little room for flexibility if a community wishes to contribute or develop a maintenance system outside the tier structure. One example from a nationwide study in Zimbabwe proves the point: a community had collected $179 toward the improvement of its water supply, but the District Council returned the money because, according to an official, the money "could not be used" (Cleaver 1990). The incentives for communities to take an active role in the system are few in such an environment.25

Box 4.3. Kenya: community participation with a parastatal agency

Effective community participation takes many different forms. The Aguthi rural water supply project in Kenya, for example, sprung from community initiative; long term operation and maintenance is handled by the National Water Conservation and Pipeline Corporation (NWCPC), a parastatal agency. The piped-water system, with metered household connections, serves 68,000 people and cost approximately $6.5 million.

Phase I of the Aguthi project started with conventional contracting methods and no community participation; it was plagued with problems, including delays in construction, cost overruns, and disagreements over consumer payment methods. Eventually construction was halted, and it appeared that the project would not move forward.

At this point, the Aguthi Water Committee met with the Danish International Development Agency (DANIDA), the major funder, and the Ministry of Water Development; the committee offered to supply the necessary labor if DANIDA would take charge of project implementation. The project was redesigned, and the water committee, working with local leaders and project staff, mobilized the community. The committee facilitated community organization by explaining the essential role that community members played in the project-without their participation the project would not go forward and there would be no improved water system. Public meetings were held to explain the project fully to community members; villages were given four to six weeks to organize their participation and discuss any concerns or questions.

Communities contributed extensively to the project, some 93,000 person days, valued at approximately Ksh 2-2.5 million in all. Phase 11 of the project, with the help of this community participation, was completed on schedule and within the budget. As agreed, the community defined its role as paying monthly tariffs after construction was completed. O&M was handled successfully by the NWCPC.

NWCPC had a salary structure about 40 percent higher than that of the government, with high salary increases going to the lowest-level staff. In addition, NWCPC, unlike the Ministry of Water Development (which had to funnel funds through the Treasury), can use revenues directly to meet the costs of the project. These incentives helped NWCPC develop motivated staff who understood their jobs and performed well. NWCPC has been successful in collecting revenues through its monthly meter reading and billing; about 91 percent of potential revenue was collected in 1990.

Source: DANIDA (1991).

Box 4.4. Pakistan: community-based rural water systems

Community-based water supply schemes are common in Azad Jammu and Kashmir (AJK) state, which has a population of over two million people.a These schemes are identified and initiated by communities and developed on a self-help basis with cost-sharing support from the Local Government Rural Development Department (LGRDD). Guided by local leaders and elites, communities make technology and service-level choices, and plan and design the systems with limited technical guidance from LGRDD. The AJK experience demonstrates both cost effectiveness and sustainability. It serves as a model for large scale replication. A recent World Bank loan of $28 million will extend the community-based approach to an additional 1,000 villages.

Bangrila village in Mirpur district (Azad Jammu and Kashmir) is an example of how a community-based, piped-water system works. Bangrila has a population of nearly 5,000, dispersed along the slopes of a hilly terrain. In 1981, in response to a desperate need for potable water, the local community decided to develop its own water supply system. The villagers formed a water committee on their own and then approached the LGRDD through the Union and District Council of the area. The community agreed to share 50 percent of the capital cost of the project and the entire cost of operation and maintenance. The water committee raised the required funds from contributions made by residents and relatives living overseas. The project was executed as a joint venture of the community and the local government department. Total project cost was Rs 830,000.b The hardware component of the scheme consists of a turbine pump with a 30-horse power electric motor and 15,400 running meters of pipe.

Nearly 250 households have water connections. The security fee for connection is Rs 300, with a monthly fee of Rs 35 per household. The total monthly contribution amounts to Rs 8,500, which covers the electricity charges and salaries of one operator and one valveman. The chairman of the water committee maintains an account register which shows the monthly contribution of each household and expenses incurred. The register can be examined by any member of the community upon request.

The scheme has been working without any major breakdowns for the last ten years. It was also successfully expanded during this period. Initially the community built one water tank with a capacity of 10,000 gallons; over time, the community gradually extended the scheme, which now has five water tanks.

a. This project was not included in the 121 cases studied.
b. Rs 35 = $1.00 in 1994.
Source: M. Khan (1992); Minatullah (1993).

Uganda: decentralizing maintenance. The struggles of the government of Uganda over the last thirty years illustrate the difficulties of achieving effective government-managed maintenance systems without fiscal and asset decentralization to the district level. Shortly after the country's independence in 1962, the government began implementing its plan to provide safe water and other social services to the entire population. Water-system implementation and maintenance was highly centralized under the control of the government's Water Development Department (WDD). Throughout the next decade, however, escalating internal instability wreaked havoc on the water systems. About 80 percent of all pumps were working in the 1960s; by the 1970s most were in disrepair. According to government statistics, by 1980 approximately 95 percent of the rural population had no access to safe water supplies.

Results of a community survey conducted by UNICEF and the government in the early 1980s showed that communities were dissatisfied with government repair services, and wanted to own the pumps and be trained and equipped to repair them. UNICEF replaced old pumps with new, standardized ones, and provided standardized training and tools. The central government, however, would not turn over ownership of the pumps to the communities. In the years after the introduction of the new pumps, the central government's maintenance program continued to run into bottlenecks. The government's response time for repairs often exceeded six months, and more than 50 percent of pumps were not in service at any given time.

These problems continued until 1986, when a coalition government was formed. Given the new government's focus on socialism and selfreliance, UNICEF again recommended a community-based maintenance system for handpumps. This time the government responded positively and project planning with communities began in the Luwero district in April 1986. Box 4.5 highlights the process used in developing the decentralized systems and the continuing problems in applying the approach nationally.

Côte d'Ivoire: maintenance problems continue. The experiences of the World Bank Second Rural Water Supply Project in Côte d'Ivoire point to the fact that, even when trained technicians are available and spare parts are distributed, functioning water systems will not be achieved without community interest, commitment, and involvement in decisionmaking. A survey in 1988 disclosed that, after the government had spent $115 million to construct 13,000 water points, only 52 percent of the handpumps were functioning. This finding resulted in the introduction of a new program financed by the World Bank, called "Programme de Restructuration de l'Hydraulique Villageoise." The objective of the program was to give full O&M responsibility to villagers, supported by a privatized system of repair technicians and nationwide distribution system of pump repair parts. Nationwide, approximately 460 repairmen were identified and trained, a ratio of 1 repairman to about 30 pumps. Spare parts were made available nationwide through an automobile parts distributor.

Sensitization meetings were held in all villages, and 94 percent of the village groups agreed to create water committees and accounts and to prepare pump sites before the government rehabilitated the pump free of charge. When the program began, however, 83 percent of the villages did not prepare the pump site, 31 percent did not create a village account, and 45 percent of the village committees were not operational. Government technicians rehabilitated most of the pumps nonetheless.

One year later, according to a survey conducted in late 1992, over 30 percent of the pumps were again out of order, and half of the village water committees were not functioning, 20 percent of which had already disbanded entirely. This unfortunate outcome occurred despite the availability of well-trained and motivated repair technicians and of spare parts.26 Interestingly, though, some villages achieved great success: committees were functioning well in areas where the community organization was already strong and there was a strong felt need for an improved water supply; committees thrived also where communities sold water and managed profits themselves (Hino 1993). The program's commitment to monitoring its own performance and to refining its approach to correct for problems is now leading to a clearer demand orientation and focus on beneficiary participation.

Box 4.5. Uganda: decentralizing maintenance

The community-based system of handpump maintenance evolved through extensive discussions with communities and administrators at each level in the Luwero district of Uganda. Community "mobilizers," provided by the Luwero district administration, central government, and UNICEF, explained the project to communities during local religious services and community meetings. Each community was then asked to appoint eleven members, plus its nine Resistance Committee (RC) members, to represent it at a series of meetings on the project. (The Resistance Committee system, which exists from the village to the national level, was created by the new government to decentralize social services.) Representatives met to discuss the project with government and UNICEF officials. Over I 00 such meetings were held before the project began.

Through these discussions, project implementers learned from the communities how best to develop a community-based maintenance system for the handpumps. The meetings also enabled the communities to participate fully in decisions about pump location and installation, maintenance and management systems, and methods for raising funds to cover maintenance and management. Community representatives reported findings from the meetings to their communities and gathered additional suggestions for future meetings.

Representatives were then chosen for a subcountylevel committee, from which five members were sent to a five-day workshop at the district level to finalize the system. Representatives from UNICEF, the central government, NGOs, and the district administration also attended. After the system was finalized, guidelines were written to help communities set up their maintenance systems. This process approach allowed maximum flexibility for the community, while still satisfying the subcounty and district levels by standardizing the system as much as possible.

The strategy for the system included the following steps: each community selected a pump caretaker, who was trained in preventive maintenance by the Water Development Department (WDD) staff and given necessary tools; every group of twenty communities selected two pump mechanics for training in major repairs and maintenance, and each of those mechanics were given tools and a bicycle; each community decided how it would pay pump mechanics and maintenance people; each subcounty RC was given a one-year supply of spare parts; and sales depots were established to sell spare parts provided by WDD at the district level.

The system was monitored closely by the government and participating communities and refined over the first two years. Over the eighteen months of the project cycle and for two years afterward, 98 percent of all pumps were working at any given time, and the time between breakdown and repair did not exceed two days, with most repairs made the same day of breakdown.

Difficulties remain in trying to apply this decentralized system throughout the country. The system depends heavily on the district level, yet district-level capacity is very limited after special "project units" are removed. In addition, although the central government has focused on giving districts the administrative, financial, and decisionmaking responsibilities for water systems, it is still reluctant to provide districts with the assets and personnel necessary to fulfill these responsibilities.

Source: CIDA/SIDA (1993).

Establishing a Participatory Process

The study findings establish that beneficiary participation in decisionmaking is an iterative process without clearly marked stages (except for analytical purposes). Hence it is important that beneficiaries be involved in decisionmaking at all stages of a project cycle, and not just carry out certain mechanical tasks, such as gathering local materials, laying the pipes, and hosting outsiders. Beneficiary participation in decisionmaking was the single most important contributor to overall quality of implementation.

Project experiences establish that, when given opportunities, communities can take the lead in design, redesign, construction, and maintenance. A few projects involved beneficiaries directly in subproject design. Direct involvement of beneficiaries in determining the overall project design was rare; reliance on indirect participation through surveys and other consultation techniques was heavy, however. The projects in the study employed different approaches to community roles in construction. The level of beneficiary participation in construction was high for 41 percent of projects, medium for 29 percent, and low for 30 percent.

Two projects in Ecuador, one funded by USAID and implemented by Catholic Relief Services (CRS) and the other funded by CARE/ Canada and implemented by CARE and various governmental agencies, are examples of the positive effects of local control in the construction phase of a project.

In the CARE project, water committees coordinated construction activities for the community, which contributed a substantial amount of labor to the project. Using local authority to organize construction also drew successfully on the local highland tradition of communal work groups, or mingas, used by the indigenous population to solve a variety of infrastructure problems.

The CRS project also relied on local water committees (juntas) to organize community members for construction, as well as to resolve logistical problems which arose during the construction phase. CRS went even further, however, in giving control to the local community. The estimated capital cost of each subproject was allocated to the community junta, which could then purchase materials and equipment and hire skilled labor, as necessary.

Both of these projects achieved good operation and maintenance of facilities. The CRS project had especially positive results, with communities initiating a number of new community projects. It is important to note, however, that beneficiary control of construction was only one part of a whole strategy of participation.

The CARE/Rwanda Byumba Southeast Water Systems Project is another example of how beneficiary participation in construction, as part of an ongoing process, can positively affect operation and maintenance. The CARE project staff had extensive dialogues with the community before construction began-which had not happened with previous projects in the area-and community members themselves worked to construct the water system. After construction was completed, community members expressed a strong sense of ownership and responsibility for the water systems. Users have since carefully maintained the premises and have protected the installations from vandalism, in marked contrast to their treatment of previous water systems constructed by outsiders.

In a demand-based approach, which treats participation as a process and community people as the clients, the most important issue in construction is not contribution of labor and materials, or even involvement in all the construction decisionmaking; it is, instead, shifting accountability for construction from outside agents to internal community groups. In this process the community decides how it will participate. It may decide to contract with an agent for all of the construction work, with the community paying for all the services or signing off on government provided assistance. Or it may decide something else. The point is that community accountability yields positive results.

It is easy to subvert any process. Experience shows that ensuring accountability for quality construction requires the community at large to assume a watchdog function. If it does not, the benefits are captured very quickly by local elites, who are in a natural alliance with the outsiders. The Orangi Pilot Project in Pakistan is the most dramatic documented case of the benefits of local accountability: system costs dropped to one-tenth of their earlier level, largely due to reduction in rents and leakages, when accountability shifted from distant agents to the community.

While participation of beneficiaries in operation and maintenance is important, project experiences show also that injecting participation for O&M without any earlier consultation does not work. The "handing over" syndrome characterizes this staged approach to participation: typically, a public works agency constructs a system without conferring with community members; then, just before leaving, the agency informs the community that it is responsible for operation and maintenance. Communities, unless they are in extremely dire straits, have no interest in making the effort to take care of a system they did not ask for and do not own. Experience of project after project confirms this conclusion.

Policy Lessons

Beneficiary participation is a key contributor to achieving project effectiveness and maintaining water systems in good condition, as well as to enhancing local management capacity and empowerment. Beneficiary participation in decisionmaking by definition implies that a community will share in or control major decisions at the subproject level. Obviously, this cannot happen if agencies continue to control all of those decisions; beneficiary participation therefore presupposes major institutional reform to shift control from agencies to communities.

Beneficiary participation by itself, however, in the absence of support and linkages to the world outside the village community, will not result in effective projects. Availability of spare parts and repair technicians and use of the right technology are particularly important. If promised inputs, training, or supplies are not delivered, beneficiary participation cannot transcend their absence. Project designers also need to pay attention to the complexity of the system, including technical complexity and number of users and different activities and organizations to be coordinated. The greater the complexity, the greater the likelihood of failure. Given the high rate of unsuccessful rural water supply projects, project design should focus on delivering water first and sanitation later.

Direct, intensive involvement of large numbers of beneficiaries is not important in influencing overall project design, but consultative processes (defined as low levels of participation) and broad stakeholder involvement may be critical to project design. Moreover, although it may not be important for beneficiaries to be directly involved in project design, it is important that project design be such that it enables beneficiary participation during implementation, once a project has been formulated and financed.

All projects are made up of subprojects, whether they are conceptualized that way or not. Involvement of beneficiaries in the implementation of these location-specific water systems is important at every stage, from planning the system through site selection, resource mobilization, social organization, construction, and operation and maintenance.

One of the characteristics of participation is that it cannot be turned on and off like a tap, that is, "now you participate, now you don't." Participation is an evolutionary process that gathers momentum and defies breakdown into neat, self contained categories, except for analytical purposes. Attempts to establish that participation is particularly important in any particular stage, especially in the later stages of operation and maintenance, are not only misguided but have contributed to ineffective projects.

Participation should be viewed as a process that starts with planning and ends with operation and maintenance, rather than as an element that can be injected in the later stages of a project whenever outsiders determine. Effective participation takes many forms-there is no one ideal form.