(introduction...)

For more than 50 years, NGDOs have been involved in a vast array of initiatives that would fall under today's understanding(s) of social development (Alkire, 1997; World Bank, 1996a).^l9 The first questions posed and tentatively answered in this section are what is there to show for all these activities? What have NGDOs achieved?²⁰ The overall answers fall short of some expectations. Another question asked and answered is why NGDOs are constrained as effective agents of social development. In this study answers to these questions emerge from comparisons of what NGDOs are expected to do by those supporting them against evidence of NGDO performance and impact. Evidence of the latter, is, however, drawn from an uneven and unconnected range of studies.

¹⁹ Paul Francis of the World Bank argues that there is no single social development paradigm (Francis, 1997)

²⁰ This section draws extensively on Fowler, 1997 and 1998.

Expectations of NGDOs in social development

Intentions expressed at Copenhagen (see footnote 1) and publications from the official aid system indicate the range of development tasks NGDOs are expected to take on.²¹ Their mix of tasks can also be deduced from publications about NGDOs actual development roles and behaviour.²² Together, they permit a summary of what NGDOs are expected to contribute to (social) development, listed in Box 1. As can be seen, the expectations are broad and complex. The roles NGDOs are expected to play run the spectrum from state substitute to activist for structural social change. As service providers, they are expected to reach those the market and state cannot. Political tasks include pushing for policy reforms, exacting civic compliance on government and business behaviour and fostering "good governance".

²¹ Selected references: ADAB, 1995; African Development Bank, 1989; Asian Development Bank, 1996; SIDA, 1998; UNDP, 1995; USAID, 1997; World Bank, 1998b.

²² Selected references are listed in footnote 13 and Craig and Mayo, 1995; Gibbs et al., 1999, ODI, 1988; OECD, 1988.

Box1 : Expectations of NGDOs

NGDOs will cost effectively help expand access to, and effectively deliver, tangible services (such as education, health care and credit) that reduce unemployment and levels of poverty among the most vulnerable of the world's population - particularly women, children and indigenous peoples.

NGDOs will have a positive influence within (civil) society. For example, they will foster social integration and contribute directly and indirectly to the pro-conditions needed for democratic governance, such as civic awareness, inclusiveness in political processes, stronger demands for accountability and active defence of people's dignity and rights.

NGDOs will engender people-centred social development processes, build local capacity and the "ownership" of benefits that will be sustained without external finance. They will themselves reach a stage where foreign aid is no longer required for their functioning.

NGDOs will gain leverage on national and international policies that condition progress towards social development goals.

NGDOs will act as watchdogs of the public good and safeguard the interests of disadvantaged sections of society.

NGDO interaction with funders will have a positive influence on the quality of aid practices employed by governments and bilateral and multilateral agencies.

NGDOs will exhibit integrity and provide unambiguous, verifiable accounts of the resources they employ

NGDOs will maintain voters' motivation to support tax allocations for aid.

NGDOs are expected to intervene at any level, from local to global. In addition, it is assumed that they will increase or multiply their impact in a variety of ways - by expanding their outreach; demonstrating viable alternatives that can be taken up by governments and aid agencies; diffusion, where their work is spontaneously adopted; and influencing policy frameworks, which then have widespread effects (Edwards and Hulme, 1992).

By and large, when added together in all their locations and variety, NGDOs do indeed span the whole range of development action expected of them. However, their direct outreach remains modest (see below). Their indirect impact on poverty, through policy reform, government adoption of their methods, and spontaneous diffusion, is impossible to judge in quantitative terms. Reasonably certain is that they do not and cannot be expected to impact development/poverty alleviation on a scale with governments and official aid programmes. Such an expectation would be unfair, unreasonable and immodest (Drabek, 1992). The question is how well do they do what they do?

Setting expectations against achievements

Assessing NGDO achievements and impact is difficult. Systematic assessments do not exist - consequently, conclusions on a global scale can only be tentative and are inevitably open to dispute - contrary examples are always available. Nevertheless, drawing on scattered existing evidence, this subsection attempts such a review.

First, the nature and quality of available information is examined. This is followed by a comparison of NGDO performance in social development set against expectations summarized in Box 1. The concluding subsection summarizes the principal factors limiting the effectiveness of NGDOs.

Availability of studies on NGDO achievements

The early 1990s saw unco-ordinated efforts to assess the degree to which, and under what conditions, Northern NGDOs - and by association their Southern and Eastern counterparts - fulfil the expectations listed above.²³ Eleven such "impact studies", evaluations and assessments were made of NGDOs receiving bilateral aid from the major official donors.²⁴ USAID applies a sectoral approach, and hence does not create a broad or composite picture of the effectiveness of its aid to and through NGDOs - targeted to be 40 per cent of its total aid disbursement. There are no similar overall assessments from United Nations agencies. The most recent official studies of NGDOs come from Finnida (Hossain and Myllyl1998)²⁵ and the World Bank, that of the latter concerning the Bank's loan portfolio (Gibbs et al., 1999).²⁶ In as far as they exist, overall (self-)assessments of major Northern NGDOs - e.g. the child-sponsoring NGDOs, such as World Vision, PLAN International, the Christian Children's Fund, ACTIONAID and the Save the Children Federation, are not publicly available.²⁷ Yet, together, these few agencies raise a significant proportion of private contributions.²⁸ On the other hand, the Bangladesh Rural Advancement Committee (BRAC), which is among the largest of the Southern NGDOs, publishes evaluations of its own rural development programmes (BRAC, 1996).²⁹

²³ Though not exhaustive, the following constitute major studies about methods and results of studies on NGDO performance or summary comparisons of them. In terms of direct (project) impact: ADAB, 1995; Danida, 1994; Duran, 1999; Fowler. 1995; Fowler and Biekart, 1996; Gibbs et al., 1999; GOM, 1991; Hossain and Myllyl1998; Howes, 1992; Maren, 1997; ODI, 1996; OECD, 1992; Riddell, Bebbington and Peck, 1994; Riddell and Robinson, 1995; Tvedt, 1995; UNDP, 1993; van Dijk, 1994. In terms of indirect (policy) impact: Chapman and Fisher, 1999, Fisher, 1993; 1998; Fox and Brown, 1998; Madrinan, 1995; Sibanda, 1994; Sogge, 1996; Tercer Mundo, 1997; Thomas, 1994; van Rooy, 1997; World Vision (UK), 1997a; 1997b.

²⁴ As part of its funding agreement with four Dutch co-financing NGDOs, the government requires periodic, independent evaluation of their programmes.

²⁵ This study spans a number of countries and themes, such as NGDOs and local government, or a sector, such as NGDOs and disability, or a specific activity, such as garbage collection in Cairo. The combination appears to be random.

²⁶ While adopting a different vantagepoint - contribution to loan performance - this investigation does not suggest significant revisions to major findings of studies undertaken earlier in the decade. One reason could be that the quality of World Bank funds is itself a constraint that may negate performance improvements NGDOs have made during the decade. In other words, the comparative quality of grant aid versus concessional loans must be taken into account.

²⁷ It could be argued that ongoing performance assessment is provided by the sponsorship mechanism. Dissatisfied sponsors may not have formal control over these NGDOs, but their chequebooks do.

²⁸ It is notoriously hard to get firm figures on official aid to NGDOs and on their other incomes. For example, donors monetarize their food aid contributions, but not all NGDOs do - which is one source of discrepancy. An estimate from recent data suggests that about $6 billion (including emergency funding) is channelled to and through NGDOs (OECD, forthcoming). However, these figures do not necessarily include aid as loans to governments that NGDOs then access and disburse in the country. For example, a figure of $1.3 billion for NGDOs/CBOs over 13 fiscal years (1985-1997) is quoted for projects included in a World Bank study of NGDO/CBO involvement (Gibbs et al., 1999:9). Given current Bank policies on participation, it can be assumed that the proportion of projects with NGDO involvement is increasing, as is the level of finance to them. Funds raised by NGDOs from private sources are also increasing. The 1999 OECD Development Co-operation Report (OECD, 1999) estimates NGDO private income sources in 1996 as $4.428 billion. Allowing for differences in years and the effects of in-country funding, an estimate of total NGDO revenues today stands at some $12 billion or more.

²⁹ Assessments of NGDO performance are more often available for Southern NGDOs than for their Northern counterparts.

Problems of method

Studies of NGDO impact also suffer from methodological dilemmas, difficulties and biases (Coudere, 1994; Fowler, 1995; Kruse et al., 1997; Oakley et al., 1998; Riddell, 1990). For example, studies have concentrated on "projects" to the exclusion of the characteristics and competencies of the organizations concerned. The selection of projects for investigation has shown a slant towards the more positive examples. Given the difficulty of directly attributing social and other changes to NGDO activities, it is not surprising that few studies contained baselines against which change could be assessed. As a result, performance has been narrowly equated with producing the envisaged "outputs" - e.g. schools and wells built, children inoculated, contraceptive adoption rates.

Impact studies have seldom included the effects of pre-conditions stemming from official financing. In addition, studies also suffer from degrees of "politicization" in design or subsequent use because of the institutional interests involved in maintaining NGDO allocations within the aid system. This influence is not the case with studies undertaken by independent observers such as Fisher (1993, 1998), Smillie (1995) and Sogge (1996).

Overall, caution must be exercised in reaching firm conclusions about NGDO achievements because of:

· the preponderance of success stories in the sample;

· the limited universe of NGDOs involved;

· the restricted array of social development interventions being considered (there is relatively little study on social policy impact);

· the complex interests that have guided formulation of the terms of reference and interpretation of findings for public consumption.

Expectations set against performance

With the preceding caveats, what findings do existing studies offer? Compared to the expectations of NGDOs in social development set out in Box 1, data on development impact, not project outputs, suggest a situation summarized in Box 2. The picture is of uneven but generally modest performance, particularly in relation to sustained benefits, participation, social inclusion and empowerment, and in the democratic behaviour of NGDOs. A more positive picture emerges in terms of NGDO influence on development policy.

NGDOs score reasonably highly when it comes to delivering project "outputs", although not necessarily with the poorest (Robinson, 1992:31). A more recent review reaches a similar conclusion:

... The studies concluded that the vast majority of [NGDO] projects assessed do succeed in achieving their narrower and immediate objectives ... Not surprisingly, when judged against broader criteria, projects scored less well (ODI, 1996:l-2).³⁰

³⁰ Wider criteria include poverty reach, participation, gender, environment, replicability, flexibility-and innovation, pre-project appraisal, evaluation and monitoring, sustainability, and cost-effectiveness.

What outreach do NGDOs have? Despite the recent rapid growth of NGDOs, the number of poor continues to grow. Nor is there credible evidence that NGDOs directly reach a larger or smaller proportion of poor than was estimated in 1993, about 15-20 per cent.³¹ Today, a reasonable "guesstimate" is that NGDOs development work directly "touches" some 450-600 million people.³² But touching people says nothing about overall NGDO impact on their lives. Evidence suggests that when project outputs are set against broader criteria of sustained change in indicators of human well-being, capacity growth, empowerment, etc., NGDO performance is very uneven but generally much poorer.

³¹ Reliable aggregate data on those "touched" by NGDOs do not exist. Using country cases and extrapolating from previous estimates, a 1993 estimate (UNDP, 1993:93) was an outreach to 250 million people, at that time about 20 per cent of the world's poor. The number of NGDOs has increased rapidly since then and others have expanded (Salamon, 1994). However, since 1993, population growth, the Asian crisis and widening wealth gaps have co-contributed to the number of the world's poor. Depending on the measures used ($1 or $2 per day), the "poor" are between 1.3 and 3 billion individuals. A direct NGDO outreach to some 15-20 per cent of people who are poor (as opposed to poorest) is still probably a fair "guesstimate" - i.e. some 450-600 million people across the globe.

³² If humanitarian response for refugees and internally displaced people is included, using country by country data on United Nations assistance, the numbers would increase by some 31 million people (IFRCRCS, 1997:127-134).

Box 2 : Expectations of NGDOs compared to achievements33
Expectation	Status
Tangible impact on reducing poverty	· Some, but by no means the majority of, NGDO projects have a positive, enduring influence on narrow aspects (sectors) of human well-being. Holistic change is the exception not the rule. · The scale of direct NGDO outreach, mainly in service provision, is probably stable, reaching about 15-20 per cent of the world's poor. However, this does not necessarily mean the poorest and most vulnerable - targeting remains a problem. · NGDO substitution for reduction in state services is on the increase, but most cannot be maintained without aid. · Gender sensitivity of NGDOs is over-estimated. · Overall, there is scant firm evidence to support high expectations about NGDO impact on sustained poverty reduction, as opposed to evidence about their efforts and project outputs.
Civic impact	· Impact studies have not taken an explicitly "civic" perspective, although much of what NGDOs have been doing could be recast in this framework. Available sources suggest that this is not yet an area of substantial achievement. Specifically, NGDO-supported groups tend to remain isolated from each other and from other civic formations. Mobilization or aggregations of local organizations into substantive civic actors has been poor. · It is difficult to find examples of substantial NGDO influence on social integration, or on political inclusiveness at national level. · There is growing success in fostering inclusion and civic influence on local government.
Sustainability	· NGDO development interventions too seldom lead to sustained change after completion. · Ongoing provision of development services is aid-dependent, with little sign of economic "rooting". However, NGDOs in a few countries in Latin America, and South and East Asia are showing positive signs of local economic embedding. · A consistent estimate is that 90-95 per cent of Southern and Eastern NGDOs would disappear without international aid. · Levels of Northern NGDOs dependency on official aid are uneven, but the average is edging above 50 per cent. ·Significant efforts to diversify and localize the resource base, especially in the South and East, are showing modest, incremental success. · A substantial fee for service income is not (yet) a viable option. · There is a trend for NGDOs to initiate credit programmes as a strategy for their own sustainability.
Policy leverage	· Primarily because of more inclusive aid policies and growth in NGDO capacity - as opposed to widespread civic mobilization - they are increasingly recognized as policy actors in many areas of social development, nationally and internationally. · Concern is being raised about NGDO legitimacy and accountability as policy actors - creating a "backlash" from governments in the South and East. · There is also disquiet about NGDOs using multilateral bodies to gain leverage on their own governments. This can undermine local political processes, erode sovereignty and weaken (local) governments' ownership of initiatives.
Acting as watchdogs	· Success in policy influence is tempered by cautionary experience of NGDO ability to exact national compliance with international agreements and conventions. · NGDOs with a human rights agenda are becoming numerous.
Influence on the official aid system	· Though uneven across governments, donor agencies and topics, there are signs of positive learning from NGDO experience. · NGDOs demonstrate significant and increasing influence in policy reform of aid agencies. · Improving government and donor effectiveness in social development has shifted from learning about what is best practice to actually implementing the organizational changes needed to put such knowledge into practice. Here there is some (decentralized) progress.
Integrity and values	· Endemic corruption attributed to governments is not a common feature of NGDOs, but instances of malfeasance do occur. · NGDO growth is supply led and entrepreneurial, the more so where civil servants are being made redundant. · Voluntary values are giving way to a contract culture, incentives and organizational behaviour. · The purpose and morality of many newly established NGDOs in the South and, especially, the East is raising concern.
Public support for aid	· There is no correlation between aid levels (as a percentage of GDP) and public support for development assistance in donor countries.34 · National aid allocations fluctuate irrespective of levels of development education and public understanding and motivation.

³³ The findings have been obtained by a systematic examination of publicly available NGDO evaluations and impact studies (footnote 23). They are further informed by the "grey" literature of internal NGDO studies and assessments that this author has had access to in his professional work.

³⁴ Public awareness of aid issues through development education is not the same as public judgement that prioritizes aid over domestic needs (Yankelovich, 1996).

Delivery of technical, social and, increasingly, micro-financial services (Otero and Rhyne, 1994; Wood and Sharif, 1997) still forms the primary weight of NGDO activity. This is a logical outcome of the intentions of most of then-funding, be it from official aid, governments in the South and East or the general public. There is scant evidence to suggest that the bulk, of NGDOs have substantially shifted their operations towards redressing the structural or root causes of poverty and insecurity. This would focus their efforts on issues such as land tenure, exploitative internal terms of trade for primary producers, conflict prevention, the legal status of women, corrupting relations between the political and business elite, lack of judicial capacity or independence, culturally embedded practices of oppression and exclusion, and unfair and detrimental international trade and patenting practices, etc.

However, from a weak base, progress is being made in the "public pressure" and civic dimensions of NGDO activity. Various observers are documenting where and how NGDOs are entering and influencing policy dialogue internationally. Less visible, but on the increase, is the impact of NGDOs at the interface of people and local governments, related to decentralization (discussed below). NGDO impact on national policies is, however, more contentious and tends to be more successful when "pushing at an open door". For example, such influence is more likely in technical areas, such as better ways of promoting environmental protection or increasing agricultural productivity (Farrington and Bebbington, 1993).

Some observers ask whether NGDO attention to advocacy and influencing policy is actually being undermined by increasing dependence on official aid - a case of creeping self-censorship (Edwards, 1993). While there are no firm data, a division can be discerned: NGDOs receiving substantial levels of official aid are, indeed, less interested or assertive advocates than those whose sources are private. For example, recipients of funds from private foundations are more likely to be dedicated to or specialized in advocacy work than those relying on ODA. However, the picture is complicated by official aid for rights-oriented NGDOs that are financed to promote good governance (van Rooy, 1998; van Tuiji, 1999). But their numbers and the amount of money they receive remain small compared to the total NGDO community.

Sustainability (of impact) is both a complex concept and a difficult goal for NGDOs to achieve. Studies do not suggest that NGDOs are doing very well in this area. Moreover, there is little evidence that their impact is better or worse than that of bilateral aid programmes, which are estimated to be sustained in about 15 per cent of cases (Cox and Healey, 1998). The picture of NGDO self-sustainability is bleak outside of continued international assistance. Despite much rhetorical but incommensurate effort on the part of funders, proven strategies and cases of self-financing or mobilizing local financial support are in their infancy. They are nowhere on a scale of the foreign transfers that NGDOs rely on.

Progress has been made in systematically gathering the information needed to fulfil a watchdog role (Tercer Mundo, 1997). However, using this information to exact compliance with agreements and covenants signed by governments appears to be problematic. Rme ideology and state-society relations are more important factors conditioning achievement.

As part of complex coalitions, NGDOs are registering success in reforming aid policies, but less so aid practices. While the value of learning about good practices generated by NGDOs is increasingly recognized by the official aid agencies, implementing the institutional reforms required to "mainstream" such lessons remains difficult. In other words, NGDOs may just be widening the gap between rhetoric and reality. Rhetorical change then acts as a placebo and tactic of co-optation by the powerful. This is one reason why it is vitally important that grassroots organizations be part of reform coalitions. Without grassroots accountability, the scope for official co-optation is enhanced. Without them, NGDOs accountability, credibility and legitimacy of agenda are often questioned and their effectiveness is diminished.

Observers are registering the fact that a rapid expansion of NGDOs has been at the cost of some of their qualities: integrity, voluntarism, flexibility, risk taking, overall professionalism, etc. Public trust is not matching an increase in NGDOs or their efforts. In addition, political rmes are concerned that NGDOs are not socially legitimate. They suspect that some NGDOs provide a shelter for political opponents. This often generates unwarranted suspicion about all NGDOs.

Finally, in terms of private funding, public support for NGDOs appears to be at a plateau. Diversification towards "partnership" with business is being actively explored, but with some caution, because this strategy contains dilemmas in terms of values and practices.

Overall, the NGDO contribution to social change, in its many aspects, is certainly there to be seen. However, it is not as substantial as some might imagine. Nor is it as good qualitatively as NGDOs themselves would like. Nor is their motivation or behaviour uncontested. They remain substantially aid-dependent and vulnerable. Improving on these and other shortcomings is already high on the NGDO agenda. However, NGDOs alone may not be able to do very much to increase their effectiveness. As the subsequent sections detail, substantial improvement will require complementary action by others.

Common constraints on NGDO performance

From the studies reviewed, NGDO performance appears to be limited by:

· macro-environmental constraints, including government suspicion, economic mismanagement and poor governance;

· the growing dominance of donor funding that, by nature, works against employing best or appropriate practices;³⁵

· too rapid NGDO expansion due to accelerated availability of official aid; compounded by

· under- and poor investment in NGDO capacity growth.

The following section analyses these constraints in greater detail. Before doing so, it is necessary to identify major parameters affecting NGDOs as agents of social development.

³⁵ For example, a study of European Union support to NGDOs in South Africa concluded that new indigenous initiatives to generate finance for local NGDOs were "highly significant, because the quality of donors' support and grant-making in general has been most inadequate" (Boule et al, 1993).