|Action Research Report on «Reflect» - Education Research Paper No.17 (DFID, 1996, 96 p.)|
|4. The evaluation of the projects|
Although the analysis of self-realisation was revealing in the three pilot studies there was clearly a need to determine a more concrete impact. One key challenge was to see whether this self-realisation had produced a concrete impact on the learners' actual participation in community organisations. This was perhaps ambitious given the short time frame but it was felt to be worth exploring to see if any tendencies were emerging. A longer term evaluation of these programmes would be additionally revealing to determine the sustainability of these tendencies.
In El Salvador, where one of the major objectives of the programme was to strengthen community organisations, this became a central focus. Community organisations in Usulutan included for example: village councils, cooperatives, school PTAs, church groups, sports groups, women's groups, credit committees and health committees. A sample was taken of 36 learners from 7 literacy circles where the REFLECT method had been effectively applied. They were asked detailed questions about their participation in different organisations before and after the REFLECT circle. The results were as follows:
· 80% of learners said that they were now members (or attended meetings) of community organisations which they were not before the REFLECT literacy programme.
· 77% of learners said that they were now actively involved in decision making in those organisations which they did not feel they were before the REFLECT literacy programme.
· 61% of learners said that they had now acquired formal positions of responsibility in those organisations which they did not hold before the REFLECT literacy programme (and in general those who had done so had assumed more than one position).
These results are corroborated by local leaders (representatives from the community councils - "junta directivas") 18 of whom were interviewed. More than 70% said that the literacy circles had significantly increased levels of participation in community organisations or cooperatives) and 50% said that the circles had helped to "renew the leadership" of these organisations.
Interviews with three sectoral coordinators of COMUS (in the health, production and organisation sectors) further emphasise this impact. All of them said that the REFLECT circles had significantly helped them in promoting their work (people who had been learners were now more organised and more active - and their literacy skills were of practical value).
Another observable change has been that now the elections for community councils and committees are held mostly by secret ballot (rather than by a hand vote at a meeting which was the norm before and which had various limitations, not least peer pressure from existing leaders for people to re-elect them). This has helped to democratise the community organisations and would not have been feasible prior to the literacy programme.
These results are striking, even more so when compared with the Control Group (the wider CIAZO literacy programme) where CIAZO's own trainers and advisers say there is rarely any significant impact on participation in other community affairs. Only one control group actually constructed this matrix - and there were no changes in community participation at all.
The reasons for this impact were discussed extensively by the evaluation team and various factors were considered to have been important. The new self confidence of the learners is clearly one factor as is the new respect that they receive from other people in the community. The level of detailed analysis of local problems which they undertook in the REFLECT circles must also have played a part in prompting them to become more active. Last but by no means least, the learners now have the basic literacy and numeracy skills which they require in order to assume formal positions.
In becoming more active in local affairs and assuming these new positions the learners will, in most cases, be required to use their literacy and numeracy skills in real situations (perhaps not on a daily basis, but certainly regularly and on an ongoing basis). This will help the learners to (see the value of, and) consolidate their literacy and numeracy skills - which is likely to mean that few will loose their skills through disuse. It will be interesting to document this is the medium term.
This close and dramatic inter-relationship between literacy and participation is perhaps one of the strongest arguments for the REFLECT approach.
As a final note on community participation it is important to consider the role of the facilitators. The REFLECT facilitators have been particularly active in the democratisation of COMUS itself. COMUS is planning to make itself accountable to the communities where it works by having community representatives sit on a "board of trustees" which will review the strategic direction of COMUS and approve plans and budgets. The literacy facilitators were the main activists in establishing meetings to set this process in motion. Interestingly this began to cause some tension within the present leadership of COMUS who initiated the democratisation process thinking that they knew who would be on the board (the traditional local leaders) but who have now found that communities are electing a new generation of leaders (either the literacy facilitators or the learners themselves). This has caused some unease as the COMUS leadership no longer feels fully in control of the process they initiated. There have even been accusations that the COMUS education promoters are organising a coup against the COMUS leadership.
In contrast to El Salvador very few learners have taken on formal positions of responsibility in community organisations. This may be because there is not the same culture of "organisation" as there is in the politicised communities of Usulutan. It may also be a question of time. The evaluation took place after just one year and there were no Resistance Council elections during that period.
However, the quality of community participation has been reported to have improved considerably. RCs have reported better attendance at village meetings, and a greater willingness to take part in development activities suggested by themselves. There is also a larger pool of people able to take on secretary's duties because they can sign their name on a cheque. Members of different village groups now see the importance of keeping records, minutes etc (which were rarely kept properly before). Over time, this must lead to increased accountability and transparency as everyone can check whether records are being properly kept.
One area of definite impact on developing new community leaders has been with the Parish Councils (established with two elected women from each village who are responsible for a budget devolved from ACTIONAID). Many of the women elected on to these councils were not literate and thus attended REFLECT circles. They are now able to manage money, record decisions and analyse local problems much more systematically than before. The REFLECT circles have also helped to ensure that these Councillors are accountable to the communities who elected them, consulting with the literacy circles and explaining (and having to defend) their decisions. In the future these practices should be widespread in all groups which will increase the democratisation of the communities. There is every likelihood that women who are parish councillors (and others from the literacy circles) will in future look to use their skills within the RC structure which has previously been very male dominated.
The significance of these changes in Bundibugyo should not be under-estimated. Previous to the period of the project it was very unusual for a woman to speak out before a mixed group of women and men. They were locked in a "culture of silence"; forbidden by tradition to express themselves. Now women speak freely in many group settings in church or the village meeting. The starting point has been the discussions in the literacy circles. Several women commented to the evaluation team that they are now able to organise their thoughts and prepare notes for what they wish to say in meetings - a practice in which they take great pride.
Again it is important to consider the facilitators themselves. Both formally and informally, they have already taken leadership roles (religious, school management committee etc.), and many express political ambitions for the next RC elections. Indications so far of their commitment to the welfare of the whole community show that their increased role will improve the quality of community life still more.
In the context of Bhola Island it is unrealistic to expect the women in REFLECT circles to have assumed active roles in community organisations. Other than their own shomitis, women are completely excluded from all existing organisations whether formal or informal. This will not change overnight. However, the seeds of change in the longer term have been planted with the REFLECT process. The increased confidence and active participation of women within the circles opens up new possibilities. The impact on resource management (section 4.6.4) is analysed later in this report and there are clear indications that women's position within existing structures is shifting as a result of the REFLECT process. In the short term the realistic focus is on greater involvement in household level decisions (rather than community decisions) and there is significant evidence of that shift taking place as outlined in the section on gender roles (section 4.6.5).