APPEAL - Training Materials for Continuing Education Personnel (ATLP-CE) - Volume 4: Quality of Life Improvement Programmes (APEID - UNESCO, 1993, 95 p.): Chapter 6: Monitoring and Evaluation

APPEAL - Training Materials for Continuing Education Personnel (ATLP-CE) - Volume 4: Quality of Life Improvement Programmes (APEID - UNESCO, 1993, 95 p.)

		(introduction...)
		Foreword
		Introduction
		Chapter 1: Principles and Rationale
		Chapter 2: Programme Framework
		Chapter 3: Organization and Implementation
		Chapter 4: Materials and Resources
		Chapter 5: Training of Personnel for Quality of Life Improvement Programmes
		Chapter 6: Monitoring and Evaluation
		Chapter 7: Challenges and Issues
		Annex: List of Participants

Chapter 6: Monitoring and Evaluation

A. NEED FOR MONITORING AND EVALUATION

All development projects need a built-in system for monitoring and evaluation (M&E). Projects aiming to make quantitative and qualitative changes of the life of the intendent beneficiaries need them most. M&E for QLIP-CE is essential because such an exercise:

i) examines whether institutional, administrative and material support is adequate for conducting the programme;
ii) provides information to the project planners and managers whether the project is being implemented in accordance with its plans and objectives towards achieving its intended goals.
iii) suggests (on the basis of ii above) mid-course corrections, modifications and improvements, if any, rather than waiting for the final outcome.
iv) contributes to national documentation for wide-scale expansion/replication of the project and formulation/implementation of similar development projects.

M & E activities of an educational development project are also important because one may like to analyze the situation and the context from where the project takes off, and what inputs and process are needed in order to achieve certain project outcomes and goals.

In the specific case of QLIPs it is important to assess the immediate need and to evaluate the present state of development in relation to that need. It is important, too, to establish a monitoring system to check on progress towards achieving development targets and to evaluate the final achievement and its impact on the community.

A critical aspect in the case of QLIPs is to identify the element of quality of life to be developed, to identify precise indicators and to formulate measures of the state of development of each indicator. For example the general element may be availability of food, the specific element, the availability of rice, the indicator, the crop yield and the quantitative measure the percentage increase in production.

The programme of development should be set into a time frame with carefully defined development steps (sub-goals) defined at set intervals. (See Chapter Two) Monitoring should aim to overcome problems in the achievement of the sub-goals.

B. STAGES OF MONITORING AND EVALUATION

As in other projects, M & E for QLIP-CE should be conducted in three stages:

Stage One:	Pre-implementation (Benchmark survey)
Stage Two:	On-going Assessment (Monitoring)
Stage Three:	Assessment of Project Performance (mid-term and/or final evaluation)

The M & E stages are summarized in Figure 6.1.

Figure 6.1: Stages in Monitoring and Evaluation of a Development Programme such as QLIP

C. RESPONSIBILITIES OF MONITORING AND EVALUATION

Many different individuals or agencies including NGOs may be involved in M & E activities. However, a Committee in each of the three levels of operation may coordinate the M & E activities in each country. Table 6.1 gives the levels and the possible compositions of the M & E Committees.

Table 6.1: Levels by M & E Committee

Level	M & E Committee^*
(A) National	A.1 Department of Education/Non-formal Education A.2 More specialized agencies (e.g. National Academy for Educational Management, Bangladesh) A.3 (Country) National UNESCO Commission/Intergovernmental Organizations (e.g. APDC,^ CIRDAP^, etc.) A.4 Major NGO representatives A.5 Others (nominated by government)
(B) Slate/Province/District, etc.	B.1 Department of Education/Non-formal Education (of that level) B.2 Local Government representatives B.3 Private Sector/People's Organization/NGO representatives
(C) Local (villages)	C.1 Local government/village council (e.g. JKKK^** in Malaysia) C.2 Grassroots NGOs/People's Organizations C.3 Learners' Group representatives (Participatory approach in M & E)

^*Suggested Composition; National Government may consider restructuring.
^**APDC: Asia-Pacific Development Centre
^**CIRDAP: Centre on Integrated Rural Development for Asia and the Pacific
^**JKKK: Village Development and Security Committee

The national Continuing Education Committee headed by the representative of the Department of Education/Non-Formal Division may coordinate activities of all the three-level committees. The National Committee may analyze data and compile reports of all the three committees. (See Chapter three).

D. HOW TO CONDUCT MONITORING AND EVALUATION

A number of M & E formats may be available within Department of Education, Social Services, Community Development, Women Affairs etc. within a country. One closely related format may be suitably adopted for the M & E purpose of QLIP-CE. However, the adopted format should be very simple in terms of words/language and length. The format must be available in the national/local language and easily understandable by the programme managers as well as the learners.

An M & E format may be worked out in a joint consultation of the members or representatives of M & E Coordination Committees. The agreed format may be printed and be made available by the NCCCE or the PCCCE. A brief orientation to the M & E format may be introduced in training courses being attended by all the QLIP-CE personnel. They may in turn train the learners in how to fill-in the format.

Variables to be incorporated in a M & E format are suggested in Tables 6.2, 6.3 and 6.4.

Table 6.2: Pre-implementation (Bench mark Survey)

Variable(s)		Question/Determinants
(A) LEARNERS
	1. Geography	· Rural/Urban
	2. Age-Group	· Most of ihem belong to (e.g. 25-30 years)
	3. Occupation	· Most of them are (e.g. landless labourers, fa workers, housewives, etc.)
	4. Income level	· Below poverty line (very poor, poor) · Above poverty line (Comfortable, Affluent)
	5. Basic education	· Five years' formal education · Non-formal education · Neo-literates · High-school graduates · Others (specify..)
	6. Attitudes to CE	· Good towards attendance at CE courses · CE does not make any difference · CE not necessary at all
	7. Reasons for participation in CE programme	· Specify up to three
(B) PROJECT OBJECTIVES/INPUTS
	1. Project duration	· Unit Courses · Period (in months)
	2. Project objectives	· List all of them as provided in the project document/records, etc.
	3. Number of learners (as targeted)	· Total No.
			- Phase I
			- Phase II
			- Phase III
	4. Dimensions (as envisaged)	· Biological · Social · Economical · Humanistic · Environment (Tick Mark, where appropriate)
	5. Specific indicators	· List and tick
	6. Measure of present level of development	· Show % or other measure
	7. Level of development to be achieved	· Show % or other measure
	8. Time frame and sub-goals	· Sub-Goal ____________ ____________		Date to be Achieved _________________ _________________
	9. Percentage of Project budget provided for initialing income generating projects (IGPs)	· Nil · Specify			.... ....
	10. Technical Support from other agencies (as envisaged)	· Nil · Sought from Government agencies · Sought from NGOs/people's organizations			.... .... ....
	11. Material support from other agencies (as envisaged)	· Nil · Sought from Government agencies · Sought from NGOs/people's organization			.... .... ....
(C) INSTITUTIONAL/ADMINISTRATIVE
	1. Proponent of Project	· GO/NGO/People's Organizations			....
	2. Implementing Agency	· Proponent itself · Other than the proponent			.... ....
			- GO
			- NGO
			- People's Organization
	3. Sponsor of Project	· Proponent itself · Other than the proponent			.... ....
			- GO
			- NGO
			- People's Organization
	4. Total budget (in all phases)	· Amount (Local currency)			....
	5. Technical/material support coordination with other agencies	· Nil · Sought from other agencies			.... ....
			- GO
			- NGO
			- People's Organization
	6. Staff to be involved in the Project execution	· Total No.			....
			- General		....
			- Technical		....
	7. Staff training	· Training by government agency · Training by NGO			.... ....

Table 6.3: On-going Assessment (Periodic Monitoring)

Variable(s)	Question/Determinants
(A) LEARNERS
1. Increased Knowledge (IK)/More participation (MP)
				IK	MP
	· Biological			....	....
	· Social			....	....
	· Economical			....	....
	· Humanistic			....	....
	· Environment			....
	(Tick mark, where appropriate)
2. Change in occupation	· Percentage of learners
		- Have changed their occupation							....
		- Toward occupation requiring higher skills							....
3. Participation in IGPsIncome Generating Programmes (IGPs)	· Activities (IGPs)			Nos		%
	____________			___		___
	____________			___		___
4. Net benefit from participation in IGPs	· Amount _________ (Local currency)
5. Change in income	· Percentage of learners whose income have changed from the baseline ones (....) %
6. Learners' perception about the flow of budget for IGPs and technical materials	· Flow has been satisfactory							....
	· Flow has been discontinued							....
	· Others (specify......)							....
7. Learner's perception about the beneficial aspects of the project	· Specify at least three of them
	__________________________
	__________________________
	__________________________
(B) OBJECTIVES/INPUTS
1. Objectives	· Percentage/proportion of project objectives achieved						.....%
2. Learners	· percentage of target learners being covered						.....%
3. CE-Course	· Percentage of CE-Courses being taught						.....%
4. Dimensions	· Learners have taken courses in the following areas:
		- Biological					........
		- Social					........
		- Economical					........
		- Humanistic					........
		- Environment					........
	(Tick mark, where appropriate)
5. Level of competence	· Learners have achieved competence leading to:
		- More awareness of immediate environment					.....
		- More participation in social and economic life					.....
		- More appreciation of values, ethics, loyalty, etc.					.....
		- Better quality of life					.....
		- Others (specify.....)					.....
	(Tick mark, where appropriate)
6. Budget utilized for IGPs	· What percentage of the budget is utilized for IGPs						.....%
7. Plan to expand the project to cover more number of learners	· Yes/No
		if Yes, what number					.....
			when				.....
8. Course instructors	· More course instructors available
		Yes/No					.....
9. Training of the Course instructors	· Are the course instructors being trained
		Yes/No					.....
		if yes, what competency					.....
		(specify......)
(C) INSTITUTIONAL/ADMINISTRATIVE
1. Inflow of project budget	· Satisfactory						......
	· Not always satisfactory						......
	· Hardly flows						......
	· Others (specify..)						......
2. Technical assistance from other agencies (GOs/NGOs/People's Organizations)	· Being regularly received						......
	· Sometime received						......
	· Not received at all						......
3. Material supplies	· Satisfactory						......
	· Not always satisfactory						......
	· Hardly available						......
	· Others (specify......)						......
4. Contact/Coordination with other agencies (GO/NGOs/People's Organization)	· Satisfactory						......
	· Not always done						......
	· Hardly being done						......
5. Target achievement	· Percentage of learners covered under the programme						......%
6. Receipt of completed M & E Formats	· Regularly received						......
	· Received some lime						......
	· Not received at all						......

Table 6.4: Evaluation of Inputs - Process - Outputs (Midterm and Final)

Variable(s)	Question/Determinants
(A) LEARNERS
1. Objectives	· Were relevant to the learners' context:
		- Highly relevant	......
		- Moderately relevant	......
		- Irrelevant	......
		- Other (specify...)	......
2. Inputs	· Input were relevant to project objectives:
		- Highly relevant	......
		- Moderately relevant	......
		- Irrelevant	......
		- Other (specify...)	......
3. IGPs	· Were instrumental to raising learners" socio-economic standards of life
		- Highly relevant	......
		- Moderately relevant	......
		- Irrelevant	......
		- Other (specify....)	......
4. Quality of Life	· Learners found the project useful for improving their quality of life
		- Highly relevant	......
		- Moderately relevant	......
		- Irrelevant	......
		- Other (specify...)	......
5. Role of Learners	· Learners encouraged others to participate in the project
		- Yes	......
		- No	......
(B) OBJECTIVES/INPUTS
1. Fulfilment of objectives and sub-goals	· Largely fulfilled		......
	· Moderately fulfilled		......
	· Fulfilled to a smaller extent		......
2. Technical inputs	· Satisfactorily available		......
	· Moderately available		......
	· Poor		......
3. Material supplies	· Highly satisfactory		......
	· Moderate		......
	· Poor		......
4. Further inputs (Technical and materials)	· Knowledge generated about further source/quality of inputs
		- Yes	......
		- No	......
5. Local inputs (Technical materials)	· Local inputs generated
		- Yes	.....
		- No	.....
		If Yes, specify
		1.________________________
		2. ________________________
		3. ________________________
(C) INSTITUTIONAL/ADMINISTRATIVE
1- Budget/resources	· New source identified (specify.....)
		1. ________________________
		2. ________________________
		3. ________________________
2. Participation of more agencies (GOs/NGOs/People's organizations)	· New agencies identified (specify....)
		1. ________________________
		2. ________________________
		3. ________________________
3. Coordination among GOs/NGOs/People's organizations	· Yes		.....
	· No		.....
	(if yes, specify with whom....)
		1. ________________________
		2. ________________________
		3. ________________________
4. Target achievement of the Project	· Yes		......
	· No		......
	(if no, specify why.....)
	1. ________________________
	2. ________________________
	3. ________________________
5. Receipt of completed M & E formats, analysis of data and report compilation	· Adequately attended		......
	· Not attended		......
	(if not, specify reasons)
		1. ________________________
		2. ________________________
		3. ________________________
(D) IMPACT OF QLIP-CE
1. Target fulfilment	· Yes		....
	· No		....
2. Encouraged new learners to participate	· Yes		....
	· No		....
3. Created awareness/demand at various levels for initiating more QLIP-CE Projects	· Yes		....
	· No		....
4. Recruited many more agencies to launch similar projects	· Yes		....
	· No		....
5. Established better co-ordination mechanisms among agencies dealing with CE	· Yes		....
	· No		....
6. Created more documentation about the real State-of-the-Art of CE	· Yes		....
	· No		....
7. Created sustainable impacts on the learners' life in terms of their greater participation in socio-economic activities, employment, increase in income and resource generation	· Yes		....
	· No		....

E. CONCLUSION

This Chapter has provided guidelines for M & E activities of QLIP-CE Programmes. The guidelines and procedures outlined above are indicative only. A NCCCE may consider modifying the guidelines depending on the particular country situation. An agreed M & E formal may be applied in both short-term and long-term studies. The government through its NCCCE may also consider the design of other instruments for M & E activities of similar development projects keeping in mind the specific QLIP-CE Projects.

M & E is important in this area not only because the approach discussed in this volume is relatively new and untried in many countries of the Asia-Pacific region but also because such M & E will contribute to the overall process of similar M & Es to be conducted with respect to other educational programmes. A meticulous approach to M & E of QLIPs project will be supportive in the following ways;

1. Assessment of the impact of QLIPs on development
2. Establishment of a nation-wide Management Information System (MJS) to support QLIPs
3. Documentation of Non-formal Education