Assessing IK

Recognizing the general value of IK is one thing; assessing specific aspects of IK for application in development projects is another. To use IK in development we must be selective. Some IK is out-of-date, some is ineffective, and some is even harmful. We must, therefore, establish methods to assess IK, giving importance to the perspectives of both insiders and outsiders.

Assessing IK means identifying potentially useful IK and evaluating its effectiveness. It is necessary to do this in Steps 2 and 3 of project implementation (see How to use this manual and Using IK in development).

Many examples in this manual contain general ideas and advice on how and what type of IK could be improved or blended. For example, you could consider employing indigenous money lenders to implement credit schemes (see Using IK in development). Mini-case studies (in Part 4) describe how a traditional animal dispersal scheme was modified to help more farmers, and how insiders and outsiders together experimented in the germination of teak tree seeds.

Criteria for assessing IK

When assessing each type of IK—practice, technology, organizational structure, human resource, etc.—consider the following criteria:

Criteria

- Efficacy: Does it work? Is it effective? Under what conditions?

- Cost-effectivenees: Is it cost-effective? Affordable to poor people?

- Availability: Are its "ingredients", available in this location? In sufficient amounts? Decreasing?

- Understandability: Is it easy to understand? Easy to handle?

- Cultural appropriateness: Is it culturally appropriate? Will it be accepted? (These two questions apply only when IK from one location is introduced to another location, ethnic group or caste.)

- Effect on different groups in communities: How will it affect the different user- and non-user groups in the village? (Who would be burdened? Who would benefit?)

- Environmental soundness: How does it affect the environment?

- Constraints: What are potential constraints to its use or application? Can they be overcome?

These criteria resemble those applied to western knowledge. Due to its special nature, however, the measurement standards for IK might have to be distinct from those applied to western knowledge.

Special characteristics of IK

IK is holistic (ace Characteristics of local systems).

Indigenous systems are often complex, their various components interrelated. This makes it difficult to measure their efficacy or economic return accurately. To overcome this, western science has tended to pick only bits and pieces of local systems for comparison with their western counterparts. For example, yields of local crops were compared to those of improved western varieties. The fact that the local crops were well adapted to specific intercropping arrangements was often ignored despite the fact that total economic return from some intercropped fields is higher than that from improved monocrops.

Western science has been slow to develop methods to assess complex systems. Father than measuring the yields of single crops, we need methods which can measure economic returns of intercropped fields over extended periods. To assess the productivity of particular livestock species, we need methods which take into account inputs— coat of feed, medicines and labor. Up to now, analyses have focused on outputs—milk production and meat production—and neglected the benefits of local breeds which thrive on minimal inputs.

The value of some IK cannot be expressed in monetary terms.

Some practices fetch low economic returns but perform valuable social functions. Other practices which seem less effective than outside technologies might preserve the environment—a benefit that is difficult to express in economic terms. In other words, assessment of
IK must recognize the context in which it was developed and in which it is applied.

Adapting measurement standards to accommodate the special nature of IK is not enough. We must also identify the criteria and standards by, which local people themselves judge IK. This can be difficult.

We can, however, find out:

- What people value most in a specific IK
- Why they chose it
- What they See as its strengths and weaknesses
- What they think would happen if the IK were not available
- Who would be most affected if the IK were not available
- What features people look for when they teat a technology, and 50 on.

In other words, we attempt to learn the people's view of IK. Methods described in this manual can be adapted to this purpose.

Only if we combine both insiders' end outriders' assessment, will we be able to identify and better understand the value and usefulness of IK.

Compiled by Evelyn Mathias

Tapping assessment

Outsiders can learn about the advantages and disadvantages of indigenous practices from local people themselves. This not only provides valuable information about IK, but also helps identify opportunities for—and constraints to—the promotion of specific practices.

While some require adaptation, many of the methods for recording IK described in Mart 2 can also be used to assess IK:

- Strengths and weaknesses and SWOT analysis can help insiders and outsiders alike assess the advantages and disadvantages of a particular item of IK in solving problems.

- Participative technology analysis can show the uses of a practice and how it might be modified or promoted.

- The five question technique can reveal whether a practice is helpful, harmful, or neutral, and therefore whether it should be promoted.

- Village reflections enable villagers themselves to analyze and make decisions as a group on complex issues facing them.

- Individual interviews or group discussions can reveal the reasoning behind an indigenous practice or technology. They can also reveal how villagers assess the effectiveness of specific IK practices or technologies.

- A series of resource maps drawn by local people can help gauge the effect that indigenous practices have on the environment.

- Historical comparison can suggest why a specific item of IK has changed and suggest possibilities for improvement.

- A matrix showing characteristics of various practices or species can highlight their relative strengths and weaknesses.

- A flowchart listing the steps of an indigenous practice can focus discussion on the problems encountered at each step, and in the process help solve these problems.

- A web can help reveal causal relationships between a specific IK and other factors. This can help determine whether to improve a specific IK, and if 50 where to start.

- Sorting and ranking allow comparison of different IK practices and technologies and can help identify which of them are most effective.

What is true for recording IK is also true for assessment: the more participation a method provokes, the more the output will reflect the people's views and experiences.

Another possibility is to study how local people themselves assess their knowledge. Some communities or regions have specific bodies that meet, discuss, and decide on practices. In Bali, Indonesia, for example, regional networks of "water temples" manage and regulate water distribution for rice cultivation. The network would be a valuable source of information for anyone wishing to study and evaluate the local irrigation system.

And, local people do experiment. They might plant a new tree species in their home gardens to check its performance, they might test herbs for new medicines, or try some new cooking ingredients. Their experiments range from just trying something new, to deliberate, systematic experimentation (for further information on indigenous experimentation, See Boef et al. 1993). Outsiders can learn a lot from these community experiments and build on them when testing technologies. Assessment of IK is more than observing indigenous practices it means looking at the various processes and structures through which IK is generated, shared, tested, and applied.

Compiled by Evelyn Mathias

Using western science methods to assess IK

First, by "western science methods" we mean methods used by western science to develop and test technologies, methods, or practices. For example, soil sample tests, measurement. of animal feed intake, or blood tests to monitor the effects of certain drugs.

But IK can be different IK is holistic. it can be difficult to differentiate into many subject matters, each treated separately by western science. And, to attempt to describe all western science methods which could be used to assess IK would be impractical. Instead, we will highlight some principles and give a few examples

Principles for assessing IK with western science methods

- As with all research, the selection of western science methods for assessment of IK should be based on objectives defined before the assessment (see Recording IK in communities).

- The assessment needs to be based on a throough understanding of the IK to be assessed.

- The experimental design should do justice to the special nature of IK (e.g.,, recognizing its holistic nature, not purely economic benefits, etc.).

- Insiders' assessment should complement western science methods.

- IK should be viewed in the broad context of culture, society, and history.

- We must recognize the limitations of western science for the assessment of IK in order to interpret our study results correctly.

1 Western science methods can lead to false conclusions when used to assess IK (see Criteria for assessing IK).

2 Western science, lacking the means to understand an indigenous practice or technology, might belittle it. A classic example is acupuncture. For a long time western science had no explanation for acupuncture and therefore disregarded it. This is changing and acupuncture is being integrated into western medicine's curricula.

Examples of western science methods used to assess IK

The following are some examples of western science methods which could be used to assess IK. This list shows that approaches developed in different disciplines can be used. Keep in mind that these methods should be combined with insiders' assessment.

Animal Production and healthcare

Let's suppose that a community wishes to expand and improve its livestock production system. The following western science methods could determine the efficiency of local animal production and healthcare practices and indicate which aspects of the indigenous system could be used, improved, or blended with western practices:

- Measure productivity of animals, recording both inputs and outputs (ace Criteria for assessing IK).

- Observe the condition of livestock kept in the community (this could be done by ocular inspection, weighing and measuring animals, etc.).

- Test for Parasites by investigating feces of randomly selected animals (this will require some laboratory tests).

- Identify medicinal plants used by the community and test their efficacy. The medicinal qualities of some plants have already been established in the scientific literature.

Indigenous paper making

- Calculate amount of raw materials and energy used in the production process.

- Test quality of paper in the laboratory (do not forget to keep the local use in mind when making any statement about the paper's quality).

Effect of IK on environment

- Assess biodiversity in the environment of the study community (e.g., count number of species in an area of a certain size).

- Measure nutrients in soil.

- Measure runoff and soil erosion from fields.

Indigenous birth attendants

- Collect data about course and outcome of deliveries assisted by indigenous birth attendants and analyze results using statistics.

- Investigate condition of instruments used by local birth attendants (e.g., whether the instruments are clean, which bacteria they contain, etc.).

Indigenous communication

- Assess number of Persons reached by messages transmitted through indigenous channels.

- Measure time needed for transmission.

Compiled by Evelyn Mathias

Monitoring and evaluation

Monitoring is the regular collection of data on an activity, technology, social event, relationship, or some other topic. Evaluation is the analysis of these data, comparing them to set objectives. For example, monitoring and evaluating (M&E) the use of herbal medicines in a small community of some 20 households could be done through regular visits to all households, recording whether any of the household members had been sick since the last visit, whether anyone had used any herbal medicines to treat an illness or for any other purpose, whether they used any other type of drug, etc. After, let us say one year, the data would be analyzed. We might count how many households and who in each household had used herbal medicines, how many different types of herbs were used and how they were prepared, for which diseases herbal medicines were used, etc. It is important that we determine, before the monitoring starts, just what data are needed and how they will be analyzed.

M&E enable development workers to determine whether their projects are meeting their objectives and to determine what changes are needed.

Monitoring and evaluating IK

Monitoring and evaluating IK means regularly collecting data on a specific IK and analyzing the data against defined objectives.

Examples

Monitoring and evaluation

Many of the recording and assessment methods described earlier in this manual can be adapted and used to collect data for monitoring and evaluating IK. Combinations of methods afford a comprehensive analysis, allowing assessment from the insiders' end outsiders' perspectives. Such a comprehensive assessment is also useful for cross-checking and validating data through triangulation (i.e., ask the same question in different forms or ask different people).

For example, let's look at a project promoting indigenous stoves to villagers. To obtain technical data on the stoves, the project might measure the stoves' energy consumption, smoke emission and cooking efficiency under village conditions during different seasons. Parallel to this testing based on western science, regular group discussions—or some other IK assessment method—would be held to learn the villagers" experiences using the various stoves: What problems were encountered with the new stoves? Which stoves meet the villagers' needs? How can village stoves be improved, etc?

What applies to M&E activities in general, also applies to M&E of IK. We must determine from the outset of a project:

- What do we want to measure? What are the objectives of our planned M&E?
- What type of data must we collect in order to learn whether we have reached our objectives?
- What methods are best suited to collect the data?
- What methods are best suited to analyze and interpret the data?

Using IK to monitor and evaluate projects

IK can be used to monitor and evaluate projects. For example, hunters might know that the disappearance of a certain wildlife species means that a certain habitat is deteriorating. A project aiming to improve this habitat could use this information. By monitoring this indicator species, the project could gauge the effectiveness of its efforts. Or, local people might have their own way of calculating profit. A project aiming to increase the number of local enterprises in a village could use this indigenous method of record-keeping to monitor and evaluate enterprise success.

The use of IK in M&E is poorly documented. Therefore there exists no easy "recipe." But similar to the steps outlined in the sections How to use the manual and Using indigenous knowledge in development, applying IK to M&E should start with thorough documentation of any IK related to the objectives of the M&E. For example, if you want to monitor changes in the condition of the environment, make a comprehensive record of IK relating to the environment. Next, screen the recorded IK looking for any information useful for your M&E.

Or, you could ask the local people how they would monitor progress of the project. Their ideas could be very useful. You could develop an M&E approach together with them.

Finally, identified IK must be integrated into your M&E design. Again, a combination of both IK and outsiders' knowledge will probably prove most effective.

Compiled by Evelyn Mathias