Research design for stage 1

The comments here on research design are not intended to take the place of meeting with a statistician to work out the design best suited to the research. They are tips to fellow investigators based on experience in the field and in data analysis.

Design

For very low cost studies, a retrospective, case-control design probably is best, comparing wellnourished children with malnourished children matched for age. Such studies may either be indepth small-sample enthnographic investigations or larger sample cross-sectional surveys. In either case, they should use forms that combine questions to the child's caretaker with direct observation of the environment and of caretaking behaviours. The advantages of the retrospective case-control model is that the data can be collected and analysed rapidly, by hand if necessary. The design is simple and clear, so that the purpose of the study and its results will be clear to everyone involved. Such studies should yield an understanding of important factors associated with positive deviance. As noted earlier on pages 81-82, they will not reveal all the information it would be useful to know.

The next step up, at a slightly higher level of expense and sophistication, is a crosssectional study design that covers a representative sample of all of the children in the age-group and the socio-economic groups of interest. This model tends to take the form of a survey of at least 80 to 100 households, because the sample needs to be big enough to make statistical comparisons. It also requires an observational component to obtain behavioural and environmental data. One advantage of including children whose nutritional state is average is that it becomes possible to describe the full distribution of values within the population. The descriptive information can also be used as a baseline for designing interventions. Collecting a full continuum of values also permits multivariate regression, which is the analytic procedure most frequently ape plicable to nutritional data.

Longitudinal studies are desirable but many times more expensive and time consuming than cross-sectional designs. They require advanced computer capabilities for analysis. Funding agencies and researchers frequently do not take into account the fact that collecting and analysing six months' worth of data may cost up to ten times as much as the amount required to conduct and analyse a cross-sectional survey that collects the same information only once. Reasons for the additional expense include: extra time spent in designing longitudinal measures; the need for a long-term field office and full-time staff; and extra time and expertise needed for all aspects of the data analysis, as will become apparent in the section on measuring growth (p. 97).

Qualitative versus Quantitative Designs

Qualitative methods are needed to identify the types of behaviours and social networks that exist in households and communities. These methods include long openended interviews and observations as well as group interviews, known as focus groups. A problem is that results of qualitative studies tend not to be replicable in a strict scientific sense, and do not provide descriptive statistics. Policy-makers frequently do not regard qualitative results alone as sufficient evidence to justify a programme.

Therefore, it is usually desirable to combine qualitative and quantitative methods in positivedeviance studies. While this can be done in a variety of ways, the simplest and least expensive way to combine qualitative and quantitative research is to start with a three-week qualitativeresearch module, using a rapid-appraisal methodology (Scrimshaw and Hurtado, 1987). The results of the rapid appraisal are written up on their own. Simultaneously, the rapid-appraisal exercise functions as the design phase for developing and pre-testing the questionnaires and observational protocols to be used for quantitative-data collection.

For example, this qualitative module may have the following steps:

1. The research team develops a conceptual framework and several lists of potentially interesting variables (see pages 88-106), by reviewing the literature and discussion with other professionals.
2. Research-team members take these lists into the homes of well-nourished and malnourished children. In the village, they walk from door to door with village guides. Using arm-circumference strips to select equal numbers of well-nourished and malnourished children, they visit the homes of 10 to 20 of each.
3. Inside the home, they enter into friendly conversation with the child's caretaker. During this conversation they either ask or attempt to directly observe information about each item on the lists, writing their results into notebooks. They also look for new aspects of family life that may affect nutrition and add these to the lists.
4. The research team meets to analyse its findings together. The principal investigator summarizes the results for each item on the lists on a blackboard, eliminating all items that were found to be less important and adding new items discovered in the homes. New revised lists are recorded.
5. The items on the new lists are drafted into focus-group question guides.
6. Research-team members use the question guide to conduct focus-group discussions separately with groups of village women, village men, and health workers, recording the results in their notebooks.
7. The research team meets to discuss the focus-group results. This discussion is recorded
8. Information recorded from the group discussion and in the team members' notebooks is used to write the report of the qualitative-research findings.
9. The same information is used to draft pre-coded questions and observational protocols for the quantitative survey.
10. The survey instrument is pre-tested and revised.

Sample Size and Unit of Analysis

Observational methods are frequently too resource-intensive to permit the collection of data from large samples. In basic ethnographic studies, for example, one observer may cover only 20 households per year (20 working days per month; one day per household per month). A full year may be needed in order to record seasonal changes and the sequential development of the child.

An obvious answer to the problem of insufficient sample size is to turn to behaviours or events as units of analysis. Instead of 25 infants, for example, one may wish to analyse the 350 feedings received by these infants during the observation period. Moreover, the feeding may be the theoretical unit of interest if one is studying the effects of the interactions or the amount of food eaten during the feeding. In moving to events or behaviours as units of analysis, it is imperative to involve a statistician to figure out whether the procedures used are statistically legitimate. Unfortunately, feeding events may not be stable units of analysis. Behaviours summarized at the event level in the Mexico study turned out to be highly non-normal in distribution (Zeitlin and Johnson, in progress). Different statistical methods used to cope with their non normality tended to yield different statistically significant results. Thus, the behaviours were finally summarized and analysed at the child level despite the small sample size.

It is also important to distinguish between statistical significance and biological significance. In the same Mexico study, the fact that the 25 well-nourished infants received on average oneand-a-half more bottles of cow's milk per child per day than the 25 malnourished was not statistically significant, but could well have been biologically significant.

Accuracy and Replicability of Observational Research

Replicable observational research must be based on structured methods of observation. Developmental psychologists and anthropologists have devised and tested numerous formats for structured observations. A time-saving strategy is to adapt and pre-test some of these formats during the same period when the qualitative research is being conducted. Once these new instruments have been developed, it is essential to test them for internal reliability and validity.