|Boiling Point No. 23 - December 1990 (ITDG Boiling Point, 1990)|
by Fred Swartzendruber, Sudan Renewable Energy Project, Energy Research Council
A market study is not an academic research project, but a management tool intended to provide informed projections about demand for a given product, and to identify trends which might positively or negatively affect these in the future.
Market research in industrialised countries is able to take advantage of a wealth of census data and other statistic and normally does not require extensive field work. Knowing the age, sex, income, and geographic distribution of the population, researchers are able to pinpoint target groups and to conduct fast, inexpensive telephone surveys, often asking no more than four or five questions per interview. Test marketing of products is done in locations and within population groups known to be typical of the larger target population. These sampling frames make the job of market analysis much more efficient and economic than if researchers had to generate the preliminary data for themselves.
In developing countries one generally finds that demographic and economic statistics are far less available. In the absence of reliable sampling frames, market researchers are faced with the difficulties of having to create their own. Furthermore, the primary tools of modern market research, automated telephone systems and computerized direct mail surveys which randomly select telephone numbers or postal codes, are not feasible in most developing countries. Often researchers are forced to use door to door surveys, the most expensive and time consuming technique available. But it is important to keep in mind that regardless which survey technique is used, if the sampling frame problem arises from a lack of basic demographic data, the reliability of findings will inevitably be in doubt to some degree.
The most common response to the problem is to increase the size of survey samples, hoping that this will result in reduced sampling error and greater reliability of the findings. Yet if samples are not selected systematically, increasing the number of cases will not necessarily improve the quality of the results. Margins of error or confidence levels are usually reported in surveys, but it is sometimes forgotten that these apply only to random sampling variation, and have nothing to do with other sources of error or bias in the collection of data.
Apart from increasing the sample size, what can be done? One solution is to devise more systematic methods for selecting the sample that is surveyed. Random population surveys are very difficult to achieve, given the poor base of demographic data. In such cases one often ends up carrying out an area sample rather than a population sample. Strictly speaking, if one does not have some form of population list, one cannot randomly sample the population. However, one can sample within geographic areas. Knowing the number of houses in an area, one can draw a sample representing a given one household per dwelling. This will provide a representative sample of households. Although an area sample is not precisely equivalent to a population sample, for many purposes the difference may not be significant, and sometimes one may be more interested in households than in individuals; this is true of the Improved Stoves Market Study because stoves are bought for households rather than for individuals. In developing countries the number of dwellings in the area to be surveyed is often not known, thus a sample based on a fixed proportion cannot be drawn accurately. Adequate maps may be difficult to locate, and rapid and unplanned urbanisation may make existing maps obsolete. In some cases satellite imagery or aerial photographs can be used in place of maps, although this requires special skills and equipment for analysis. Often a survey will be carried out in an area chosen because it is believed to be roughly similar to other areas, because it is convenient (i.e., provides easy access to enumerators), or because the area is familiar to the designers of the survey. Then, in order to compensate for the sampling problem, a large number of cases are selected; sometimes every household within a specified area will be surveyed. The basic problem with this approach is that it is not necessarily a representative sample of the larger area. Perhaps it is typical in some respects, but atypical in others. This being the case, increasing the size of the sample would not necessarily increase the reliability of the findings when they are extrapolated to the population at large. In other words, a survey carried out in one neighbourhood is not necessarily representative of other neighbourhoods, and enlarging the sample size will not change that fact. The improved stoves market study for the Khartoum area found very little useful data on which to build. Existing census figures were out of date, and no income distribution data has been collected within the last ten years. For these reasons, an area sampling approach was selected, and efforts made to acquire the best maps available. The Survey Department was hired to provide a large-scale map of the Khartoum area, which covers roughly 600 square kilometers, and identifies some 260 sub-municipal areas or districts. These areas were numbered, and certain non-residential areas such as the airport and several cemeteries deleted from the list. Military bases were also removed, as surveying would not have been permitted in those areas. From the resulting index of 253 areas, which we will term "neighbourhoods, a sample of 30 percent (77 areas) were selected using a computerised random number generator.
On a 'Three Towns' basis, the 77 neighbourhoods selected include 124 (40.3%) in Khartoum, 79 (25.6%) in Khartoum North, and 105 (34.1%) in Omdurman. Each of these 77 neighbourhoods was subdivided into twelve blocks, providing a total of 924 blocks in which households would be interviewed. Four blocks were selected at random from each of the 77 neighbourhoods. The survey plan was to interview one household in each block, for a total sample of 308. This provided a reasonable sized sample of neighbourhoods distributed randomly throughout the greater Khartoum area. It appears that the areas were originally demarcated to some extent on the basis of population, perhaps for the purpose of establishing representation for local councils. In general, neighbourhoods in Omdurman and in the central areas of Khartoum are quite small and numerous, while outlying areas are much larger. This accounts for the lower representation of Khartoum North, with fewer areas than either of the other two towns. One implication of this is that the area sample is to some extent weighted by actual density of housing and thus by density of population. However, this is also likely to be offset by in-migration in recent years into such areas as Haj Yussif, which are now heavily populated. These are probably under-represented in our sample, though not completely excluded. Because population and housing density data were not available, one cannot be completely sure that all households had an equal chance to be selected. It is likely that larger houses built in lower-density neighbourhoods would be picked at a higher rate than their true proportion in the population, and that the very small, crowded shelters found in displaced areas would be significantly under-represented. Without reliable census data it would be difficult to attempt to adjust for this factor. This point is significant given that a larger number of displaced and refugees live in such neighbourhoods, a population which forms a larger but thus far unquantified proportion of the total Khartoum area population.During the designing of the survey methodology, attempts were made to get the best available estimates for the base population as well as the displaced and refugee numbers in the Khartoum area. The variation in estimates was extremely large, ranging from 3 million to as high as 7 million total population. Most estimates, however, ranged between 4 and 5 million; given a household size of 6 (from 1983 Census data) this would imply that the total number of households in the greater Khartoum area is between 600,000 and 833,000. A 0.5% sample, which had been suggested, would therefore have implied interviewing some 4,000 households, an unrealistic effort given staff and transport constraints. In addition, given the lack of accurate population and housing data, this large sample would not necessarily provide a truly representative picture of the total Khartoum population. In other words, it might provide very detailed information about the same segments of the population who have usually been included in surveys, and exclude those who are most difficult to reach. Without a major change in the way survey samples are carried out, increasing their size would not be likely to provide additional benefits in line with the added cost.