|The MUSTER Synthesis Report - Researching Teacher Education: New Perspectives on Practice, Performance and Policy (CIE, 2003, 237 p.)|
|CHAPTER 2: DOING THE RESEARCH|
MUSTER used a wide range of research methods, drawing on both quantitative and qualitative techniques of data collection and analysis. This was designed to gain a full a picture as possible of the complex issues. There were also cultural reasons for using a variety of approaches. As one principal researcher put it:
.... The socio-political values of African societies and the attitudes they promote present those researching into the lives and experiences of its people with a different set of methodological challenges. This reality required the picking and choosing of a combination of research approaches, across what is often seen as the methodological divide of qualitative and quantitative research [....] to enhance the validity of findings and their interpretation. (Akyampong et al 2000: 4)
There was a tension between collecting data that would be directly comparable across sites using common instruments, and wanting to respond to the local situation in different ways. The process of designing instruments collaboratively was commenced at the first workshop (March 1998). The Sussex team subsequently drew up research memoranda for the different strands, containing draft instruments that could be adapted locally. In the case of survey data, draft instruments were developed which had a high degree of commonality. This was largely retained after piloting, though in some cases particular wording was modified to reflect local variations in expression and interpretation to enhance construct validity. There was more variation in the methods used to acquire qualitative data, though broad themes were agreed which were then explored through interviews, focus groups etc. These varied in the detail of their realisation.
The main data collection methods, how they were applied, and any special features that were employed to meet specific circumstances are indicated below. Where appropriate we link methods to particular strands and arenas. More detailed accounts of the methods used in each case are available in the relevant Discussion Papers.
Questionnaires were used to collect data on student teachers at the points of entry to and exit from their training course, and from Newly Qualified Teachers (NQTs). The Entry Questionnaire contained sections on biodata (age, sex, family background, schooling etc), on memories of schooling, expectations of college, and career plans, using a mixture of closed and open-ended questions. A final section contained a number of Likert-type statements about pedagogy, the role and status of teachers, and other professional issues. Although the open-ended questions produced rich data, they proved difficult and time-consuming to code and analyse. The Exit and NQT questionnaires were therefore drawn up at the Durban workshop to a closed-response format so they could be quickly analysed by computer, using SPSS. They contained a similar section on biodata, evaluations of the college experience including Teaching Practice (TP), careers plans, and for NQTs items about induction and early experiences in school. A revised list of Likert items was developed and included. Because of the time scale, these surveys were cross-sectional rather than longitudinal, with different cohorts being surveyed.
In Lesotho (one college) and Trinidad and Tobago (two colleges) it was possible to survey trainees from each training institution. In Malawi, two Colleges were sampled out of six, and in Ghana four out of thirty eight. Overall the entry samples included 400 student teachers in Ghana, 90 in Lesotho, 176 in Malawi, and 299 in Trinidad. These samples have their limitations, particularly as regards gender. Over two-thirds of the student teachers in the samples in Lesotho, Malawi, and Trinidad and Tobago were female. In Ghana only about a third were female. In Ghana and Lesotho these proportions were broadly consistent with the intake in previous years. However, in Trinidad and Tobago, and in Malawi, the samples had an over-representation of females when compared to the overall proportions enrolled in training. In Malawi this was due to the fact that one of the colleges used in the sample was all female. In Trinidad and Tobago the reason appears to be because of differential response rates between males and females. Further information on the samples can be found in MUSTER country reports and discussion papers (e.g. Akyeampong and Lewin 2002, George, Mohammed et al 2001, Kunje et al 2001, and Lefoka, Molise et al 2001).
Similar, though slightly smaller, samples were used for the Exit questionnaire. Administering questionnaires in the colleges to entering and exiting students ensured fairly high response rates. The NQTs were more difficult to contact and responses were lower: data was collected from 134 respondents in Ghana, 64 in Lesotho and 64 in Malawi. The Lesotho team drew up a separate questionnaire to elicit views on teaching practice from 120 recently returned third-year students.
A predominantly closed-item questionnaire was used in Ghana, Malawi and Lesotho for college tutors. It requested information on qualifications and career paths, views of the college, its curriculum and their students, and contained a Likert item section inviting agreement or disagreement with a range of statements, many of which paralleled those given to students.
The main strength of the surveys was to capture broadly comparable data across the different countries. This was particularly useful in analysing the origins, characteristics and attitudes of those entering teaching. The Likert sections enabled us to make some tentative estimates about how far attitudes might be changed by training (Akeayampong and Lewin 2002, Coultas and Lewin 2002).
The researchers were aware that many of the survey respondents indicated quite high levels of satisfaction with aspects of the teacher education curriculum although interviews often revealed more general discontent. These sometimes contradictory results had to be interpreted carefully. There was a general consensus that questionnaires may be seen as 'official' instruments, and respondents did not wish to criticise the colleges because of traditions of respect for elders and for those in authority, and this could have led to patterns of response that exaggerated levels of satisfaction. Judgements were made to mediate the interpretation of responses accordingly. Another problem was language; all the instruments were designed in English - the medium of instruction. After piloting there were uncertainties that respondents understood all the questions in the same way as the researchers, and further attempts were made to try to improve wording and presentation to reduce this problem. In addition there were problems with the administration of some survey instruments. In some cases student respondents were under pressure from exams or other events and were unwilling to devote time to questionnaires; in other cases it was judged more productive to collect some data using other methods since levels of cooperation were problematic. These difficulties were recognised in the interpretation of data and the decisions made on sampling which were designed to be as representative as circumstances allowed.
These were used widely in a variety of forms wherever the research teams felt comfortable with this approach. Key informants in colleges and Ministries were interviewed for background information, and to provide views on policies. Semi-structured interviews with samples of college tutors were used in the four sites to elicit both information about their careers, and their perspectives, attitudes and beliefs about their work in preparing teachers (see Stuart et al 2000). In Trinidad and Tobago the experience of teaching practice was explored in depth using interviews with college tutors, students, principals and cooperating teachers in the schools (see George, Worrell et al 2000 (parts 1 and 2). Similar methods were used there to explore the views of NQTs (Morris & Joseph, 2000).
2.3.3. Focus Groups
These were used in a limited number of places for specific purposes. In cultural contexts where much respect is given to those in authority, the one-to-one interview may not always elicit full and open answers, especially where there is a large perceived power distance. Interviewing students in groups proved useful, encouraging respondents to speak out, and allowing for alternative views and underlying assumptions to emerge. For example, in Ghana groups were used effectively with NQTs to discuss their first year of teaching, and also with trainees about their TP (see MDP 17, Akyeampong et al). Similarly in Malawi and Lesotho small groups of students in college were brought together to give their views on the curriculum.
These were used both in college lecture halls and school classrooms. Tutors were observed teaching in all four sites, most extensively in Lesotho, where eight tutors were observed 3 or 4 times each, and in Malawi, where 16 tutors, drawn from two colleges, were each observed once. In Ghana and Trinidad and Tobago smaller samples were observed. No schedules were used, but observers noted down key points of the classroom events and processes with as much detail as they could, in some cases supplemented by tape recordings.
Classroom observations were used in all the studies of NQTs, albeit with small samples (6 in Lesotho, 8 in Trinidad and Tobago,11 in Ghana.), again using simple notes. Small numbers of untrained teachers were also observed to give some insight into differences. In Malawi NQTs were observed using the standard TP assessment instrument to explore how far their teaching differed from those of their untrained colleagues.
2.3.5. Combining classroom observation with interview
In several of the studies in both colleges and schools observations and interviews were used together to try to elicit teachers' and tutors' views about their work in more depth. The shared experience of the lesson provided a context for the interview; more deliberately, the interviewer could use incidents from the lesson to probe some of the underlying assumptions and reasoning (e.g. Croft 2002a). The importance of taking time to build up rapport and trust was emphasised in interactions between researchers and respondents (See Hedges 2002a). In the Trinidad and Tobago TP study, the researchers observed not only the lessons but also the post-lesson conferences, and then interviewed students and tutors. Such forms of triangulation enabled researchers to gain a more holistic view of the processes by drawing out the participants' meanings (George, Mohammed, et al 2001).
2.3.6. Autobiographical essays and student diaries
These were used to supplement the Entry Questionnaire. A small sample of students (18 in Ghana, 27 in Lesotho, 20 in Malawi, 16 in Trinidad and Tobago) were asked to write about good and bad experiences of schooling, their best and worst teachers, and their current hopes and expectations. The essays were coded and analysed qualitatively by theme, along with the open-ended questions on similar topics in the entry questionnaire. They provided very valuable information on the kinds of images and experiences of schooling and of teachers that trainees bring to their training.
A small group of trainees in Lesotho were given diaries in which to write comments about their classes, what they were learning, and their problems. Language, time and the unfamiliarity of diary keeping made this difficult to sustain, but some rich insights were obtained that cast a new light on students' lives.
2.3.7. Document analysis
Documents were used extensively. The Baseline studies were compiled from secondary sources. Past student files were accessed where appropriate. Curriculum documents were analysed using Eraut's (1976) scheme that identifies aims and objectives, content, pedagogy, and teaching-learning materials to see whether they form a consistent curricular strategy. In some sites exam papers were also analysed to ascertain the coverage of syllabus topics and the cognitive demands made.
In the four sites tests in Maths and English were drawn up using items from, or similar to, the local school-leaving examinations which are used for college entrance. In Malawi and Ghana pedagogical knowledge tests were also devised. These tests were administered cross-sectionally to entering and exiting cohorts since it was not possible to follow cohorts longitudinally.
2.3.9. Financial analysis and modelling supply and demand
Statistical analysis was undertaken based on school census data, national planning reports and budgetary allocations. This was supplemented with data collected at institutional level to establish patterns of expenditure. Interviews were conducted with those with financial responsibilities at different levels. Supply and demand was modelled using simulations adapted for each country to derive projections of future needs linked to national policy goals. This was used to profile budgetary demand and ascertain whether current methods and costs of training were likely to prove sustainable.