Appendix 5. Summary of selected methods of data collection for organisational analysis

1. Observation

The observer looks at what is happening, watches how things are done, sometimes with the aid of an observational checklist or classification scheme.

Advantages

- useful for getting an overall picture;

- may highlight information that will not be revealed by interviews or questionnaires, either because the respondent is unaware, or unwilling to disclose it.

Disadvantages

- only shows what is on the surface; needs further investigation to explore in depth and to get at underlying issues, problems and causes.

2. Structured interview

Interviewer plans the interview carefully, preparing all the questions in advance according to a predetermined schedule of what he or she wants to know.

Advantages

- relatively quick;

- unskilled interviewers usually feel more confident when supported by a pre-structure;

- lends itself to subsequent numerical or quantitative analysis of data;

- enables data collection to be standardised - i.e. the same questions put to every respondent.

Disadvantages

- closed-ended, and dependent on the framework underlying the questions;

- inflexible and unresponsive to cues that the respondent may give unless these happen to form the subject of a predetermined question;

- unresponsive to other aspects or areas of investigation not considered in design of predetermined questions.

3. Unstructured interview

Interviewer "plays it by ear", encourages the respondent to talk, looking for cues and areas of interest to explore with subsequent questions.

Advantages

- flexible; responsive to new areas and data raised by the respondent, thereby opening up the possibility of new discoveries and obtaining a wider picture of what is happening in the organisation.

Disadvantages

- usually takes longer than structured interview;

- requires more skill on part of interviewer;

- less readily lends itself to subsequent numerical or qualitative analysis;

- allows qualitative analysis (although this requires considerable skill).

4. Semi-structured interview

Synthesis of structured and unstructured. Interviewer notes down in advance the main areas he or she wants to examine, but allows the respondent to reply freely, playing it by ear to a large extent, following up cues and so on. At the same time, the interviewer makes sure that all the main areas (from the pre-plan) have been covered before the end.

Advantages

- ensures that ground is covered, whilst allowing for flexibility and exploring new avenues;

- allows for both quantitative and qualitative analysis of data.

Disadvantages

- takes longer than structured interview;

- requires considerable skill;

- danger of lapsing either into "must cover all my main points" or of forgetting to cover them.

5. Counselling interview

As with either unstructured or semi-structured interview, but in addition the interviewer reflects back the respondent's answers, and works with the latter on considering the implications. Thus, as well as collecting data, this has potential for actual development.

Advantages

- flexible, may lead to first steps in development.

Disadvantages

- time-consuming;

- difficult; requires skill both by interviewer/counsellor, and by the respondent.

6. Questionnaires

Investigator plans questions and predetermined data needs, then presents these questions in written form to the respondents. Presentation may be face to face or, more often, at a distance (e.g. through the mail).

Advantages

- once designed and prepared, many respondents may be given questionnaires;

- thus, this can be a relatively very quick way of obtaining data from a large sample;

- lends itself to subsequent analysis:

· quantitative, if questions are closed-ended, or require yes/no answers, or are of multiple-choice format;

· qualitative, if questions are open-ended, calling for descriptive written answers from each respondent.

Disadvantages

- good questionnaires are surprisingly difficult to design;

- bad questionnaires are annoyingly easy to design;

- inflexible and unresponsive to answers;

- dependent on framework underlying the questions.

7. Scales

Instead of questions as such, scales require the respondent to make some form of rating. For example: How often are you consulted about your development needs?

There is a very wide variety of response formats, some of which are included in this book.

Advantages

- as with questionnaires, may be used with many respondents, with rapid response;

- readily lend themselves to numerical, quantitative analysis;

- provide a convenient way of plotting scores to feed back to respondents, or to include in reports.

Disadvantages

- dependent on framework or model for designing the items;

- danger of falling into trap of spurious and unjustified statistical analysis;

- surprisingly difficult to design a scale that is in fact valid and reliable;

- extremely easy to design a scale that looks valid and reliable, but in fact is not.

8. Diaries and critical incidents

In this book, these have been described as ways in which the individual can assess him or herself. As well as this, they can be used at an organisational level, with the investigator collecting in the diaries or records of incidents and carrying out an overall analysis.

Advantages

- require less preparation/design than interviews, questionnaire and scales;

- not dependent on any predetermined structure or model;

- enable real, significant issues to be highlighted.

Disadvantages

- time-consuming and, often, unpopular with respondents;

- difficult to analyse.

9. Role set analysis

Again, this is an organisational level application of a method already described for individual self-assessment (appendix 1). At the organisational level, a whole team group or department works on the exercise together, giving each other feedback about what they expect from each other, and negotiating around those expectations that they feel are not being met.

Advantages

- can be very useful for developing a team or department;

- has developmental as well as diagnostic validity.

Disadvantages

- time-consuming;

- threatening;

- requires skill and commitment.

10. Data feedback meetings

Data collected by any of the various methods is fed back to the respondents at a special meeting. With help of a "neutral" third party (e.g. a trainer), this is then discussed and looked at in terms of members' thinking, feeling and willing.

Advantages

- can lead to movement and development;

- participative.

Disadvantages

- may require many meetings over a period of time;

- usually requires great skill on behalf of trainer (e.g. skills in group decision making, handling conflict).