Section 4 - Economic evaluation of health care programmes

This often neglected stage is nonetheless essential. It alone can highlight the sort of questions which will help correct any measures taken in future:

- were the objectives aptly chosen?
- were the results what we expected?
- are the methods used the most suitable?
- were the resources deployed the most appropriate?

A - Nature of economic evaluation

These notes summarise the introductory chapters of the work : MichaelF. Drummond, Greg L. Stoddart and George W. Torrance, Methods for the Economic Evaluation of Health Care Programmes, Oxford Medical Publications, Oxford University Press, 1987 (181 p).

The underlying concept of economic analysis is that of opportunity cost. The decisions taken by economic players (the employer who hires or invests, the consumer who saves or buys) hang on choices which reflect these players' preferences when faced with multiple needs and limited resources. Choosing one thing means sacrificing something else: allocating 1000 francs to the purchase of compact discs means forgoing a 1000 franc meal at a restaurant, for example. Similarly, in the health care sphere, the real cost of a particular health care programme is not the number of francs shown in the programme budget, but rather the "value" of the results in health terms of any other programme passed over in favour of the first programme. It is this "opportunity cost" which economic evaluation endeavours to measure and compare with the results of the programme assessed.[1]

Examples:

- Is a particular service, programme or activity worth setting up compared with what we could obtain otherwise, with the same resources?

- Is the way in which a particular service, programme or activity operates satisfactory compared with some other mode of operation?

- Should all hospitals be equipped with tomodensitometers or should preference be given to the geriatric departments?

This latter type of question highlights the fact that the answer is seldom "yes" or "no" but more often "to what extent"?

Economic analysis will endeavour to:

- relate the costs (inputs) to the effects or consequences (outputs)

- evaluate the possible choices in relation to each other or to make implicit choices explicit (many medical activities are performed out of habit and have never given rise to an economic evaluation).

Economic evaluation is an analysis of the alternative forms of action in terms of costs (inputs) and consequences (outputs), which must be identified, measured and where appropriate, optimised.

B - Types of economic evaluation

Remarque - For the purposes of this report, we will use the term "programme" to refer not just to health care programmes in the proper sense, but also services, activities, schemes, institutions, procedures, methods, etc.)

These apparently clear distinctions are not always evident in scientific literature.

The widespread confusion concerning the terms used and the description of the type of approach employed has prompted the suggestion that we classify economic evaluations as follows:

A cost study is purely concerned with costs. If it compares several programmes with different costs, the preferred term is cost-minimization analysis.

A cost-effectiveness study relates the effects of a given programme, measured in physical units (life-years gained, number of accurate diagnoses, number of cases detected) to their cost. Ordinarily, such a study does not consider the fact that these effects may not be worth pursuing; they are assumed to be desirable.

A cost-utility study relates the effects of a given programme, measured in quality-adjusted life-years (QALYs) to their cost. It is therefore a type of cost-effectiveness analysis, which is particularly useful for programmes where it is the result in terms of sickness ratio which matters, or in terms of life-years gained. These extra life-years are "modulated" or "adjusted" according to the quality of life afforded: sound health is rated 1, death 0, a minor handicap (side-effects of chronic treatment, etc.) 0.2 (for example) and a major handicap (impotence, incontinence, etc.) 0.7 (for example).The "utility" thus weights the life-years gained by the quality of the remaining life.

A cost-advantage study or cost-benefit study (synonymous) relates the effects of a given programme, measured in monetary terms (francs, dollars, etc.) to their cost. In theory, this ought to be the broadest and most sophisticated form of economic evaluation, in so far as one compares all of the costs with all of the advantages, translated into monetary value. In practice however, translating all of the effects into monetary terms is difficult and cost-benefit studies often confine themselves to that which can be easily assessed in this manner. In many cases therefore, it constitutes a more limited approach than a cost-effectiveness analysis.

Broadly speaking, economic analysis establishes the connection between costs and effects: it is an evaluation of yield or efficiency (synonymous).

2) Classification Criteria:

- Is a particular situation compared (i.e. is some attempt made to examine one or more alternatives) or simply described ?

- Are the costs (inputs) AND the consequences (outputs) of the different alternatives examined ?

"Partial" does not mean pointless!It does mean, however, that there is no attempt to address the issue of efficiency, making it possible to choose between several options.

In cases 1A, 1B and 2, there is no comparison between various alternative solutions: what we have is a straightforward description rather than an economic analysis:

- description of effects in 1A
- description of costs in 1B
- description of cost-effectiveness in 2.

Please note - This latter type of study is sometimes mistakenly referred to in literature as "cost-benefit analysis". E.g. Reynell PC and Reynell MC, The cost-benefit analysis of a coronary care unit, Br. Heart J. 34, 897-900, 1972. The authors present data on the costs of a coronary care unit and the number of lives saved. They do not compare the costs and effects of a coronary care unit with the costs and benefits of some alternative solution however. What is more, there is no attempt to convert the effects into monetary value.

Cases 3A and 3B offer comparative analyses, but never of both costs and effects at the same time.

Case 3A is obviously that of randomised controlled clinical trials relating to a new treatment (or screening procedure)

A cost analysis (case 3B) will compare, for instance, the respective cost of two vaccination strategies, but without considering their effectiveness.

Full economic evaluation

This is represented by case 4.There are basically four types of analysis.

a) Cost-minimization analysis

Cost-minimization analysis compares the costs of two programmes whose effects are identical.

Cost-minimization analysis differs from straightforward cost analysis (case 3B) in that it forms part of a study (often a randomised controlled trial) which proves that the two programmes are equally effective (e.g. conventional surgery and ambulatory surgery for one and the same pathology).Cost analysis often postulates or does not even seek to discover whether the treatments differ in terms of their results.

b) Cost-effectiveness analysis

The cost-effectiveness analysis compares the effects and costs of two or more programmes. E.g.: life extension following a kidney transplant or dialysis. One calculates:

- a cost per life-year gained
- or the number of life-years gained per franc spent.

A cost-effectiveness analysis does not necessarily compare two programmes applied to the same health problem: one might very well compare the number of life-years gained as a result of heart surgery, kidney transplants and wearing a safety belt. What matters is that the effects of the activities compared can be measured using the same criterion (life-years gained, number of days unfit for work avoided, etc.).

Furthermore, the effect does not have to be a treatment result, but may very well be a diagnosis or detection result: cost per case detected according to a particular procedure; cost per case diagnosed according to a particular diagnostic strategy. E.g.: Hull R et al., Cost-effectiveness of clinical diagnosis, venography and noninvasive testing in patients with symptomatic deep-vein thrombosis, N. Engl. J. Med., 304, 1561-7, 1981.

c) Cost-benefit analysis

Often it is not possible to measure one particular effect of several alternative solutions. Either the effects are of a different nature, or they are multiple. How in that case, can we reducemultiple or different effects to a common denominator?

Examples: compare home-based dialysis, hospital dialysis and kidney transplants not only in terms of life-years gained but also in terms of frequency of medical complications (multiple effects); compare high blood pressure screening in terms of life-years gained and anti-flu vaccination in terms of days unfit for work avoided (different types of effects).

In these instances, one has to go beyond the actual effect itself and assign a monetary value to the various effects being compared. This is basically a cost-benefit analysis.

The result can be presented in the form of a ratio (cost in $ / benefits in $) or a difference (net benefit of a particular programme - net benefit of some other programme... or of having no programme at all).

Translating life-years gained, days of unfit for work avoided and medical complications avoided or entailed into francs is no easy task. Which is why we sometimes quantify just some of the costs, namely those which can be easily measured.

Sometimes too, we measure costs and benefits from a particular point of view: from the point of view of the Nation as a community, or the social security system, the State, households, companies, etc. The differences can be considerable (e.g. cost-benefit of vaccinating workers in the catering industry against hepatitis A: the costs and benefits will vary markedly depending on your point of view).

d) Cost-utility analysis

Cost-utility analysis also attaches a value to the effects obtained but rather than being expressed in monetary terms, this value is measured in terms of the degree of utility of the effects obtained. That one and the same state of health (or illness) can be of varying utility depending on the individuals concerned can be seen from the following simple example: twins, one of whom is a bank clerk and the other a violinist, break their second finger on their left hand; although both are in the same state, if we ask each to rate on a scale of 0 to 10 the inconvenience of not having the use of their second finger (hence the utility of the treatment), we will receive very different replies.

This notion of utility allows us to take account of the quality of life afforded by different effects and at the same time provides a common denominator for comparing the costs and effects of different programmes.

This common denominator is normally expressed in "days of good health" or in "quality-adjusted life years" (QALYs).The number of life-years gained is weighted by an index expressing the degree of utility attached to the state of survival. Such an index must form the subject of opinion polls (conducted among patients or the general public) if it is to be valid.

C - Choosing a type of analysis

1) There is no fixed order of preference for these four types of economic evaluation.

2) Often, the researcher does not know in advance exactly what type of analysis he can apply; it may depend on the results of a clinical study associated with the evaluation: two treatments which ultimately prove to be equivalent will reduce a cost-minimization analysis to a cost-effectiveness description. Or a cost-benefit analysis may be combined with a cost-utility analysis, for particularly tricky problems (e.g. neonatal care).

3) The most important thing is to ascertain whether the complexity of the analysis is really commensurate with the question posed:

a) cost-benefit or cost-utility analyses primarily seek to determine whether a particular programme is "worth the trouble" compared with some other programme.

b) cost-minimization or cost-effectiveness analyses tacitly assume that the effect of the programme concerned is "worth the trouble".

4) Economic evaluation does not exempt us from having to think. It simply highlights the options or renders explicit certain options which have been ill-discerned or accepted without discussion. Whether or not economic considerations should prevail in the final decision is a matter for the decision-maker.

D - Types of cost and effects

1) Costs

Direct costs

- to the patient (and those close to him):cost of the treatment, of travelling, etc.
- to the health service (in terms of staff, premises, equipment, running costs):
- variable costs: these vary in proportion to the volume of activity
- fixed costs: these do not vary (or do so only by leaps and bounds, by thresholds) according to the volume of activity.

Indirect costs (to the patient):

- loss of productivity (time wasted) in so far as it results from participation in the programme
- psychological costs (loss of quality of life)

Externalities (external costs)

These are the costs occasioned by the programme beyond the scope of the health care system and the patient (or his family) - and tend to be fairly imponderable.

2) Effects

1_ Therapeutic effects (effectiveness, measured objectively, without any value judgements):reduction in the sickness ratio, reduction in the death rate, etc.

2_ Direct and indirect benefits (benefits, advantages)

Direct benefits for the health service: resources saved (although this benefit can sometimes be a "negative" one: helping people to live longer can increase the amount of use made of the health service!) Direct benefits for the patient: resources saved (in terms of money or leisure time)
Indirect benefits for the patient: production gains (much debated area).

3_ Quality of life (utility):needs to be assessed separately.

E - Three important concepts

1) Marginal cost, marginal benefit, marginal effectiveness

This notion is defined here with regard to costs; it may equally be applied to benefits and effectiveness.

Average cost: total costs divided by total units of operation (patients, persons screened, vaccinated, etc.).

Marginal cost: additional cost entailed by one extra unit of operation.

Famous example: cost per detected case of cancer of the colon by looking for blood in the stools. The cost per case detected (average cost) obviously increases with the number of tests, but the cost per case detected thanks to the 6th test (marginal cost) was estimated at 47 million dollars in 1975!A prime example of how economic evaluation can clarify the cost of a particular option and make decision-makers aware of it.

2) Timescale adjustments to costs and benefits: "Present-worthing"

Quite apart from the effects of inflation, 100 francs available today is "worth" more than 100 francs available in one year's time: it is always preferable to obtain a particular benefit immediately and to defer the costs until later. The concept of "time preference" as used in economics, is a subjective one, which can vary from one society to another.

Estimating the present value of future costs and future benefits is done by means of a "present-worthing" (or discounting) procedure.

Discounting (in economics) is an operation whereby someone (normally a bank) advances the amount of a negotiable instrument ahead of its maturity date, less a deduction (discount rate), in return for ownership of this instrument.

The interest rate on loans operates according to the same mechanism; it "compensates" for the "loss of value" due to the "time preference". Thus, today's franc (F) will be worth more in n years (F') according to the equation:

F' = F (1 + r)n

where r is the "present-worthing" rate, the discount rate or the interest rate (the same formula applies for compound interest).

Conversely, the present value (F) of a franc in n years (F') is equal to:

F = F' / (1 + r)n

Since the time preference is not an exclusively financial concept, "present-worthing" also needs to be applied in cost -effectiveness and cost-utility studies (even though the effects here are not measured in monetary terms).The techniques for doing this can be examined elsewhere.

Some countries, in order to calculate the cost of the various programmes envisaged (health care, public works, etc.) officially recommend using a specific discount rate.

Reminder: the discounting procedure must be strictly distinguished from adjusting costs for inflation.

3) Sensitivity test

Economic evaluations can vary according to the degree of uncertainty, inaccuracy or controversy over the methods employed.

Examples:

- the number of attacks due to a flu epidemic can vary from one year to another and cause the effects of a vaccination programme to vary

- the costs of hospitalisation can change rapidly

- various discount rates may be applied

- the indirect costs and benefits may be included or excluded from the analysis

- certain parameters are only roughly known and assumptions must be made as to their true value.

A sensitivity analysis will test the variation in the results (effects and costs) by simulating different variations in the parameters considered.

If a major variation in the parameters has little effect on the outcome of the analysis, the latter is said to be sound. Otherwise, some attempt will have to be made to discover the true value of the parameters by means of a validation procedure (test the working hypothesis under actual conditions, in the field).