Cost Effectiveness Analysis: Interpreting CEA Results

Interpreting CEA Results

Cost Effectiveness Ratios

Once a CEA has been conducted and cost effectiveness has been calculated, ratios combining the expected results are calculated and reported. The cost referred to in the CER is a net cost value discussed earlier in the section: "Which Costs Are Included in the CEA?"

There are three types of cost effectiveness ratios (CERs):

Average cost-effectiveness ratio (ACER),
Marginal cost-effectiveness ratio (MCER), and
Incremental cost-effectiveness ratio (ICER).

ACER

An ACER

deals with a single intervention and evaluates that intervention against its baseline option (e.g., no program or current practice).
is calculated by dividing the net cost of the intervention by the total number of health outcomes prevented by the intervention.

An Example of an ACER

Intervention	Net cost	Total outcomes (life-years saved)	ACER cost per life-year saved
Home vaccination program	$50,000	8	$6,250

MCER

The marginal cost-effectiveness ratio (MCER) assesses the specific changes in cost and effect when a program is expanded or contracted.

Because the majority of programs that are cost effective are considered good investments only at a certain level, the MCER and ACER are often considered simultaneously.

This figure shows the ACER and MCER for the vaccination program example:

At low vaccination coverage rates (approximately 70%), the ACER is negative, indicating a savings in cost. As that percentage grows, however, so does the cost per case prevented because the marginal cost per each additional person vaccinated is much higher than the average cost.

In general, the MCER is used in conjunction with the ACER as a tool to determine the most efficient level of program implementation.

Once this level is determined, the ACERs of that and other independent programs that result in the same outcome can be compared.

ICER

An ICER

compares the differences between the costs and health outcomes of two alternative interventions that compete for the same resources, and
is generally described as the additional cost per additional health outcome.

When comparing two competing programs incrementally, one program should be compared with the next-less-effective alternative.

The ICER numerator includes the differences in program costs, averted disease costs, and averted productivity losses if applicable. Similarly, the ICER denominator is the difference in health outcomes.

An Example: Decision Analysis CERs

The values and estimates used in this analysis were modified from various studies for the purposes of this example and should not be considered literally.

Consider a policymaker that is trying to decide whether or not to implement a sobriety checkpoints program if BAC and MLDA laws (baseline) are already in place. The probabilities and payoffs (costs and outcomes) for each branch pathway are shown in this decision tree:

Decision tree for the sobriety checkpoints intervention when BAC and MLDA laws (baseline) are already in place.

The outcome of interest is deaths per 100,000 persons.
Total costs for the baseline option include trial and sanctioning and law enforcement costs.
Total costs for the sobriety checkpoint program include officer wages, equipment, publicity, travel delay, and sanctioning costs.
Both total cost estimates also include the costs (medical and productivity losses) associated with a fatality resulting from an alcohol-related crash.

The expected total cost estimates (i.e., values that take probabilities into account) are shown in the decision tree below. Each value is shown in a green box to the right of the chance node (green circle).

The additional cost of sobriety checkpoints is:

$12,360

–

$9,200

$3,160

The expected total outcome values indicate that sobriety checkpoints are more effective (i.e., result in fewer deaths per 100,000 persons) than the baseline option. This analysis is presented in this decision tree:

This decision tree shows that the expected total outcome values indicate that sobriety checkpoints are more effective (i.e., result in fewer deaths per 100,000 persons) than the baseline option.

The cost and outcome results considered separately warrant the use of CEA. Sobriety checkpoints are both costlier and more effective than what is currently being done. This decision tree shows both results.

This decision tree shows the cost and outcome results with and without sobriety checkpoints.

From the numbers in this decision tree, the ICER comparing sobriety checkpoints to the baseline option is:

ICER

(

Total cost_B

–

Total cost_A

)

(

Total outcomes_A

–

Total outcomes_B

)

ICER

(

$12,360

–

$9,200

)

(

0.37

–

0.22

)

ICER

$21,066.67 per death prevented

If program costs and health outcomes are different but averted disease costs (i.e., alcohol-related crash costs) are the same, then the ICER should be computed as:

ICER

(

Add'l program cost

–

Add'l cost of disease averted

)

Add'l health outcomes

ICER

(

$3,160

–

)

0.15

ICER

$21,066.67 per death prevented

This analysis suggests that each death prevented by sobriety checkpoints will cost $21,066.67.

Exclusion Criteria

The programs are arrayed in order from least to most effective in terms of outcomes prevented.
This is done first so that the ICERs may be calculated and resources allocated appropriately.

This table shows data and calculations for Program A, an expanded Program A, and Programs B and C:

Programs	Prevented outcomes	Total costs	ACER	MCER	ICER
Independent programs
Program A	10	$150	$15
Expanded program A	12	$200		$25
Mutually exclusive programs
Program A	10	$150
Program B	20	$300			$15
Program C	25	$250			–$10

Notes for Table Columns

ACER

ACERs can be calculated with an alternative formula similar to that for ICERs above if the baseline is "no program":

ACER

(

$150

–

)

(

–

)

ACER

$15 per additional outcome prevented

MCER

MCER

(

$200

–

$150

)

(

–

)

MCER

$25 per additional outcome prevented

ICER

ICER_{B to A}

(

$300

–

$150

)

(

–

)

ICER_{B to A}

$15 per additional outcome prevented

and

ICER_{C to B}

(

$250

–

$300

)

(

–

)

ICER_{C to B}

–$10 per additional outcome prevented

If program A is expanded, the MCER is $25. Because $25 is higher than $15, a decisionmaker needs to decide whether or not expansion is worth the additional cost.

If two competing programs are compared, such as programs A and B, the ICER is $15. The comparison between programs C and B shows a cost saving of $10 for program C over program B.

The negative ICER for program C indicates that program B is "strongly dominated." In other words, program B is more costly and less effective than program C.

Strongly dominated programs should be excluded from the set of alternatives so they do not consume limited resources. In addition, the ICERs should be recalculated so that the magnitude of the negative ICER is not misleading, as shown in this table:

Mutually exclusive programs	Prevented outcomes	Total costs	ICER
No program	0	0	—
Program A	10	$150	$15
~~Program B~~	20	~~$300~~	~~$15~~
Program C	25	$250	$6.67

ICER Calculation for Program C

ICER

(

$250

–

$150

)

(

–

)

ICER

$6.67 per additional outcome prevented

If a program's ICER is higher than the next most effective program, the program is "weakly dominated." In effect, weakly dominated programs produce effectiveness at a higher marginal cost, which means that some combination of at least two other alternative programs is more cost effective.

In this table, Program D is compared with Program C because they are ranked in order of increasing effectiveness:

Mutually exclusive programs	Prevented outcomes	Total costs	ICER
Program A	10	$150	—
~~Program B~~	20	~~$300~~	~~$15~~
Program C	25	$250	$6.67
Program D	40	$325	$5

In this case, some combination of programs A and D is more cost effective than program C. Therefore, program C should be removed from the list of alternatives and the results recalculated accordingly (i.e., program D compared with program A). This table shows the final results:

Mutually exclusive programs	Prevented outcomes	Total costs	ICER
Program A	10	$150	—
~~Program B~~	20	~~$300~~	~~$15~~
~~Program C~~	25	~~$250~~	~~$6.67~~
Program D	40	$325	$5.83

Decision Guidelines

Decision making is a complex process that takes into account much more than numbers and calculations.

Value judgments are often implicit in the choice made by a decisionmaker.

Even so, calculations that allow comparisons to be made between independent and competing programs are important and generally valued by decisionmakers.

Users of CEA can allocate limited resources and make decisions more efficiently if certain decision rules or guidelines are followed.

When Assessing Independent Programs

Order the programs from least to most effective.
Eliminate the strongly dominated programs.
Calculate ACERs.
Implement programs in order of increasing ACER until either resources are exhausted or the ACER is equal in value to one unit of effectiveness.

When Assessing a Mix of Independent and Mutually Exclusive Programs

Form groups of mutually exclusive programs.
Order programs within each group from least to most effective.

Within Each Group

Calculate the ICER.
Eliminate both strongly and weakly dominated programs.
Calculate the ACER of each independent program.
Rank all programs in order of increasing ratio.
Implement programs in order of increasing ACER until either resources are exhausted or the ratio is equal in value to one unit of effectiveness.

Presentation of Results

Presenting CEA results in a comprehensive and concise manner is imperative to CEA's overall usefulness among decisionmakers. The information included in a CEA is practical and advantageous to the decision making process.

Thus, the presentation of a CEA should include these eight major elements:

Eight Elements of a CEA
A clear study perspective, time frame, and analytic horizon An explicitly defined study question Relevant assumptions underlying the study Detailed descriptions of the interventions Existing evidence of the interventions' effectiveness Proper identification of all relevant costs: decide whether to include or exclude productivity losses apply appropriate discount rate confirm that included costs are relevant to perspective An appropriate choice of outcome: calculate a suitable CER report ICER results (unless the only comparator is baseline) conduct sensitivity analyses A comprehensive discussion of the results: deal with issues of concern address implications of underlying assumptions

Eight Elements of a CEA

A clear study perspective, time frame, and analytic horizon
An explicitly defined study question
Relevant assumptions underlying the study
Detailed descriptions of the interventions
Existing evidence of the interventions' effectiveness
Proper identification of all relevant costs:
- decide whether to include or exclude productivity losses
- apply appropriate discount rate
- confirm that included costs are relevant to perspective
An appropriate choice of outcome:
- calculate a suitable CER
- report ICER results (unless the only comparator is baseline)
- conduct sensitivity analyses
A comprehensive discussion of the results:
- deal with issues of concern
- address implications of underlying assumptions

Test Your Understanding

Should a highly effective intervention always be supported? Why or why not?
Why is the net cost — rather than total program costs — relevant for CERs?
Provide a criticism of using the human capital (HC) approach to measure productivity losses.
Calculate the ACER for a program with these costs and outcome:
- Program cost = $10,000
- Disease cost averted = $2,000
- Productivity losses averted = $3,000
- Life-years saved relative to no program = 5
Calculate the MCER for expanding the program:
- Total cost_A = $15,000
- Total cost_Ax = $20,000
- Total outcomes_A = 5
- Total outcomes_Ax = 7
where subscripts:
- "A" refers to the original program and
- "Ax" refers to the expanded program.
Calculate the ICER for two alternative programs, "A" and "B," competing for resources, given:
- Total cost_A = $15,000
- Total cost_B = $ 30,000
- Total outcomes_A = 8
- Total outcomes_B = 5
where a program outcome is the count of disease cases attributable to the program.
Explain the difference between strongly and weakly dominated programs.

Our Answers

Should a highly effective intervention always be supported? Why or why not?

A highly effective intervention should not necessarily be supported regardless of cost.
Both cost and outcome should be taken into account when deciding whether or not to allocate funding to a particular program.
Why is the net cost — rather than total program costs — relevant for CERs?

The net cost takes into account the benefits that are attributable to the intervention and therefore provides a "true" estimate of cost.
Without subtracting future savings, the cost would be overestimated, resulting in a large CER.
Provide a criticism of using the human capital (HC) approach to measure productivity losses.

By using forgone wages to calculate productivity losses, the HC approach systematically undervalues persons who receive lower wages (e.g., women and certain blue-collar workers).
Calculate the ACER for a program with these costs and outcome:
- Program cost = $10,000
- Disease cost averted = $2,000
- Productivity losses averted = $3,000
- Life-years saved relative to no program = 5
ACER = ( Program costs – Averted disease costs – Averted productivity losses ) / Health outcomes prevented

ACER = ( $10,000 – $2,000 – $3,000 ) / 5

ACER = $1,000 per life-year saved
Calculate the MCER for expanding the program:
- Total cost_A = $15,000
- Total cost_Ax = $20,000
- Total outcomes_A = 5
- Total outcomes_Ax = 7
where subscripts:
- "A" refers to the original program and
- "Ax" refers to the expanded program.
The MCER is the ratio of the differences in total costs and total outcomes between the initial program level and expansion level.

MCER = ( Total cost_Ax – Total cost_A ) / ( Total outcomes_Ax – Total outcomes_A )

MCER = ( $20,000 – $15,000 ) / ( 7 – 5 )

MCER = $5,000 / 2

MCER = $2,500 per outcome

Comparing the MCER for expanding program A to its ACER in the answer to question 4 suggests maintaining the program at its current production level rather than funding expansion.
Calculate the ICER for two alternative programs, "A" and "B," competing for resources, given:
- Total cost_A = $15,000
- Total cost_B = $ 30,000
- Total outcomes_A = 8
- Total outcomes_B = 5
where a program outcome is the count of disease cases attributable to the program.

The ICER is the ratio of the differences in total costs and total outcomes between the two programs.

ICER = ( Total cost_B – Total cost_A ) / ( Total outcomes_A – Total outcomes_B )

ICER = ( $30,000 – $15,000 ) / ( 8 – 5 )

ICER = $15,000 / 3

ICER = $5,000 per disease case prevented
Explain the difference between strongly and weakly dominated programs.

A strongly dominated program is one for which an alternative exists that is both more effective and less expensive. Weakly dominated programs generate effectiveness at a higher marginal cost than an alternative program.
Any program that is dominated by another should be removed from the list of competing alternatives so that the most efficient allocation of resources can be achieved.