Cost Effectiveness Analysis: PrintAll | CDC Econ Eval Tutorials (E)

Factor	Value
Decisionmaker	NBCCCEDP
Resources	Congressional appropriations
Alternatives	Women aged 40–69 versus 50–69 years
Group affected	Low income women
Cost of including women aged 40–49 years	Resources that could be used to: screen more women for cervical cancer, increase the percentage of eligible women aged 50–69 years receiving breast cancer screening, and expand the program to cover treatment costs.
Benefits of including women aged 40–49 years	increase in early detection rates, and possible decreased mortality attributable to breast cancer that is left untreated for too long.

Framing CEA

General concepts on framing have been presented in the Framing

tutorial. This tutorial will focus on how framing pertains to CEA.

Defining the Problem

In a CEA, health-related outcomes are relevant in the problem specification section. Since CEA is a comparative analysis, knowing costs and outcomes for both the existing and the proposed or alternative available programs is relevant.

Specific questions that could guide the problem identification section include:

Which intervention, a telephone help line or group treatment, is more cost effective for increasing the rate of smoking cessation among middle-aged adults?
Which intervention is more cost effective in treating persons who are intermediate or heavy smokers:
- a transdermal nicotine patch in addition to physician counseling, or
- counseling alone?

When defining the study problem for a CEA, asking these four questions is often helpful:

What is the problem to be analyzed?
Why is this problem important?
What aspects of the problem need to be explained?
What questions need to be answered?

Once the study problem has been identified, a research strategy must be adopted to further direct the framing process. This requires deciding how to structure the CEA.

An Example: Defining the Problem

This example is abstracted from a CEA conducted in 2000 to assess the cost effectiveness of two interventions (i.e., lifestyle and structured) designed to increase physical activity among sedentary adults in Dallas, Texas.

Texas' Project ACTIVE:
Lifestyle versus Structured Interventions

What is the problem to be analyzed?	Sedentary lifestyles are prevalent among adults in the United States.
Why is this problem important?	An association exists between inactivity and morbidity and mortality, especially with regard to chronic diseases, such as diabetes and cardiovascular disease.
What aspects of the problem need to be explained?	In an effort to reduce overall health-care costs and utilization by promoting prevention activities, information on costs and effectiveness was needed to evaluate the program.
What questions need to be answered?	What was the cost of the first two years of running the program? Did the program effectively increase physical activity among its participants?

Adopting a Research Strategy

Once the study problem has been identified and defined, a research strategy must be adopted that can direct an evaluator through the remainder of the framing process.

This requires deciding how to structure the CEA.

Following is a list of questions to ensure that all aspects of the research strategy are considered.

What Intervention(s) Will Be Analyzed?

The first task is to define with precision the programs or interventions to be evaluated. For each intervention or program included in the study, identify the following elements:

Nature of the intervention — e.g., a screening and vaccination program for varicella zoster virus (VZV) or "chickenpox" infection.
Target population — e.g., employees of a health-care institution.
Delivery site — e.g., hospital.
Personnel delivering the service — e.g., hospital nurses.
Technology to be used — e.g., an enzyme-linked fluorescent immunoassay test to determine serologic status.
Timing of the intervention — e.g., screening performed at program onset with subsequent vaccination for those patients with no immunity immediately following.

Identifying the elements stated above precisely is important because programs that appear similar might have very different costs and outcomes.

For example, in the example above, program costs would be different if only high-risk workers were screened rather than all hospital personnel.

Who Is the Audience?

The CEA study must address the needs of all persons, clinicians, legislators, and public health officials involved. A well-written problem identification section typically specifies the audience.

Whose Perspective?

Typically, the societal perspective is used in CEA studies dealing with public health issues.

What is the Time Frame?

The study must also specify the time during which the intervention is in effect. In other words, the time frame for a CEA of a varicella vaccination program should span the entire vaccination delivery schedule, not just the screening process.

What is the Analytic Horizon?

The researcher must make sure to include all costs and outcomes attributable to the intervention over the entire period that these costs and outcomes are observable or measurable.

In a CEA study for the varicella vaccination program described above, the analytic horizon would include the time frame (screening and delivery) plus the time during which the vaccine provides protective effects against acquiring the disease.

What is the Study Format?

The study format for a CEA may be:

a prospective study,
a retrospective study, or
a model.

What Outcome Measures Are We Interested In?

CEA outcomes are measured in natural or physical units and can be intermediate outcomes or final outcomes.

What Are the Available Alternatives?

After the study has been carefully framed, all appropriate interventions for comparison must be selected. For a CEA to be useful, the program under evaluation must be compared with all relevant alternatives.

The baseline comparator generally refers to the existing program and must be included as an option in the analysis. Alternatively, the baseline can be "no program." Including the baseline measure meets the needs of policy makers, who typically are interested in a new program's incremental costs (i.e., the costs beyond current expenditures) as well as in its effectiveness.

As a general rule, all programs that effectively achieve the same outcome among a given population and are deemed socially and politically acceptable should be considered as alternatives.

Which Costs Are Included in the CEA?

A CEA typically includes all tangible costs and excludes intangible costs.

Cost is referred to as the net cost, which is program costs minus 1) the cost of disease averted and 2) the cost of productivity losses averted. In a CEA, the costs of the disease averted and the cost of productivity losses averted are assessed by using the cost of illness (COI) method.

Net cost

Program cost

–

Cost of disease averted

–

Cost of productivity losses averted

Net cost is a summary measure that subtracts overall savings from gross program costs. Cost of disease averted and cost of productivity losses averted are subtracted because they are savings, and excluding them from the net cost equation will result in overestimating costs.

If the analysis is conducted from a societal perspective, averted productivity losses should be included in the net cost calculation. From a health-care system perspective, productivity losses might not be relevant.

Cost of Productivity Losses Averted

In a CEA, the human capital (HC) approach, a COI method, can be used to assess productivity losses.

The HC approach uses as its assessment measure forgone income attributable to morbidity and premature mortality as the result of some injury or illness. The HC methods are generally used to value productivity changes attributable to health problems.

An Example

Adopting a Research Strategy: Lifestyle versus structured interventions

Problem	Sedentary lifestyles are: thought to be associated with cardiovascular disease and diabetes, and prevalent among adults in the United States.
Target audience	health-care providers who wish to improve the health of their patients, and third-party payers who must consider health, costs, and service utilization of their enrollees.
CEA perspective	The provider perspective was used, because costs individual to the patient are not included.
Time frame	Twenty-four months for both interventions, the first 6 months being more intensive than the last 18 months.
Analytic horizon	In this study, the analytic horizon is identical to the time frame, e.g., 24 months, because only intermediate outcome measures were assessed.
Study format	Prospective.
Outcome measure under investigation	The study used intermediate outcomes in the form of average units of improvement, including such measures as: "change in energy expenditure," self-reported activity levels, weight, and blood pressure.
Costs included in the CEA	Program costs included direct nonmedical costs, such as: personnel (e.g., health educator and nutritionist), capital equipment (e.g., computerized tracking system), curriculum materials, supplies (e.g., postage and printing), facilities, and health club memberships. Costs borne by the patient (e.g., travel expenses) were not included in the cost inventory.

Test Your Understanding

Is it appropriate to define the research problem clearly before adopting a strategy?
Who is the audience for a study?
What is the perspective of a study?
Which costs are included in a study conducted from a societal perspective?
Would patient travel costs be included in a study conducted from a health-care system perspective?
What is the difference between a study time frame and analytic horizon?
Is it possible to conduct a prospective study to assess the cost of a program that was recently terminated?
List two intermediate outcomes that could be used to assess the effectiveness of a nursing home vaccination program.
Give an example of a final outcome for this same program.
List two alternative programs for a community wide education campaign for increasing child safety seat use.
What are the components of the net cost equation?
Why are cost of disease averted and cost of productivity losses averted deducted from the program cost in the net cost equation?
In which of these studies should the cost of productivity losses averted be included in the net cost equation?
A study conducted from a societal perspective

A study conducted from the patient's perspective

A study conducted from the health-care system perspective
The cost of an intervention was estimated at $740,000 for 50 participants. The program averted 60% of disease conditions, and the average cost of the disease per person was estimated at $20,000. The total cost of productivity losses averted was estimated at $300,000.
1. Calculate the cost of disease averted.
2. Calculate the net cost from the societal perspective.
3. Calculate the net cost from the health-care system perspective.
Given your answer to question 14B above, should the program be implemented?

Our Answers

Is it appropriate to define the research problem clearly before adopting a strategy?

Generally, yes, because once projects are under way, their objectives and characteristics need to be defined so that appropriate research is conducted and succinct results are reported.
Who is the audience for a study?

The users or consumers of the final results.
What is the perspective of a study?

The perspective of a study is the viewpoint from which it is conducted. A study's perspective defines both the costs and benefits that are measured.
Which costs are included in a study conducted from a societal perspective?

A study conducted from a societal perspective includes all relevant costs, regardless of who incurs them.
Would patient travel costs be included in a study conducted from a health-care system perspective?

No. Travel cost borne by the patient would not be relevant to a health-care system; however, a study conducted from a patient or societal perspective would include such costs.
What is the difference between a study time frame and analytic horizon?
- The time frame is the period over which program/intervention costs are tracked.
- The analytic horizon is the period over which the costs and outcomes associated with the impact of the program/intervention are tracked.
Is it possible to conduct a prospective study to assess the cost of a program that was recently terminated?

No. In a prospective study, information about costs is collected as the costs are incurred. No information can be collected prospectively if no program activity exists.
List two intermediate outcomes that could be used to assess the effectiveness of a nursing home vaccination program.
- Number of vaccine doses administered
- Number of persons receiving vaccine
Give an example of a final outcome for this same program.

Number of life years gained from use of vaccine.
List two alternative programs for a community wide education campaign for increasing child safety seat use.
- Incentive programs
- Child safety seat laws
- A combined distribution and education program
What are the components of the net cost equation?
The three components are:
1. program cost,
2. cost of disease averted, and
3. cost of productivity losses averted.
Why are cost of disease averted and cost of productivity losses averted deducted from the program cost in the net cost equation?

Cost of disease averted and the cost of productivity losses averted are deducted from the program cost because they are savings.
Ignoring costs averted understates the benefits and consequently overstates the cost of the program.
In which of these studies should the cost of productivity losses averted be included in the net cost equation?
A study conducted from a societal perspective

A study conducted from the patient's perspective

A study conducted from the health-care system perspective
The cost of productivity losses averted should be included in the net cost equation if the study is conducted from either:
- a societal perspective, or
- the patient's perspective.
The cost of an intervention was estimated at $740,000 for 50 participants. The program averted 60% of disease conditions, and the average cost of the disease per person was estimated at $20,000. The total cost of productivity losses averted was estimated at $300,000.
1. Calculate the cost of disease averted.
  
  Using:
  
  Cost of disease per person = $20,000
  
  Number of persons with averted disease conditions = 0.6 x 50
  
  = 30
  
  Then:
  
  Total cost of disease averted = $20,000 x 30
  
  = $600,000
2. Calculate the net cost from the societal perspective.
  
  Using:
  
  Program cost = $740,000
  
  Cost of disease averted = $600,000
  
  Cost of productivity losses averted = $300,000
  
  Then:
  
  Net cost = Program cost – Cost of disease averted – Cost of productivity losses averted
  
  Net cost = $740,000 – $600,000 – $300,000
  
  = –$160,000
3. Calculate the net cost from the health-care system perspective.
  
  Using:
  
  Net cost = Program cost – Cost of disease averted
  
  Then:
  
  Net cost = $740,000 – $600,000
  
  = $140,000
Given your answer to question 14B above, should the program be implemented?

Negative net costs imply savings, and such programs typically are implemented.

Which Outcomes are Relevant in CEA?

Introduction

Health outcomes set a CEA apart from other forms of economic evaluation (i.e., CA, CBA, and CUA).

The table below lists possible health outcomes. Each outcome should be considered in terms of "cost per health outcome" (e.g., cost per additional vaccination and cost per life-year saved).

Health Outcomes
Additional person vaccinated	Additional person screened
Fatal injury prevented	Increase in child safety seat use
Pregnancy prevented	Case of lung cancer prevented
Child educated	Work days lost
Reduction in blood pressure	Increase in physical activity
Case of depression averted	Length of hospital stay
Quality-adjusted life-year saved	Life-year saved

Decisionmakers typically prefer to assess program costs against final outcomes, but when final outcomes are unavailable, CEAs must depend on intermediate outcomes. Therefore, intermediate outcomes should be used only in the following situations:

Intermediate outcomes are more closely associated with the intervention being examined:

Program Outcome chosen Rationale

Patient reminders Fully vaccinated child Link between vaccination and disease is not well established
Intermediate outcomes are directly measurable within the time frame of the study:

Program Outcome chosen Rationale

Smoking cessation campaign Number of quitters Cases of lung cancer prevented are too far in future
Cost data for a final outcome measure are insufficient:

Program Outcome chosen Rationale

Nurse home visitation Number of families reached Total child maltreatment cost is unknown
The relationship between the intermediate and final outcome is unknown:

Program Outcome chosen Rationale

HIV risk reduction Number of persons counseled Lack of evidence to link patients who have undergone counseling to cases of HIV prevented

Program	Outcome chosen	Rationale
Patient reminders	Fully vaccinated child	Link between vaccination and disease is not well established

Program	Outcome chosen	Rationale
Smoking cessation campaign	Number of quitters	Cases of lung cancer prevented are too far in future

Program	Outcome chosen	Rationale
Nurse home visitation	Number of families reached	Total child maltreatment cost is unknown

Program	Outcome chosen	Rationale
HIV risk reduction	Number of persons counseled	Lack of evidence to link patients who have undergone counseling to cases of HIV prevented

Outcome Implications in CEA

Costs. One implication of the outcome chosen for a CEA deals with its related costs. To a large extent, the outcome defines which costs will be included in the analysis.
Value. The value of health outcomes used in a CEA is important not only for defining costs but also for assessing the program's overall health benefit.

Outcomes include the tangible benefits of an intervention, which are subsequently weighed against the intervention's associated costs. However, the ultimate decision of whether to implement a program often depends more on the implicit value of the outcome.

The example below illustrates the decision-making process for choosing an appropriate outcome.

Outcomes: A Hypothetical Example

Nearly 40,000 people die in the United States annually as a result of motor-vehicle crashes. Approximately 40% of these crashes are alcohol-related.

Substantial research has been devoted to preventing the number of injuries and deaths suffered as a result of motor-vehicle crashes.

The Problem

A state policy maker is provided a list of interventions that have been proven to reduce the number of injuries and deaths attributable to motor-vehicle crashes. The interventions are in these three primary areas:

child safety seat use,
safety belt use, and
alcohol-impaired driving.

"Which interventions should receive funding and support?"

The Data

If we assume that total costs of the interventions are similar, the information below is useful for solving this problem.

Promote Child Safety Seat Use

Interventions	Outcomes
Community wide information and enhanced enforcement campaigns promote the use of child safety seats required by law in all 50 states. An example of this intervention would be a public display of proper safety seat use or mass mailings containing safety seat use information.	12.3% increase in correct child safety seat use
Distribution and education programs provide child safety seats to parents of low socioeconomic status at no cost or at a low cost. In addition, educational materials explaining the importance of child safety seat use are included.	6.4% decrease in fatal and nonfatal injuries 7% increase in correct child safety seat use
Incentive and education programs provide educational information to parents regarding the appropriate use and importance of child safety seats as well as incentive rewards (e.g., movie tickets or food coupons) for subsequent correct use.	9.9% increase in safety seat use

Promote Seat Belt Use

Interventions	Outcomes
Primary enforcement laws empower law enforcement officers to stop and issue citations to drivers for not wearing safety belts. (Without these laws, officers may not stop drivers solely for safety belt violations.)	9% decrease in fatal injuries 17% increase in observed safety belt use
Enhanced enforcement includes boosting current efforts to enforce existing safety belt laws (e.g., increasing the number of officers on duty to issue citations or providing more safety belt checkpoints).	11% decrease in fatal and nonfatal injuries 24% increase in observed safety belt use

Reduce Alcohol-Impaired Driving

Interventions	Outcomes
.08 Blood alcohol content (BAC) laws lower the BAC limit for drivers.	7% decrease in fatalities 5% decrease in alcohol-related crashes
Minimum legal drinking age (MLDA) laws set an age floor (e.g., age 21 years) for the purchase or consumption of alcoholic beverages.	17% decrease in alcohol-related crashes (resulting from increasing the MLDA)
Sobriety checkpoints allow law enforcement officers to stop and administer selective breath testing to drivers suspected of being intoxicated.	20% decrease in fatalities 13% decrease in alcohol-related crashes

Recommendations

The outcomes indicate that the state policy maker should support community-wide information and enhanced enforcement campaigns because the intervention increases the correct use of child safety seats by 12.3%, compared with 7% and 9.9% increases for the other two interventions.
The outcomes indicate that the policy maker should fund enhanced enforcement campaigns that increase the use of safety belts by nearly 25%.
The outcomes indicate that minimum legal drinking age laws should be supported to reduce alcohol-impaired driving because of their effect on alcohol-related crashes compared with .08 BAC laws and sobriety checkpoints.

Question

The state legislature reduces the budget; as a result, the policy maker has funding sufficient to support only the single intervention that will have the greatest effect on final outcomes (e.g., fatalities prevented).

Which intervention should the policy maker choose?

Answer

Only five interventions — distribution and education programs, primary enforcement laws, enhanced enforcement, .08 BAC laws, and sobriety checkpoints — report final outcomes such as fatal injuries or fatalities prevented.

If one category (e.g., exceptionally low child safety seat use) is not a primary concern for that particular state, the policy maker should allocate the dollars to sobriety checkpoints.

Sobriety checkpoints result in a 20% reduction in fatalities whereas the other four interventions result in fatality reduction ranging from 6.4% to 11%.

Before the sobriety checkpoints intervention is implemented in the community, policy makers would generally assess the overall intervention cost.

If the intervention cost is predicted to be approximately $165,000, a CEA would be useful to evaluate the outcome (fatalities prevented) relative to the $165,000 cost.

Decision Analysis

Decision analysis is a modeling technique that provides a systematic framework for decision making.

The decision model is an applied tool that has been adapted to public health for patient- and population-based decision making.
Decision models are informative and visually appealing for breaking down complicated problems into their components, including the primary goal, alternatives, chance events, and payoffs (outcomes). This decision tree shows a goal with two alternative interventions:

This decision tree shows a goal with two alternative interventions.

In public health, decision analysis is particularly useful for solving problems for which different alternatives (e.g., lower BAC laws, MLDA laws, or sobriety checkpoints for reducing the number of fatalities associated with alcohol-related motor-vehicle crashes) might be selected.

Decision modeling is important in CEA because it takes into account the uncertainty in events and outcomes.

For example, in the case of a sobriety checkpoints intervention, if "prevented fatality" is the outcome chosen for the study, a thorough decision model will include the uncertainty associated with effectiveness of the intervention, probable risk, and measurement error.

Decision Tree Components

The decision tree above is only one form of decision models. Decision trees are made up of three types of nodes: decision, chance, and terminal nodes

The "Goal" Component

The goal component explains the problem or labels the objective for the decision at hand.

For example, a state experiencing high fatalities as a result of alcohol-related motor-vehicle crashes could decide whether or not to implement sobriety checkpoints in all counties. The goal of the related decision problem is to reduce the number of fatalities. This decision tree outlines two alternatives:

This decision tree shows two alternatives to reduce motor-vehicle crash fatalities.

The "Alternative" Component

The alternative component begins the pathway for each program or intervention competing for the same resources.
The number of branches included at this first decision node (blue square) of the tree should be commensurate with the number of alternatives being considered.

For example, the state policy maker is considering only sobriety checkpoints. Note that current practice could include previously mandated BAC and MLDA laws, as shown in this decision tree:

This decision tree shows alternatives: BAC laws, sobriety checkpoints, and MLDA laws.

The "Chance Event" Component

An alternative generally leads to a chance event. A numerical probability is assigned to each event to account for uncertainty.
Probabilities are typically provided by previous studies of randomized clinical trials or expert opinion. Because the probabilities for any chance node (green circle) must sum to 1.0, the list of events must be exhaustive.

For example, if a pneumococcal conjugate vaccine is administered to a child, multiple chance events (e.g., pneumococcal disease, meningitis, or ear infections) might occur.

A separate branch should represent each chance event or disease state, and all probabilities assigned to the branches must sum to 1.0.

Now assume that prior studies suggest the probability of an alcohol-related crash is 40% under current practice (with no program) and 13% lower with sobriety checkpoints. These values are incorporated into this decision tree:

This decision tree shows the probabilities of an alcohol-related crash for no intervention and with sobriety checkpoints.

Note that the sum of probabilities at each chance node is 1.0 (e.g., 0.27 + 0.73 = 1.0).

The "Payoff" Component

The desirability of each outcome determines its payoff.
The payoff for a particular outcome is relative to the payoffs for other outcomes included in the analysis.

In the example below, assume the outcome is deaths per 100,000 persons. Currently, one fatality for every 100,000 persons is attributable to alcohol-related motor-vehicle crashes whereas only 0.8 deaths per 100,000 persons occur with sobriety checkpoints (i.e., the intervention reduces the fatality rate by 20%).

In this decision tree, values of 0.0 are assigned to the crash-free outcomes that result in no fatalities:

This decision tree shows values of 0.0 assigned to the crash-free outcomes that result in no fatalities.

Expected Value

Expected values use the probabilities associated with each chance event and the payoff value assigned to each outcome to determine which alternative should be chosen.
Whether or not to choose the alternative with the lower or higher expected value depends on the outcome.

For instance, the earlier "Payoff" Component

example uses deaths per 100,000 persons as the outcome; therefore, the lower expected value will determine the correct decision.

In contrast, if life-years saved were the outcome, the decision would be made based on the higher expected value.

Expected values are calculated simply by summing the probability-payoff products for each alternative. In the earlier example, the expected value (EV) for sobriety checkpoints is:

EV	=	(0.8 x 0.27)	+	(0.0 x 0.73)
	=	0.22

The decision tree below shows the expected values at each chance node (green circle). Each value is shown in a green box to the right of the chance node.

This decision tree shows the expected values for the sobriety checkpoints intervention and for no intervention to be 0.22 and 0.40, respectively.

In the decision tree, the expected values for the sobriety checkpoints intervention and for no intervention are shown as 0.22 and 0.40, respectively.

The appropriate decision is to implement sobriety checkpoints because they reduce the number of fatalities attributable to alcohol-related motor-vehicle crashes compared with current practice.

In this example, the better choice (i.e., sobriety checkpoints) is straightforward.

Steps in Decision Analysis

Define the problem and specify the goal or objective of the analysis.
Develop a comprehensive model that takes into account all possible pathways and chance events over time.
Estimate probabilities at chance nodes by using published sources and expert opinion. (Be sure the sum of all probabilities at each node is 1.0.)
Assign payoff values to outcomes.
Calculate expected values.
Further evaluate uncertainty.

Sensitivity Analysis

Uncertainty can be further evaluated by sensitivity analysis. Essentially, sensitivity analysis involves providing a range for all probabilities and outcome values included in the decision model that are particularly important to the results.
The reason for conducting sensitivity analyses after the model has been run with initial estimates is to increase the validity of conclusions drawn from the model.
If the decision does not change after key inputs are tested, results will be more substantive.

For example, the chance of an alcohol-related crash occurring at current practice might be tested from 20% to 60%.

Likewise, the payoff value of 0.8 deaths per 100,000 persons attributable to alcohol-related crashes might be tested from 0.2 to 1.5 deaths per 100,000 persons.

Advantages and Limitations of Decision Analysis

Decision analysis is most useful for problems with near-term outcomes or final outcomes that occur in straightforward paths.
The more complicated the pathway, the more difficult it is for decision analysis to take into account all possible avenues of chance events and decisions.
Both a limitation and advantage of decision analysis is its dependence on estimated values and hypothetical events.

The "guesswork" inherent in the model is a limitation but applies to any modeling technique.
The primary advantage of decision analysis is its ability to tackle difficult research problems that otherwise would not be assessed because of time or resource constraints.

An Example: Decision Analysis

Suppose that current practice for curbing alcohol-related driving includes MLDA laws only.

If alcohol-related motor-vehicle crash prevalence is high, a policy maker might consider imposing tighter restrictions and increasing law enforcement capabilities. Therefore, BAC laws and sobriety checkpoints should be included in the decision.

Assume that under MLDA laws only, 40% of crashes are related to alcohol and that BAC laws reduce that rate by 5%. In addition, assume that one death per 100,000 persons is attributable to MLDA laws whereas BAC laws reduce that number 7%.

This information as well as that for sobriety checkpoints provided in the decision tree for the earlier "Payoff" Component

example can be used to add a third alternative branch to the decision tree.

This decision tree shows 3 alternatives: BAC laws, sobriety checkpoints, and MLDA laws.

Once the new branch is added, expected values are calculated to generate a new decision rule.

EV	=	(0.93 x 0.35)	+	(0.0 x 0.65)
	=	0.33

This decision tree shows that the sobriety checkpoints intervention has a greater impact than either BAC or MLDA laws.

In this case, the decision rule is the same. The sobriety checkpoints intervention should be implemented because it has a greater impact on reducing the number of fatalities attributable to alcohol-related accidents compared with BAC and MLDA laws.

Interpreting CEA Results

Cost Effectiveness Ratios

Once a CEA has been conducted and cost effectiveness has been calculated, ratios combining the expected results are calculated and reported. The cost referred to in the CER is a net cost value discussed earlier in the section: "Which Costs Are Included in the CEA?"

There are three types of cost effectiveness ratios (CERs):

Average cost-effectiveness ratio (ACER),
Marginal cost-effectiveness ratio (MCER), and
Incremental cost-effectiveness ratio (ICER).

ACER

An ACER

deals with a single intervention and evaluates that intervention against its baseline option (e.g., no program or current practice).
is calculated by dividing the net cost of the intervention by the total number of health outcomes prevented by the intervention.

An Example of an ACER

Intervention	Net cost	Total outcomes (life-years saved)	ACER cost per life-year saved
Home vaccination program	$50,000	8	$6,250

MCER

The marginal cost-effectiveness ratio (MCER) assesses the specific changes in cost and effect when a program is expanded or contracted.

Because the majority of programs that are cost effective are considered good investments only at a certain level, the MCER and ACER are often considered simultaneously.

This figure shows the ACER and MCER for the vaccination program example:

At low vaccination coverage rates (approximately 70%), the ACER is negative, indicating a savings in cost. As that percentage grows, however, so does the cost per case prevented because the marginal cost per each additional person vaccinated is much higher than the average cost.

In general, the MCER is used in conjunction with the ACER as a tool to determine the most efficient level of program implementation.

Once this level is determined, the ACERs of that and other independent programs that result in the same outcome can be compared.

ICER

An ICER

compares the differences between the costs and health outcomes of two alternative interventions that compete for the same resources, and
is generally described as the additional cost per additional health outcome.

When comparing two competing programs incrementally, one program should be compared with the next-less-effective alternative.

The ICER numerator includes the differences in program costs, averted disease costs, and averted productivity losses if applicable. Similarly, the ICER denominator is the difference in health outcomes.

An Example: Decision Analysis CERs

The values and estimates used in this analysis were modified from various studies for the purposes of this example and should not be considered literally.

Consider a policymaker that is trying to decide whether or not to implement a sobriety checkpoints program if BAC and MLDA laws (baseline) are already in place. The probabilities and payoffs (costs and outcomes) for each branch pathway are shown in this decision tree:

Decision tree for the sobriety checkpoints intervention when BAC and MLDA laws (baseline) are already in place.

The outcome of interest is deaths per 100,000 persons.
Total costs for the baseline option include trial and sanctioning and law enforcement costs.
Total costs for the sobriety checkpoint program include officer wages, equipment, publicity, travel delay, and sanctioning costs.
Both total cost estimates also include the costs (medical and productivity losses) associated with a fatality resulting from an alcohol-related crash.

The expected total cost estimates (i.e., values that take probabilities into account) are shown in the decision tree below. Each value is shown in a green box to the right of the chance node (green circle).

The additional cost of sobriety checkpoints is:

$12,360

–

$9,200

$3,160

The expected total outcome values indicate that sobriety checkpoints are more effective (i.e., result in fewer deaths per 100,000 persons) than the baseline option. This analysis is presented in this decision tree:

This decision tree shows that the expected total outcome values indicate that sobriety checkpoints are more effective (i.e., result in fewer deaths per 100,000 persons) than the baseline option.

The cost and outcome results considered separately warrant the use of CEA. Sobriety checkpoints are both costlier and more effective than what is currently being done. This decision tree shows both results.

This decision tree shows the cost and outcome results with and without sobriety checkpoints.

From the numbers in this decision tree, the ICER comparing sobriety checkpoints to the baseline option is:

ICER

(

Total cost_B

–

Total cost_A

)

(

Total outcomes_A

–

Total outcomes_B

)

ICER

(

$12,360

–

$9,200

)

(

0.37

–

0.22

)

ICER

$21,066.67 per death prevented

If program costs and health outcomes are different but averted disease costs (i.e., alcohol-related crash costs) are the same, then the ICER should be computed as:

ICER

(

Add'l program cost

–

Add'l cost of disease averted

)

Add'l health outcomes

ICER

(

$3,160

–

)

0.15

ICER

$21,066.67 per death prevented

This analysis suggests that each death prevented by sobriety checkpoints will cost $21,066.67.

Exclusion Criteria

The programs are arrayed in order from least to most effective in terms of outcomes prevented.
This is done first so that the ICERs may be calculated and resources allocated appropriately.

This table shows data and calculations for Program A, an expanded Program A, and Programs B and C:

Programs	Prevented outcomes	Total costs	ACER	MCER	ICER
Independent programs
Program A	10	$150	$15
Expanded program A	12	$200		$25
Mutually exclusive programs
Program A	10	$150
Program B	20	$300			$15
Program C	25	$250			–$10

Notes for Table Columns

ACER

ACERs can be calculated with an alternative formula similar to that for ICERs above if the baseline is "no program":

ACER

(

$150

–

)

(

–

)

ACER

$15 per additional outcome prevented

MCER

MCER

(

$200

–

$150

)

(

–

)

MCER

$25 per additional outcome prevented

ICER

ICER_{B to A}

(

$300

–

$150

)

(

–

)

ICER_{B to A}

$15 per additional outcome prevented

and

ICER_{C to B}

(

$250

–

$300

)

(

–

)

ICER_{C to B}

–$10 per additional outcome prevented

If program A is expanded, the MCER is $25. Because $25 is higher than $15, a decisionmaker needs to decide whether or not expansion is worth the additional cost.

If two competing programs are compared, such as programs A and B, the ICER is $15. The comparison between programs C and B shows a cost saving of $10 for program C over program B.

The negative ICER for program C indicates that program B is "strongly dominated." In other words, program B is more costly and less effective than program C.

Strongly dominated programs should be excluded from the set of alternatives so they do not consume limited resources. In addition, the ICERs should be recalculated so that the magnitude of the negative ICER is not misleading, as shown in this table:

Mutually exclusive programs	Prevented outcomes	Total costs	ICER
No program	0	0	—
Program A	10	$150	$15
~~Program B~~	20	~~$300~~	~~$15~~
Program C	25	$250	$6.67

ICER Calculation for Program C

ICER

(

$250

–

$150

)

(

–

)

ICER

$6.67 per additional outcome prevented

If a program's ICER is higher than the next most effective program, the program is "weakly dominated." In effect, weakly dominated programs produce effectiveness at a higher marginal cost, which means that some combination of at least two other alternative programs is more cost effective.

In this table, Program D is compared with Program C because they are ranked in order of increasing effectiveness:

Mutually exclusive programs	Prevented outcomes	Total costs	ICER
Program A	10	$150	—
~~Program B~~	20	~~$300~~	~~$15~~
Program C	25	$250	$6.67
Program D	40	$325	$5

In this case, some combination of programs A and D is more cost effective than program C. Therefore, program C should be removed from the list of alternatives and the results recalculated accordingly (i.e., program D compared with program A). This table shows the final results:

Mutually exclusive programs	Prevented outcomes	Total costs	ICER
Program A	10	$150	—
~~Program B~~	20	~~$300~~	~~$15~~
~~Program C~~	25	~~$250~~	~~$6.67~~
Program D	40	$325	$5.83

Decision Guidelines

Decision making is a complex process that takes into account much more than numbers and calculations.

Value judgments are often implicit in the choice made by a decisionmaker.

Even so, calculations that allow comparisons to be made between independent and competing programs are important and generally valued by decisionmakers.

Users of CEA can allocate limited resources and make decisions more efficiently if certain decision rules or guidelines are followed.

When Assessing Independent Programs

Order the programs from least to most effective.
Eliminate the strongly dominated programs.
Calculate ACERs.
Implement programs in order of increasing ACER until either resources are exhausted or the ACER is equal in value to one unit of effectiveness.

When Assessing a Mix of Independent and Mutually Exclusive Programs

Form groups of mutually exclusive programs.
Order programs within each group from least to most effective.

Within Each Group

Calculate the ICER.
Eliminate both strongly and weakly dominated programs.
Calculate the ACER of each independent program.
Rank all programs in order of increasing ratio.
Implement programs in order of increasing ACER until either resources are exhausted or the ratio is equal in value to one unit of effectiveness.

Presentation of Results

Presenting CEA results in a comprehensive and concise manner is imperative to CEA's overall usefulness among decisionmakers. The information included in a CEA is practical and advantageous to the decision making process.

Thus, the presentation of a CEA should include these eight major elements:

Eight Elements of a CEA
A clear study perspective, time frame, and analytic horizon An explicitly defined study question Relevant assumptions underlying the study Detailed descriptions of the interventions Existing evidence of the interventions' effectiveness Proper identification of all relevant costs: decide whether to include or exclude productivity losses apply appropriate discount rate confirm that included costs are relevant to perspective An appropriate choice of outcome: calculate a suitable CER report ICER results (unless the only comparator is baseline) conduct sensitivity analyses A comprehensive discussion of the results: deal with issues of concern address implications of underlying assumptions

Eight Elements of a CEA

A clear study perspective, time frame, and analytic horizon
An explicitly defined study question
Relevant assumptions underlying the study
Detailed descriptions of the interventions
Existing evidence of the interventions' effectiveness
Proper identification of all relevant costs:
- decide whether to include or exclude productivity losses
- apply appropriate discount rate
- confirm that included costs are relevant to perspective
An appropriate choice of outcome:
- calculate a suitable CER
- report ICER results (unless the only comparator is baseline)
- conduct sensitivity analyses
A comprehensive discussion of the results:
- deal with issues of concern
- address implications of underlying assumptions

Test Your Understanding

Should a highly effective intervention always be supported? Why or why not?
Why is the net cost — rather than total program costs — relevant for CERs?
Provide a criticism of using the human capital (HC) approach to measure productivity losses.
Calculate the ACER for a program with these costs and outcome:
- Program cost = $10,000
- Disease cost averted = $2,000
- Productivity losses averted = $3,000
- Life-years saved relative to no program = 5
Calculate the MCER for expanding the program:
- Total cost_A = $15,000
- Total cost_Ax = $20,000
- Total outcomes_A = 5
- Total outcomes_Ax = 7
where subscripts:
- "A" refers to the original program and
- "Ax" refers to the expanded program.
Calculate the ICER for two alternative programs, "A" and "B," competing for resources, given:
- Total cost_A = $15,000
- Total cost_B = $ 30,000
- Total outcomes_A = 8
- Total outcomes_B = 5
where a program outcome is the count of disease cases attributable to the program.
Explain the difference between strongly and weakly dominated programs.

Our Answers

Should a highly effective intervention always be supported? Why or why not?

A highly effective intervention should not necessarily be supported regardless of cost.
Both cost and outcome should be taken into account when deciding whether or not to allocate funding to a particular program.
Why is the net cost — rather than total program costs — relevant for CERs?

The net cost takes into account the benefits that are attributable to the intervention and therefore provides a "true" estimate of cost.
Without subtracting future savings, the cost would be overestimated, resulting in a large CER.
Provide a criticism of using the human capital (HC) approach to measure productivity losses.

By using forgone wages to calculate productivity losses, the HC approach systematically undervalues persons who receive lower wages (e.g., women and certain blue-collar workers).
Calculate the ACER for a program with these costs and outcome:
- Program cost = $10,000
- Disease cost averted = $2,000
- Productivity losses averted = $3,000
- Life-years saved relative to no program = 5
ACER = ( Program costs – Averted disease costs – Averted productivity losses ) / Health outcomes prevented

ACER = ( $10,000 – $2,000 – $3,000 ) / 5

ACER = $1,000 per life-year saved
Calculate the MCER for expanding the program:
- Total cost_A = $15,000
- Total cost_Ax = $20,000
- Total outcomes_A = 5
- Total outcomes_Ax = 7
where subscripts:
- "A" refers to the original program and
- "Ax" refers to the expanded program.
The MCER is the ratio of the differences in total costs and total outcomes between the initial program level and expansion level.

MCER = ( Total cost_Ax – Total cost_A ) / ( Total outcomes_Ax – Total outcomes_A )

MCER = ( $20,000 – $15,000 ) / ( 7 – 5 )

MCER = $5,000 / 2

MCER = $2,500 per outcome

Comparing the MCER for expanding program A to its ACER in the answer to question 4 suggests maintaining the program at its current production level rather than funding expansion.
Calculate the ICER for two alternative programs, "A" and "B," competing for resources, given:
- Total cost_A = $15,000
- Total cost_B = $ 30,000
- Total outcomes_A = 8
- Total outcomes_B = 5
where a program outcome is the count of disease cases attributable to the program.

The ICER is the ratio of the differences in total costs and total outcomes between the two programs.

ICER = ( Total cost_B – Total cost_A ) / ( Total outcomes_A – Total outcomes_B )

ICER = ( $30,000 – $15,000 ) / ( 8 – 5 )

ICER = $15,000 / 3

ICER = $5,000 per disease case prevented
Explain the difference between strongly and weakly dominated programs.

A strongly dominated program is one for which an alternative exists that is both more effective and less expensive. Weakly dominated programs generate effectiveness at a higher marginal cost than an alternative program.
Any program that is dominated by another should be removed from the list of competing alternatives so that the most efficient allocation of resources can be achieved.

Glossary — CEA

Analytic horizon

The period over which costs and outcomes associated with the intervention accrue. Costs and outcomes are measured during the intervention and after it ends.

Audience

The consumers or users of the results of a cost effectiveness analysis.

Average cost-effectiveness ratio (ACER)

Ratio that deals with a single intervention and evaluates that intervention against its baseline option (e.g., no program or current practice).

The ACER is calculated by dividing the net cost of the intervention by the total number of health outcomes prevented by the intervention.

Cost effectiveness analysis (CEA)

An economic evaluation method in which all costs are related to a single common outcome or natural unit (e.g., life years saved).

Cost of illness (COI)

A method generally used to determine the economic burden of a disease by assessing the direct medical and nonmedical costs of disease averted as well as productivity losses averted.

Cost-effectiveness ratio (CER)

An end result that summarizes the intervention's net cost and effectiveness. The three types of CERs are:

Average cost-effectiveness ratio (ACER),
Marginal cost-effectiveness ratio (MCER), and
Incremental cost-effectiveness ratio (ICER).

Decision analysis

A modeling technique that systematically aids in decision making by considering the uncertainty associated with alternative solutions to a particular problem

Disease costs averted

All expected costs that are related to a disease or condition and that are not incurred by society (or some other entity) as a result of implementing a program or intervention.

Economic burden

The total cost incurred by society for a particular disease or condition.

Economic evaluation

The use of applied analytic techniques to identify, measure, value, and compare the net costs and outcomes of alternative interventions.

Effective

The improved health outcome that a prevention strategy can achieve under typical community-based settings.

Efficacious

The improved health outcome that a prevention strategy can achieve under ideal settings.

Final outcome

The ultimate outcome of interest, such as years of life gained or deaths prevented.

Human capital approach

As a method of COI, the human capital approach assesses productivity losses by using forgone income attributable to morbidity and premature mortality of a disease or condition.

Incremental cost-effectiveness ratio (ICER)

The additional cost of one unit of outcome when we change to a more effective, mutually exclusive intervention.

Intermediate outcome

Near-term effects of an intervention, such as persons screened, rate of condom use, or number of vaccine doses administered.

Marginal cost-effectiveness ratio (MCER)

The additional cost of one unit expansion of a single intervention.

Model

A simplified yet accurate representation of a program or intervention based on a set of assumptions.

Outcome measure

A measurement unit used to assess the effectiveness of a program or intervention.

Output

The product or service produced (e.g., patients seen, patient days).

Payer

An individual or organization that provides money to pay for health-care services.

Perspective

The viewpoint of the bearers of the costs and benefits of an intervention (e.g., society, government, health-care providers, businesses, or patients).

Productivity losses

Refer to the value of time (usually in terms of wages) that is forgone as a result of the morbidity or premature mortality associated with a disease or condition.

Program costs

Include all fixed and variable costs that are incurred as a result of program implementation and maintenance.

Prospective study

A study in which the events of interest (costs and outcomes) have not yet taken place when the study begins.

Retrospective study

A study in which the events of interest (costs and outcomes) have already occurred when the study begins.

Sensitivity analysis

Method for testing the validity of decision analysis findings by providing a range for all probabilities and outcome values included in the decision model that are particularly important to the results.

Societal perspective

The broadest possible perspective for an economic evaluation. It includes all program costs, no matter who incurs them, and all program consequences, no matter who experiences them.

Target population

The population(s) for whom the program is intended.

Time frame

The period over which program or intervention costs are tracked.

Number of persons with averted disease conditions	=	0.6 x 50
	=	30

Net cost	=	$740,000	–	$600,000	–	$300,000
	=	–$160,000

Net cost	=	$740,000	–	$600,000
	=	$140,000