Cost Effectiveness Analysis Page 3    HHS    CDC

See the previous page in the tutorialPrev   NextSee the next page in the tutorial ContentsOpen this tutorial's Contents page in a new window   GlossaryOpen this tutorial's Glossary page in a new window   PrefaceOpen the Series Preface page in a new window   Print AllIn a new window, see a composite of all the pages in this tutorial. It will have all sections expanded - ready to print.   HelpOpen the Help page in a new window. 
Find out how to run the tutorial
 - what the commands on this line do,
 - how to print,
 - and more.
 IntroductionJump to page 1.
 Framing CEAJump to page 2.
Which Outcomes are Relevant in CEA?
Introduction
Health outcomes set a CEA apart from other forms of economic evaluation (i.e., CA, CBA, and CUA).
The table below lists possible health outcomes. Each outcome should be considered in terms of "cost per health outcome" (e.g., cost per additional vaccination and cost per life-year saved).
Health Outcomes
Additional person vaccinated Additional person screened
Fatal injury prevented Increase in child safety seat use
Pregnancy prevented Case of lung cancer prevented
Child educated Work days lost
Reduction in blood pressure Increase in physical activity
Case of depression averted Length of hospital stay
Quality-adjusted life-year saved Life-year saved
Decisionmakers typically prefer to assess program costs against final outcomes, but when final outcomes are unavailable, CEAs must depend on intermediate outcomes. Therefore, intermediate outcomes should be used only in the following situations:
  • Intermediate outcomes are more closely associated with the intervention being examined:
    Program Outcome chosen Rationale
    Patient reminders Fully vaccinated child Link between vaccination and disease is not well established
  • Intermediate outcomes are directly measurable within the time frame of the study:
    Program Outcome chosen Rationale
    Smoking cessation campaign Number of quitters Cases of lung cancer prevented are too far in future
  • Cost data for a final outcome measure are insufficient:
    Program Outcome chosen Rationale
    Nurse home visitation Number of families reached Total child maltreatment cost is unknown
  • The relationship between the intermediate and final outcome is unknown:
    Program Outcome chosen Rationale
    HIV risk reduction Number of persons counseled Lack of evidence to link patients who have undergone counseling to cases of HIV prevented
Outcome Implications in CEA
  • Costs. One implication of the outcome chosen for a CEA deals with its related costs. To a large extent, the outcome defines which costs will be included in the analysis.
  • Value. The value of health outcomes used in a CEA is important not only for defining costs but also for assessing the program's overall health benefit.
Outcomes include the tangible benefits of an intervention, which are subsequently weighed against the intervention's associated costs. However, the ultimate decision of whether to implement a program often depends more on the implicit value of the outcome.
The example below illustrates the decision-making process for choosing an appropriate outcome.
Outcomes: A Hypothetical Example
Nearly 40,000 people die in the United States annually as a result of motor-vehicle crashes. Approximately 40% of these crashes are alcohol-related.
Substantial research has been devoted to preventing the number of injuries and deaths suffered as a result of motor-vehicle crashes.
The Problem
A state policy maker is provided a list of interventions that have been proven to reduce the number of injuries and deaths attributable to motor-vehicle crashes. The interventions are in these three primary areas:
  • child safety seat use,
  • safety belt use, and
  • alcohol-impaired driving.
"Which interventions should receive funding and support?"
The Data
If we assume that total costs of the interventions are similar, the information below is useful for solving this problem.
Promote Child Safety Seat Use
Interventions Outcomes
Community wide information and enhanced enforcement campaigns promote the use of child safety seats required by law in all 50 states. An example of this intervention would be a public display of proper safety seat use or mass mailings containing safety seat use information.
  • 12.3% increase in correct child safety seat use
Distribution and education programs provide child safety seats to parents of low socioeconomic status at no cost or at a low cost. In addition, educational materials explaining the importance of child safety seat use are included.
  • 6.4% decrease in fatal and nonfatal injuries
  • 7% increase in correct child safety seat use
Incentive and education programs provide educational information to parents regarding the appropriate use and importance of child safety seats as well as incentive rewards (e.g., movie tickets or food coupons) for subsequent correct use.
  • 9.9% increase in safety seat use
Promote Seat Belt Use
Interventions Outcomes
Primary enforcement laws empower law enforcement officers to stop and issue citations to drivers for not wearing safety belts. (Without these laws, officers may not stop drivers solely for safety belt violations.)
  • 9% decrease in fatal injuries
  • 17% increase in observed safety belt use
Enhanced enforcement includes boosting current efforts to enforce existing safety belt laws (e.g., increasing the number of officers on duty to issue citations or providing more safety belt checkpoints).
  • 11% decrease in fatal and nonfatal injuries
  • 24% increase in observed safety belt use
Reduce Alcohol-Impaired Driving
Interventions Outcomes
.08 Blood alcohol content (BAC) laws lower the BAC limit for drivers.
  • 7% decrease in fatalities
  • 5% decrease in alcohol-related crashes
Minimum legal drinking age (MLDA) laws set an age floor (e.g., age 21 years) for the purchase or consumption of alcoholic beverages.
  • 17% decrease in alcohol-related crashes (resulting from increasing the MLDA)
Sobriety checkpoints allow law enforcement officers to stop and administer selective breath testing to drivers suspected of being intoxicated.
  • 20% decrease in fatalities
  • 13% decrease in alcohol-related crashes
Recommendations
  • The outcomes indicate that the state policy maker should support community-wide information and enhanced enforcement campaigns because the intervention increases the correct use of child safety seats by 12.3%, compared with 7% and 9.9% increases for the other two interventions.
  • The outcomes indicate that the policy maker should fund enhanced enforcement campaigns that increase the use of safety belts by nearly 25%.
  • The outcomes indicate that minimum legal drinking age laws should be supported to reduce alcohol-impaired driving because of their effect on alcohol-related crashes compared with .08 BAC laws and sobriety checkpoints.
Question
The state legislature reduces the budget; as a result, the policy maker has funding sufficient to support only the single intervention that will have the greatest effect on final outcomes (e.g., fatalities prevented).
Which intervention should the policy maker choose?
Answer
Only five interventions — distribution and education programs, primary enforcement laws, enhanced enforcement, .08 BAC laws, and sobriety checkpoints — report final outcomes such as fatal injuries or fatalities prevented.
If one category (e.g., exceptionally low child safety seat use) is not a primary concern for that particular state, the policy maker should allocate the dollars to sobriety checkpoints.
Sobriety checkpoints result in a 20% reduction in fatalities whereas the other four interventions result in fatality reduction ranging from 6.4% to 11%.
Before the sobriety checkpoints intervention is implemented in the community, policy makers would generally assess the overall intervention cost.
If the intervention cost is predicted to be approximately $165,000, a CEA would be useful to evaluate the outcome (fatalities prevented) relative to the $165,000 cost.
Decision Analysis
Decision analysis is a modeling technique that provides a systematic framework for decision making.
  • The decision model is an applied tool that has been adapted to public health for patient- and population-based decision making.
  • Decision models are informative and visually appealing for breaking down complicated problems into their components, including the primary goal, alternatives, chance events, and payoffs (outcomes). This decision tree shows a goal with two alternative interventions:
This decision tree shows a goal with two alternative interventions.
In public health, decision analysis is particularly useful for solving problems for which different alternatives (e.g., lower BAC laws, MLDA laws, or sobriety checkpoints for reducing the number of fatalities associated with alcohol-related motor-vehicle crashes) might be selected.
Decision modeling is important in CEA because it takes into account the uncertainty in events and outcomes.
For example, in the case of a sobriety checkpoints intervention, if "prevented fatality" is the outcome chosen for the study, a thorough decision model will include the uncertainty associated with effectiveness of the intervention, probable risk, and measurement error.
Decision Tree Components
The decision tree above is only one form of decision models. Decision trees are made up of three types of nodes: decision, chance, and terminal nodes
The "Goal" Component
The goal component explains the problem or labels the objective for the decision at hand.
For example, a state experiencing high fatalities as a result of alcohol-related motor-vehicle crashes could decide whether or not to implement sobriety checkpoints in all counties. The goal of the related decision problem is to reduce the number of fatalities. This decision tree outlines two alternatives:
This decision tree shows two alternatives to reduce motor-vehicle crash fatalities.
The "Alternative" Component
  • The alternative component begins the pathway for each program or intervention competing for the same resources.
  • The number of branches included at this first decision node (blue square) of the tree should be commensurate with the number of alternatives being considered.
For example, the state policy maker is considering only sobriety checkpoints. Note that current practice could include previously mandated BAC and MLDA laws, as shown in this decision tree:
This decision tree shows alternatives: BAC laws, sobriety checkpoints, and MLDA laws.
The "Chance Event" Component
  • An alternative generally leads to a chance event. A numerical probability is assigned to each event to account for uncertainty.
  • Probabilities are typically provided by previous studies of randomized clinical trials or expert opinion. Because the probabilities for any chance node (green circle) must sum to 1.0, the list of events must be exhaustive.
For example, if a pneumococcal conjugate vaccine is administered to a child, multiple chance events (e.g., pneumococcal disease, meningitis, or ear infections) might occur.
A separate branch should represent each chance event or disease state, and all probabilities assigned to the branches must sum to 1.0.
Now assume that prior studies suggest the probability of an alcohol-related crash is 40% under current practice (with no program) and 13% lower with sobriety checkpoints. These values are incorporated into this decision tree:
This decision tree shows the probabilities of an alcohol-related crash for no intervention and with sobriety checkpoints.
Note that the sum of probabilities at each chance node is 1.0 (e.g., 0.27 + 0.73 = 1.0).
The "Payoff" Component
  • The desirability of each outcome determines its payoff.
  • The payoff for a particular outcome is relative to the payoffs for other outcomes included in the analysis.
In the example below, assume the outcome is deaths per 100,000 persons. Currently, one fatality for every 100,000 persons is attributable to alcohol-related motor-vehicle crashes whereas only 0.8 deaths per 100,000 persons occur with sobriety checkpoints (i.e., the intervention reduces the fatality rate by 20%).
In this decision tree, values of 0.0 are assigned to the crash-free outcomes that result in no fatalities:
This decision tree shows values of 0.0 assigned to the crash-free outcomes that result in no fatalities.
Expected Value
  • Expected values use the probabilities associated with each chance event and the payoff value assigned to each outcome to determine which alternative should be chosen.
  • Whether or not to choose the alternative with the lower or higher expected value depends on the outcome.
For instance, the earlier "Payoff" ComponentJump to another section on this page. Use the Back command to return. example uses deaths per 100,000 persons as the outcome; therefore, the lower expected value will determine the correct decision.
In contrast, if life-years saved were the outcome, the decision would be made based on the higher expected value.
Expected values are calculated simply by summing the probability-payoff products for each alternative. In the earlier example, the expected value (EV) for sobriety checkpoints is:
EV = (0.8 x 0.27) + (0.0 x 0.73)
= 0.22
The decision tree below shows the expected values at each chance node (green circle). Each value is shown in a green box to the right of the chance node.
This decision tree shows the expected values for the sobriety checkpoints intervention and for no intervention to be 0.22 and 0.40, respectively.
In the decision tree, the expected values for the sobriety checkpoints intervention and for no intervention are shown as 0.22 and 0.40, respectively.
The appropriate decision is to implement sobriety checkpoints because they reduce the number of fatalities attributable to alcohol-related motor-vehicle crashes compared with current practice.
In this example, the better choice (i.e., sobriety checkpoints) is straightforward.
Steps in Decision Analysis
  1. Define the problem and specify the goal or objective of the analysis.
  2. Develop a comprehensive model that takes into account all possible pathways and chance events over time.
  3. Estimate probabilities at chance nodes by using published sources and expert opinion. (Be sure the sum of all probabilities at each node is 1.0.)
  4. Assign payoff values to outcomes.
  5. Calculate expected values.
  6. Further evaluate uncertainty.
Sensitivity Analysis
  • Uncertainty can be further evaluated by sensitivity analysis. Essentially, sensitivity analysis involves providing a range for all probabilities and outcome values included in the decision model that are particularly important to the results.
  • The reason for conducting sensitivity analyses after the model has been run with initial estimates is to increase the validity of conclusions drawn from the model.
  • If the decision does not change after key inputs are tested, results will be more substantive.
For example, the chance of an alcohol-related crash occurring at current practice might be tested from 20% to 60%.
Likewise, the payoff value of 0.8 deaths per 100,000 persons attributable to alcohol-related crashes might be tested from 0.2 to 1.5 deaths per 100,000 persons.
Advantages and Limitations of Decision Analysis
  • Decision analysis is most useful for problems with near-term outcomes or final outcomes that occur in straightforward paths.
  • The more complicated the pathway, the more difficult it is for decision analysis to take into account all possible avenues of chance events and decisions.
  • Both a limitation and advantage of decision analysis is its dependence on estimated values and hypothetical events.
    The "guesswork" inherent in the model is a limitation but applies to any modeling technique.
  • The primary advantage of decision analysis is its ability to tackle difficult research problems that otherwise would not be assessed because of time or resource constraints.
An Example: Decision Analysis
Suppose that current practice for curbing alcohol-related driving includes MLDA laws only.
If alcohol-related motor-vehicle crash prevalence is high, a policy maker might consider imposing tighter restrictions and increasing law enforcement capabilities. Therefore, BAC laws and sobriety checkpoints should be included in the decision.
Assume that under MLDA laws only, 40% of crashes are related to alcohol and that BAC laws reduce that rate by 5%. In addition, assume that one death per 100,000 persons is attributable to MLDA laws whereas BAC laws reduce that number 7%.
This information as well as that for sobriety checkpoints provided in the decision tree for the earlier "Payoff" ComponentJump to another section on this page. Use the Back command to return. example can be used to add a third alternative branch to the decision tree.
This decision tree shows 3 alternatives: BAC laws, sobriety checkpoints, and MLDA laws.
Once the new branch is added, expected values are calculated to generate a new decision rule.
EV = (0.93 x 0.35) + (0.0 x 0.65)
= 0.33
This decision tree shows that the sobriety checkpoints intervention has a greater impact than either BAC or MLDA laws.
In this case, the decision rule is the same. The sobriety checkpoints intervention should be implemented because it has a greater impact on reducing the number of fatalities attributable to alcohol-related accidents compared with BAC and MLDA laws.
Test Your Understanding
  1. What distinguishes CEA from CBA?
  2. Why are value judgments inherent in CEA decision making?
  3. Why is "number of children educated" not a suitable health outcome for state policy makers wishing to distribute tobacco settlement monies appropriately?
  4. What is necessary for the comparison of two alternative health conditions with CEA?
  5. Which one of these reasons does NOT justify using an intermediate outcome in CEA?
    Insufficient cost data
    Researcher interest in the intermediate effect
    No close association between final outcome and intervention
    Outcome measurement exceeds the study time frame
    No link between intermediate and final outcome
  6. Suppose a lifetime UV exposure reduction program effectively prevents cases of melanoma. Assume that 20% of persons with high lifetime exposures develop melanoma.
    1. If 150 persons undergo the program, how many cases of melanoma are prevented?
    2. Would medical costs associated with melanoma be included in the scenario?
  7. Which step in decision analysis is missing from this list?
    • Define problem
    • Develop model
    • Estimate probabilities
    • Assign payoff values
    • Calculate expected values
Our Answers
  1. What distinguishes CEA from CBA?
    CEA assesses health outcomes in natural or physical units (e.g., additional patient screened) whereas CBA converts outcomes to dollars.
  2. Why are value judgments inherent in CEA decision making?
    Decisionmakers often exercise judgment when assessing the results of CEA studies. In effect, cost cutoff points are established above which no program will be funded.
    Decisionmakers must answer questions such as:
    • Is an additional life-year worth $100,000?
    • Is a case of depression prevented worth $50,000?
    • Is the money used more efficiently if allocated in a different manner?
  3. Why is "number of children educated" not a suitable health outcome for state policy makers wishing to distribute tobacco settlement monies appropriately?
    State policy makers are responsible to their constituents for how funds are allocated to improve the public's health.
    Tobacco-control programs that are definitively linked to final outcomes (e.g., "cases of lung cancer prevented") would be more appropriate for state support.
  4. What is necessary for the comparison of two alternative health conditions with CEA?
    For CEA to be useful in this case, a final outcome common to both interventions is necessary.
  5. Which one of these reasons does NOT justify using an intermediate outcome in CEA?
    Insufficient cost data
    Researcher interest in the intermediate effect
    No close association between final outcome and intervention
    Outcome measurement exceeds the study time frame
    No link between intermediate and final outcome
    The second reason:
    Researcher interest in the intermediate effect
    is NOT reason enough to warrant its use in place of a final outcome measure.
  6. Suppose a lifetime UV exposure reduction program effectively prevents cases of melanoma. Assume that 20% of persons with high lifetime exposures develop melanoma.
    1. If 150 persons undergo the program, how many cases of melanoma are prevented?
      20% of 150 is 30. So 30 cases of melanoma are prevented under these circumstances.
    2. Would medical costs associated with melanoma be included in the scenario?
      Yes. If the link between persons in the program and prevented cases of melanoma is known, all costs associated with the final outcome should be included.
  7. Which step in decision analysis is missing from this list?
    • Define problem
    • Develop model
    • Estimate probabilities
    • Assign payoff values
    • Calculate expected values
    The final step, "Further evaluate uncertainty," is not included in the list.
    The study is not complete after construction of the tree and assignment of numerical estimates to relative branches. Sensitivity analysis must be employed to test important parameters and substantiate the findings.
 Interpreting CEA ResultsJump to page 4.
 Glossary — CEAJump to page Glossary.
See the previous page in the tutorialPrev   NextSee the next page in the tutorial ContentsOpen this tutorial's Contents page in a new window   GlossaryOpen this tutorial's Glossary page in a new window   PrefaceOpen the Series Preface page in a new window   Print AllIn a new window, see a composite of all the pages in this tutorial. It will have all sections expanded - ready to print.   HelpOpen the Help page in a new window. 
Find out how to run the tutorial
 - what the commands on this line do,
 - how to print,
 - and more.

Centers for Disease Control and Prevention
U.S. Department of Health & Human Services
Hosted by
Office of Workforce and Career Development
Acknowledgements
Produced by
Prevention Effectiveness Branch
Division of Prevention Research and Analytic Methods
Epidemiology Program Office
Funded by
Office of Terrorism Preparedness and Emergency Response
Developed by
Norbert Denil, OWCD (Webmaster)
Kwame Owusu-Edusei, NIOSH (Content)
Kakoli Roy, OWCD (Project Supervision)
Amanda Schofield (Content)
Ara Zohrabian, OWCD (Content)
Based on earlier, paper-based Framing &
Cost Analysis self-study guides by
Phaedra Corso, NCIPC
Odile Ferroussier, NCHSTP
Amanda Schofield
Additional acknowledgements
Vilma Carande-Kulis, OCSO
Sajal Chattopadhyay, OSI
Martin Meltzer, NCID
Contacts
Norbert Denil (Site design and production) 321-633-6150 ngd1@cdc.gov
Ara Zohrabyan (Technical content) 404-498-6322 aqz0@cdc.gov

Top
 
Top
 
Top