Modeling in Real Time During the Ebola Response
Supplements / July 8, 2016 / 65(3);85–89
Martin I. Meltzer, PhD1; Scott Santibanez, MD1; Leah S. Fischer, PhD1; Toby L. Merlin, MD1; Bishwa B. Adhikari, PhD1; Charisma Y. Atkins, MPH1; Caresse Campbell, PhD2; Isaac Chun-Hai Fung, PhD1; Manoj Gambhir, PhD1; Thomas Gift, PhD3; Bradford Greening, PhD1; Weidong Gu, MD, PhD4; Evin U. Jacobson, PhD5; Emily B. Kahn, PhD1; Cristina Carias, PhD1; Lina Nerlander, MD5; Gabriel Rainisch, MPH1; Manjunath Shankar, MD, PhD1; Karen Wong, MD4; Michael L. Washington, PhD1 (View author affiliations)View suggested citation
- pdf icon [PDF]
To aid decision-making during CDC’s response to the 2014–2016 Ebola virus disease (Ebola) epidemic in West Africa, CDC activated a Modeling Task Force to generate estimates on various topics related to the response in West Africa and the risk for importation of cases into the United States. Analysis of eight Ebola response modeling projects conducted during August 2014–July 2015 provided insight into the types of questions addressed by modeling, the impact of the estimates generated, and the difficulties encountered during the modeling. This time frame was selected to cover the three phases of the West African epidemic curve. Questions posed to the Modeling Task Force changed as the epidemic progressed. Initially, the task force was asked to estimate the number of cases that might occur if no interventions were implemented compared with cases that might occur if interventions were implemented; however, at the peak of the epidemic, the focus shifted to estimating resource needs for Ebola treatment units. Then, as the epidemic decelerated, requests for modeling changed to generating estimates of the potential number of sexually transmitted Ebola cases. Modeling to provide information for decision-making during the CDC Ebola response involved limited data, a short turnaround time, and difficulty communicating the modeling process, including assumptions and interpretation of results. Despite these challenges, modeling yielded estimates and projections that public health officials used to make key decisions regarding response strategy and resources required. The impact of modeling during the Ebola response demonstrates the usefulness of modeling in future responses, particularly in the early stages and when data are scarce. Future modeling can be enhanced by planning ahead for data needs and data sharing, and by open communication among modelers, scientists, and others to ensure that modeling and its limitations are more clearly understood.
The activities summarized in this report would not have been possible without collaboration with many U.S. and international partners (http://www.cdc.gov/vhf/ebola/outbreaks/2014-west-africa/partners.html).
During CDC’s response to the 2014–2016 Ebola virus disease (Ebola) epidemic in West Africa, U.S. and international public health decision-makers and stakeholders needed information early in the epidemic. This information included the number of Ebola cases that could be expected over time; the resources and personnel needed to respond adequately; and the impact of interventions, such as Ebola treatment units (ETUs), community care centers (CCCs), and safe burials. On August 4, 2014, CDC activated a Modeling* Task Force (CDC Modeling) and incorporated the task force into its incident management structure.
Models were used at the outset of the Ebola response in early September 2014 to estimate the impact of the epidemic with and without intervention. These models indicated not only that public health agencies had the means to stop the epidemic by using existing tools and strategies but also that the international community needed to act quickly with sufficient resources to stop the spread of the epidemic. The simple models (1–3) used by CDC Modeling enabled decisions to be made quickly during the response. This report summarizes 1) CDC Modeling’s role, accomplishments, and impact; 2) key issues, challenges, and lessons learned; and 3) suggestions for modeling in future responses.
Personnel conducting this assessment comprised both CDC Modeling staff and other CDC staff. All documents produced by CDC Modeling during August 4, 2014–July 13, 2015, were assessed: five publications, approximately 40 internal memoranda that included multiple versions documenting the modeling process, 1,000 technical consulting e-mails, and 30 presentations, as well as numerous meeting notes. In-depth after-action discussions with CDC Modeling staff aided understanding of how models were developed and used in the response, the difficulties encountered, and the impact of the models on decision making. Other assessments were the amount and type of data used (data requirements), how readily data were available, and the time available before response leadership needed preliminary results (turnaround time). Within the context of CDC’s Ebola response, “impact” referred to the use of models to provide information for the response and make decisions. Models associated with major projects that resulted in a publication, written report, or internal memorandum were categorized into 1 of 3 phases of the West African epidemic curve: 1) start and incidence acceleration, 2) peak and incidence deceleration, and 3) final phase and extinguishing (Figure). Eight reviewed projects resulted in either a publication or an internal, predecisional memorandum.
CDC Role, Accomplishments, and Impact
Start and Acceleration
Initial modeling questions concerned resource needs and thus predicted the number of Ebola cases that could be expected over time with and without isolation, treatment, and safe burials. These estimates enabled CDC Modeling to evaluate the impact that interventions, such as ETUs, CCCs, and safe burials, could have on the epidemic. On the basis of these questions and the knowledge available at the time about virus characteristics and transmission, CDC Modeling developed a simple Microsoft Excel spreadsheet–based model called EbolaResponse (4). This model included input values that could be easily changed (e.g., the number of Ebola patients placed in ETUs) and thus, could estimate potential outcomes if the disease remained unchecked and assess the relative impact of interventions. Because data from the field were limited, CDC Modeling calculated a correction factor for underreporting (4). In late August 2014, the correction for underreporting was estimated to be approximately 2.5; in other words, the true case count was 2.5 times greater than the reported case count.
By using data available through late August 2014, CDC Modeling estimated that 550,000 total reported Ebola cases could occur in Liberia and Sierra Leone (1.4 million when corrected for underreporting) by January 20, 2015, if no additional interventions or behavior changes occurred and if current parameters continued without change (4). Conversely, CDC Modeling predicted that transmission would decline substantially by mid-January 2015 if approximately 70% of Ebola patients were placed into ETUs or CCCs (or an equivalent) and if safe burials were conducted when needed. If this 70% goal could be reached, it would “bend the curve,” causing transmission to drop off substantially. If a large-scale response was delayed, the projected number of cases at the epidemic’s peak most likely would more than double and thus require more resources to control (4).
Initial estimates from EbolaResponse were published in September 2014 (4), with scenarios predicting that each month of delayed response would cause approximately 3.1 times more cases. Perhaps the most important message contained in the report was that public health agencies and the international community needed to act quickly with sufficient resources to stop the epidemic. On the basis of this information and other factors, including the United Nations Ebola virus disease outbreak overview of needs and requirements document (5), CDC leadership and U.S. government officials recommended a rapid increase in aid for the Ebola response. International donors provided approximately U.S. $154.6 million to support Ebola response activities in West Africa, including approximately $71 million from the United States (5). By March 2015, 10 countries had supplied approximately U.S. $2.2 billion in aid, with the United States providing approximately $1.05 billion of that amount (6). Later analyses demonstrated how the increase in resources helped to ensure that the actual number of cases was far less than if prompt action had not been taken (7).
In another modeling project, CDC Modeling analyzed the regional spread of Ebola in West Africa. CDC Modeling used geographic information system software and various regression models to identify factors that could be used to calculate the probability of individual areas becoming affected next (8) and helped to provide data to decision-makers about allocating resources for surveillance, especially in the countries surrounding Guinea, Liberia, and Sierra Leone.
CDC also needed data on the staff, equipment, and materials required to operate a typical ETU. CDC Modeling helped conduct a cost analysis to determine the budget needed to start up and run up to 1,000 ETU beds for 6 months, using the cost per bed from an interagency partner and publicly available data. CDC used this information internally to help guide resource allocation decisions.
Another study estimated the impact of ETUs and CCCs on Ebola transmission (7). Results suggested that during September 23–October 31, 2014, hospitalizing approximately 20% of all Ebola patients in ETUs prevented an estimated 2,244 cases. In addition, placing 35% of patients in CCCs or equivalent community settings that prevent transmission through reduced contact with patients, coupled with the use of safe burials, prevented an estimated 4,487 cases. Together, these interventions prevented an estimated 9,097 cases (7). The findings of this analysis provided evidence that interventions were working.
In September 2014, CDC Modeling began producing weekly predecisional memoranda for internal CDC use only, which provided estimates of the number of active cases (e.g., persons with Ebola and in need of a bed, either in an ETU or CCC) based on updated case counts from the field. These weekly updates provided senior leadership with situational awareness about the epidemic as it evolved. As data and case reporting improved through November 2014–March 2015, the need for projections to support decision making declined. However, projections indicated that the situation could change quickly and bolstered the need for public health agencies to avoid becoming complacent (4) (CDC, unpublished data, 2014).
Peak and Deceleration
As the epidemic progressed, public health officials developed plans to increase ETU capacities and developed methods to isolate patients in non-ETU settings to disrupt Ebola transmission (5,6). Senior U.S. leadership authorized personnel, funds, and supplies to help control the Ebola epidemic (5,6). Various philanthropic organizations and the U.S. Public Health Service agreed to operate ETUs and CCCs built by the U.S. military and others, and funded by the U.S. government. The governments of the United Kingdom, France, China, and other countries also helped to build and support treatment units that were run by international nongovernmental organizations and agencies (9). Once increased resources were allocated, questions arose about stocking and staffing the ETUs and preventing Ebola’s spread to patients within ETUs whose illnesses had been misclassified as Ebola. In one analysis, modelers considered ways to prevent Ebola’s spread to febrile persons with malaria whose illness had been misclassified as Ebola. Modelers analyzed the feasibility of treating all contacts of Ebola patients for malaria to prevent the onset of febrile malaria and subsequent admission to ETUs. If implemented, the intervention could avert admissions even for low levels of treatment compliance (10). Only a few clinics used this strategy. Although these strategies were implemented on a small scale, they might be useful on a wider scale for future responses.
CDC Modeling also contributed to the domestic U.S. response when the first imported case of Ebola prompted CDC and health departments to collaborate to improve hospital preparedness. State and local public health planners needed to know where in the United States travelers from West Africa were most likely to arrive, where they might seek treatment, and whether the United States had enough facilities designated to treat patients with Ebola. In a research letter, CDC Modeling and the U.S. Department of Health and Human Services’ Biomedical Advanced Research and Development Authority estimated the rate of new Ebola cases expected in the United States based on three categories of persons arriving from Liberia, Sierra Leone, and Guinea: 1) travelers who were not health care workers, 2) health care workers, and 3) medical evacuees (11). The rate of new infections in the United States was multiplied by treatment length to determine the number of Ebola patients expected to need treatment at any given time while the epidemic in West Africa continued. According to this analysis, the capacity of Ebola treatment centers in the United States (49 hospitals with 71 total beds) was sufficient to care for the model’s highest estimated number of patients with Ebola, with a large reserve capacity if epidemic conditions worsened (11).
Final Phase and Extinguishing
By May 2015, transmission of Ebola in West Africa had diminished substantially, and the response focused on eliminating transmission (12). Ebola virus was cultured from the semen of male Ebola survivors several months after clinical illness. Therefore, sex partners of these Ebola survivors could be infected but unaware of their infection while the illness is incubating. CDC Modeling was asked to project how often a person who acquired Ebola through sexual transmission and in whom the illness is incubating might arrive from West Africa into the United States (13). Using data from May and June 2015, modelers estimated that the projected frequency of a person traveling from West Africa who has acquired Ebola through sexual transmission and whose Ebola is incubating to be one traveler every 2.75 to 8.3 years (CDC, unpublished data, 2015). These estimates were specific for May and June 2015. As long as a resurgence of Ebola does not occur, the risk of importation will decline over time as the number of survivors capable of transmitting Ebola declines.
CDC Modeling produced a predecisional memorandum for internal CDC use that provided estimates of the potential impact of rapid Ebola diagnostic tests, specifically, the ability to rapidly test a patient with fever or other symptoms possibly indicative of Ebola. Although nucleic acid tests are more accurate, they require well-established laboratories and fully trained personnel. Rapid diagnostic tests produce quick results, are simple to perform, and do not require electricity, which is an important consideration in remote areas. Decision-makers needed to know the best possible strategies for using rapid diagnostic tests and how decreasing or low prevalence of Ebola might affect potential strategies for using these tests. The model results suggested that using rapid tests during low-prevalence periods most likely would require a second sequential confirmatory test at the treatment center to decrease the false-positive test results (CDC, unpublished data, 2015). This modeling provided evidence to support the use of rapid Ebola tests in low-prevalence settings as an effective screening tool to rule out Ebola infection (<1% false negatives), enabling patients with Ebola-like symptoms, but with negative rapid test results, to be treated outside of Ebola isolation units. In addition, this modeling predicted the number of false-positive (and true negative) rapid test results that could be expected at various disease prevalence levels in the community (CDC, unpublished data, 2015).
Key Challenges and Lessons Learned
Throughout the response, relevant data were not always available, and available data frequently contained inconsistencies that took time and effort to resolve. In addition, reporting delays made the incidence of Ebola difficult to accurately calculate, a crucial input in the models produced by CDC Modeling. Expert opinion was needed when data were not available (e.g., when data from the field were limited and a correction factor was used to estimate the actual number of cases). Even when adequate data existed, because no data sharing agreements had been developed and executed, questions arose about who owned the data and who could use them for analysis. As a result, some modeling projects were delayed.
The urgency of the Ebola response required a short turnaround time for projects. CDC Modeling typically was given ≤1 week to answer questions. To reduce errors and ensure the reproducibility and accuracy of the estimates in this short time frame, the CDC Modeling had two teams that cross-checked each other’s calculations, and other modelers at CDC reviewed the calculations during the clearance process. Each team had to clearly document its modeling methods so that other modelers could replicate the model. Therefore, for each model, CDC Modeling provided a technical appendix accessible to scientists within and outside of CDC.
Models and findings needed to be shared with the public, technical experts, responders, and other stakeholders. However, communicating about modeling is difficult when persons unfamiliar with modeling have difficulty interpreting what the estimates mean and understanding the nature of assumptions, uncertainty, and context. For example, if one assumption is disputed, nonmodelers might perceive this dispute as a reason to dismiss the entire model, rather than understand that defining assumptions and improving inputs is part of model development. Publishing manuscripts in scientific journals was, by itself, insufficient for communication. The immediate and primary purpose of the models was to provide information for decision-making, whereas publishing articles about the work provides information for future emergency responses.
Modeling in Future Responses
The benefits of incorporating modeling into major emergency responses were clear in the 2014–2016 Ebola epidemic response (14). Models provided critical decision-making tools in real time and helped demonstrate to public health authorities that the epidemic could be stopped by using existing tools and strategies (4). Although initial model estimates of Ebola represented a worst-case scenario, the international community responded to ensure that these dire predictions would not be realized (14). The following comments were made regarding the accuracy of CDC models that forecasted the trajectory of the epidemic: “the model predicted that when the tipping point was reached, transmission would decline rapidly. This prediction was shown to be accurate in the following months in Liberia and Sierra Leone. . . . The predictions also closely matched the actual case trajectory after effective intervention” (14).
In future emergency responses, modeling can be improved in several ways. First, flexibility is needed to enable data collection to focus on data needs relative to the size of the epidemic and to collect the types of data modelers can use to produce improved, more accurate models (e.g., number of cases that can be expected over time based on available knowledge about pathogen characteristics and transmission, as well as the impact that interventions could have on the epidemic). For example, in a large-scale epidemic, data collection might focus on small amounts of very specific data from sentinel surveillance (1). Second, data-sharing agreements should be in place before an event. Finally, promoting ongoing dialogue will ensure that scientists and audiences understand data limitations and what can and cannot be reliably concluded from models.
Modeling is an important but underused public health tool. Prioritizing modeling in future responses will take hard work, commitment, education, and an openness in public health to new disciplines and approaches.
Corresponding author: Martin I. Meltzer, Division of Preparedness and Emerging Infections, National Center for Emerging and Zoonotic Infectious Diseases, CDC. Telephone: 404-639-7778; E-mail: MMeltzer@cdc.gov.
1Division of Preparedness and Emerging Infections, National Center for Emerging and Zoonotic Infectious Diseases, CDC; 2Division of High-Consequence Pathogens and Pathology, National Center for Emerging and Zoonotic Infectious Diseases, CDC; 3Division of Sexually Transmitted Disease Prevention, National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention, CDC; 4Division of Foodborne, Waterborne, and Environmental Diseases, National Center for Emerging and Zoonotic Infectious Diseases, CDC; 5Division of HIV/AIDS Prevention-Surveillance and Epidemiology, National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention, CDC
- Shrestha SS, Swerdlow DL, Borse RH, et al. Estimating the burden of 2009 pandemic influenza A (H1N1) in the United States (April 2009–April 2010). Clin Infect Dis 2011;52(Suppl 1):S75–82. CrossRefexternal icon PubMedexternal icon
- Reed C, Angulo FJ, Swerdlow DL, et al. Estimates of the prevalence of pandemic (H1N1) 2009, United States, April–July 2009. Emerg Infect Dis 2009;15:2004–7. CrossRefexternal icon PubMedexternal icon
- Meltzer MI, Cox NJ, Fukuda K. The economic impact of pandemic influenza in the United States: priorities for intervention. Emerg Infect Dis 1999;5:659–71. PubMedexternal icon
- Meltzer MI, Atkins CY, Santibanez S, et al. Estimating the future number of cases in the Ebola epidemic—Liberia and Sierra Leone, 2014–2015. MMWR Suppl 2014;63. PubMedexternal icon
- US Agency for International Development. West Africa—Ebola outbreak. Fact sheet #6. Washington, DC: US Agency for International Development; 2014. https://www.usaid.gov/sites/default/files/documents/1864/09.17.14%20-%20USG%20West%20Africa%20Ebola%20Outbreak%20Fact%20Sheet%20%236.pdfpdf iconexternal icon
- US Agency for International Development. West Africa—Ebola outbreak. Fact sheet #23. Washington, DC: US Agency for International Development; 2015. https://www.usaid.gov/sites/default/files/documents/1864/03.04.15%20-%20USG%20West%20Africa%20Ebola%20Outbreak%20Fact%20Sheet%20%2323.pdfpdf iconexternal icon
- Washington ML, Meltzer ML. Effectiveness of Ebola treatment units and community care centers—Liberia, September 23–October 31, 2014. MMWR Morb Mortal Wkly Rep 2015;64:67–9. PubMedexternal icon
- Rainisch G, Shankar M, Wellman M, Merlin T, Meltzer MI. Regional spread of Ebola virus, West Africa, 2014. Emerg Infect Dis 2015;21:444–7. CrossRefexternal icon PubMedexternal icon
- Fink S, Belluck P. One year later, Ebola outbreak offers lessons for next epidemic. The New York Times, March 22, 2015. http://www.nytimes.com/2015/03/23/world/one-year-later-ebola-outbreak-offers-lessons-for-next-epidemic.html?_r=0external icon
- Carias C, Greening B , Campbell CG, Meltzer MI, Hamel MJ. Preventive malaria treatment for contacts of patients with Ebola virus disease in the context of the West Africa 2014–15 Ebola virus disease response: an economic analysis. Lancet Infect Dis 2016;16:449–58. CrossRefexternal icon PubMedexternal icon
- Rainisch G, Asher J, George D, et al. Estimating Ebola treatment needs, United States. Emerg Infect Dis 2015;21:1273–5. CrossRefexternal icon PubMedexternal icon
- CDC. Ebola: getting to zero. Atlanta, GA: CDC. http://www.cdc.gov/features/ebola-zero
- Christie A, Davies-Wayne GJ, Cordier-Lassalle T, et al. Possible sexual transmission of Ebola virus—Liberia, 2015. MMWR Morb Mortal Wkly Rep 2015;64:479–81. Erratum in: MMWR Morb Mortal Wkly Rep 2015;64:1180. PubMedexternal icon
- Frieden TR, Damon IK. Ebola in West Africa—CDC’s role in epidemic detection, control, and prevention. Emerg Infect Dis 2015;21:1897–905. CrossRefexternal icon PubMedexternal icon
* For practical purposes, modeling is divided into two broad categories: statistical modeling and mathematical modeling. Statistical modeling is used when analysts have all, or almost all, the data from a given population, in an identified time step and locale, needed to analyze potential differences over time, between subgroups, or both. For this modeling, analysts use accepted means of statistically testing hypotheses, such as the t test or regression statistical models. Usual results include causality between variables. Mathematical modeling is used when not all the data are available to answer a given question. Analysts then either use data collected from different populations, often at different points in time and locales, or make assumptions based on expert opinion. Using these “islands of data,” analysts then construct a series of equations or simulations that describes the situation (e.g., disease transmission, logistics, or interventions). Usual results include possible decisions that could mitigate or improve a situation.
FIGURE. Timeline* of CDC Modeling Task Force projects for decision making in response to the Ebola epidemic in West Africa — August 2014–July 2015
Abbreviations: Ebola = Ebola virus disease; ETUs = Ebola treatment units.
*Arrows indicate approximate start and completion dates of projects.
Suggested citation for this article: Meltzer MI, Santibanez S, Fischer LS, et al. Modeling in Real Time During the Ebola Response. MMWR Suppl 2016;65(Suppl-3):85–89. DOI: http://dx.doi.org/10.15585/mmwr.su6503a12external icon.
MMWR and Morbidity and Mortality Weekly Report are service marks of the U.S. Department of Health and Human Services.
Use of trade names and commercial sources is for identification only and does not imply endorsement by the U.S. Department of Health and Human Services.
References to non-CDC sites on the Internet are provided as a service to MMWR readers and do not constitute or imply endorsement of these organizations or their programs by CDC or the U.S. Department of Health and Human Services. CDC is not responsible for the content of pages found at these sites. URL addresses listed in MMWR were current as of the date of publication.
All HTML versions of MMWR articles are generated from final proofs through an automated process. This conversion might result in character translation or format errors in the HTML version. Users are referred to the electronic PDF version (https://www.cdc.gov/mmwr) and/or the original MMWR paper copy for printable versions of official text, figures, and tables.
Questions or messages regarding errors in formatting should be addressed to firstname.lastname@example.org.