Behind the Model: CDC's Tools to Assess Epidemic Growth

Let's imagine a world in which we can observe every disease transmission event exactly when it occurs. In this hypothetical world, we count the number of new infections that occurred on day t and divide by the number of infected persons who caused them, to give us R_t: the average number of new infections that each previously infected person caused (Fig. 2).

View Larger

Fig. 2: Rt is a data-driven quantity. If we could directly observe every transmission chain, we could identify all the individuals infected on a particular date (this is referred to as the "infectee generation", with four newly infected persons), and divide by the number of people who caused those infections (or the "parents" of those cases, making up the "infector generation").

Show More

In reality, it's almost impossible to know exactly when transmission occurred or who infected whom in epidemiological data. While sometimes epidemiologists run focused studies designed to observe transmission events and transmission chains, these studies require intensive monitoring of a small group of participants and are the exception, not the rule. To get around these challenges, we estimate R_t using data that are relatively easy to obtain: daily counts of the number of new cases, hospitalizations, or deaths. We input these data into a mathematical model designed to deal with three main challenges of data observation:

1. We almost never know who infected whom
2. There is a lag between the moment someone is infected (an unobservable event) and the date their infection could become observable and/or reported, e.g., as a symptomatic case, a positive test, or hospitalization
3. Not all infections will be observed. For example, not all cases will have symptoms and not all cases who have symptoms will be tested or hospitalized

Below, we'll walk through the basic logic, assumptions, and weaknesses of this model. The technical details of our approach are described (here).

To estimate R_t we need to divide the total number of newly infected people on day t by the number of people who caused those infections (Fig. 2). But how can we do this if the data only contain counts of the total numbers of infections observed each day?

In count data, we can directly read the numerator of the R_t ratio, the number of newly infected people on day t, from the data. The denominator is more difficult to assess. Instead of trying to infer exactly who infected whom, we make assumptions grounded in infectious disease biology. For COVID-19, for example, we know that individuals infected yesterday are just becoming infectious as their viral loads increase. Meanwhile, individuals infected weeks ago have likely recovered and are no longer infectious.^* We can assume that individuals who were infected some intermediate number of days in the past are now causing the bulk of new infections (Fig. 3).

View Larger

Fig 3: Assumptions of infectivity over time, for COVID-19, by time of infection. We know that those infected yesterday, at time t-1, are likely just becoming infectious, while those infected 28 days ago have likely since recovered and are no longer infectious. We know that individuals infected some intermediate number of days ago are likely responsible for new cases we are observing now.

Show More

To estimate R_t, we must develop a model that turns the assumption "individuals infected some days in the past are the ones causing transmission now" into an equation. Our equation is a more complex version of the R_t ratio in Fig. 1, inferred using observable variables. To count the number of individuals in the infector generation on day t, we need to sum across all the individuals who became infected in the recent past – starting yesterday and going back weeks ago – weighted by their current infectiousness. For COVID-19, we assume that individuals infected between 1 and 7 days ago are most infectious,* but individuals infected earlier or later may still cause infections. For details of our R_t equation, see Fig 4.

_{*Day 1-7 covers the central 95% of the generation interval distribution for Omicron from Park et al., 2023; and day 1-5 covers the central 80% of the generation interval distribution. See Figure 5 for a definition of the generation interval.}

View Larger

Fig. 4: Calculating Rt from daily counts. The numerator is the mean number of infections observed on day t, or It. The denominator is the mean number of individuals in the infector generation that gave rise to It, which is based on the number of infections at earlier time points (on day t-1, t-2, t-3, and earlier) weighted by the infectiousness of those individuals based on when they became infectious. The weighting function is based on the generation interval, or the interval (duration) between the time of infection of the infectee and the time of infection of its infector.

Show More

To formally estimate how long the expected wait between infections in a chain of transmission is – and to establish the infectiousness weighting function in Fig. 4 above – infectious disease models use a distribution called the generation interval (G), defined as the interval between the infection times of an infector-infectee pair (Fig 5). For example, if person i was infected on Monday, and if person i infects person j on Friday, then the G_ij is four days. We know that the generation interval varies between transmission pairs, and so we want the distribution of times between infector-infectee pairs. We can estimate the generation interval distribution using data from household or contact tracing studies, in which the approximate timing of infections is observed, or by using the serial interval (the time between onset of symptoms of an infector-infectee pair) as a proxy.

View Larger

Fig 5: Illustrating the generation interval (Gij) between a person in the infector generation and a person in the infectee generation.

Show More

When we estimate R_t, we want to know how many new infections occur on day t. In the real world, we observe events that occur days to weeks afterwards like cases, hospitalizations, or deaths (Fig. 6). These delays are unavoidable and fall into two main categories:

1. Biological delays between the moment a person is first infected and the moment their infection could become observable and/or reportable as a confirmed case, hospitalization, or death.
2. Reporting delays. Let's focus on hospitalization as the observable event because at CDC we currently estimate R_t for COVID-19 and influenza using counts of daily hospital admissions. Individuals transit through many stages of illness from infection to symptomatic illness to testing positive (by either a home test or a PCR test in a health care setting), and some then become hospitalized (Fig. 6). Currently, hospitalization data are one of the most complete and reliable data sources for COVID-19 and influenza. We do not currently have timely, reliable or complete counts of individuals with symptomatic illnesses or positive tests.

View Larger

Fig. 6: An individual with COVID-19 transiting through stages of disease, some of which are unobservable and/or unreportable. Due to current reporting requirements in the United States, nearly 100% of individuals with laboratory-confirmed COVID-19 who are admitted to the hospital are reported to CDC. Additionally, the fraction of infections that result in a laboratory-confirmed hospital admission is fairly stable over time, apart from when there is a shift in disease severity – such as one caused by a new variant – or rapid change in testing availability or practices. By contrast, the fraction of infections that we observe as positive tests and symptomatic persons is likely highly variable over time.

Show More

Caveats and complications:

1. On the most recent dates, we have not yet observed all infections that have occurred, as some infected people have not yet developed symptoms or been hospitalized. This is a challenge because people are usually most interested in recent trends, but recent data are incomplete.
2. There are day-of-week effects in healthcare visits and reporting, where the data consistently show more reports on weekdays vs. weekends.
3. Hospitalizations are not always reported on the day that they occur. For example, sometimes test results take a few days to come back from the lab, diagnoses undergo review, or there are delays in transferring data.

To adjust for incomplete reporting on recent dates, CDC is implementing "nowcasting" approaches. Essentially, we can look back at past reporting patterns to estimate the fraction of total reports that were observed 1, 2,..., n days after the reported event. Then we can scale up accordingly to estimate the number of events that will eventually be reported on each day.

In our data, we only observe the fraction of infected individuals that were admitted to the hospital and tested positive (Fig. 7). Because facilities are currently required to report individuals with laboratory-confirmed COVID-19 and influenza who are admitted to the hospital, we think that the fraction of infections we observe as laboratory-confirmed hospitalizations is quite stable.

View Larger

Fig. 7: The observed fraction of infections among hospitalization data

Mathematically, we expect our R_t estimates to be unbiased as long as the fraction of observed infections in hospitalization data is not changing rapidly. That is because the observed fraction impacts both the numerator – the infectee generation – and denominator – the infector generation – of our R_t equation equivalently (Fig. 8). In reality, there is probably no epidemic dataset where there is no change at all in the fraction of observed infections over time. We have chosen to focus on hospitalizations because we think that using hospitalizations, we observe one of the most stable fractions of infections of any available data source (Sherratt et al., Richardson et al.) (Fig 6). There are some situations where the fraction of observed infections could change quickly enough to temporarily cause biases in R_t estimates, such as the emergence of a more severe variant, lack of diagnostic tests, or a clinical or testing practice change within a healthcare setting. If such a situation occurs, we will flag the estimates we publish noting that there is some possibility of inaccuracy during the brief period before the fraction observed re-stabilizes.

View Larger

Fig. 8: The fraction of infections that we observe in hospitalization data equivalently impacts both the numerator (the fraction of infectees we observe) and the denominator (the fraction of infectors we observe) of the Rt equation and thus has no effect on the Rt estimate.

Show More

It is important to note that hospitalized individuals are systematically different from the general population; for example, they tend to be older for COVID-19 and influenza-related hospitalizations. Furthermore, severe infection and hospitalization fundamentally change a person's behavior in ways that probably affect their opportunities for onward transmission. However, these differences in age, behavior, and contacts don't directly affect our estimates, because we are not measuring the number of new infections that hospitalized individuals go on to cause. Instead, our estimates reflect the population average level of transmission that caused those individuals to become hospitalized themselves.

In fact, though counterintuitive, mathematically, in an epidemic system without rapid changes in severity, infectiousness, or precautionary behavior, different age groups should experience roughly similar epidemic growth rates over time after an initial mixing period. While early in the COVID-19 pandemic these conditions were probably not met, at this point, we believe these effects are minor. Although the total number of infections in each group will be different, the relative change should be the same. This means that estimates of R_t based on incident events from a subgroup (individuals who are hospitalized, for example) of a population are unbiased as long as the fraction of observed infections in hospitalization data stays roughly constant.

In some epidemic modeling analyses, we get to check our answers. For example, if we generate a short-term epidemic forecast, we can wait a few weeks, and then check our predictions against what really happened. But we're never able to observe R_t directly, and so we don't have a gold standard source of truth to check our models against. As a result, we use a few different methods to check that our estimates are reliable.

1. We run simulation studies. We run an epidemic simulation using a dynamic mathematical model with four compartments: susceptible (S), exposed (E), infected (I), and recovered (R) (Fig. 9.1) where we can calculate the 'true' R_t value at all times. The simulation produces an epidemic time series with counts of the number of new infections per day (Fig. 9.2), and we add lags to these data to make them more similar to the case, hospitalization, or death data that we observe in the real world (Fig. 9.3). We can run these simulated data through our R_t estimation models just like real data, only in this case we know exactly what the answer (R_t) should be, as we specified it when simulating the data. We then compare results to the correct answers (Fig. 9.4). If our models do not accurately estimate R_t, we know we need to make changes, until the model accurately estimates R_t.

View Larger

Fig. 9: Mechanism for validating Rt estimates with simulation results. 1) We run a simulation model using an SEIR compartmental model with specified transmission parameters, enabling us to calculate the 'true' Rt at all times in the simulation, with estimated values following a weekly random walk (i.e., changing every week). 2) The SEIR model outputs a case series, to which we 3) add data lags to better approximate real-world data. 4) We run this simulated, lagged data through our Rt estimation model and compare the 'true' value of Rt at all time points to the estimated value. When we see the estimated values of Rt are centered around the true value it validates that our models are working appropriately.

Show More

1. We run a few different R_t estimation models side by side, and we check their answers against each other. We don't expect different models to give the same answer, but we do expect the answers to be similar. If they're not, we investigate, and make changes if needed.
2. We perform common sense checks. If the data show that the epidemic is growing rapidly, then we should see R_t estimates, including confidence intervals, above one for the corresponding time period, after adjusting for lags.
3. We evaluate short-term forecasts from our models. The R software package we use to do R_t modeling, EpiNow2, has produced forecasts that have been submitted to both the CDC and ECDC COVID-19 forecasting hubs, and we are currently further validating these forecasts as part of this season's FluSight Challenge.