Using Randomized Controlled Trials to Estimate the Effect of Community Interventions for Childhood Asthma

Introduction The Centers for Disease Control and Prevention’s Controlling Childhood Asthma and Reducing Emergencies initiative aims to prevent 500,000 emergency department (ED) visits and hospitalizations within 5 years among children with asthma through implementation of evidence-based interventions and policies. Methods are needed for calculating the anticipated effects of planned asthma programs and the estimated effects of existing asthma programs. We describe and illustrate a method of using results from randomized control trials (RCTs) to estimate changes in rates of adverse asthma events (AAEs) that result from expanding access to asthma interventions. Methods We use counterfactual arguments to justify a formula for the expected number of AAEs prevented by a given intervention. This formula employs a current rate of AAEs, a measure of the increase in access to the intervention, and the rate ratio estimated in an RCT. Results We justified a formula for estimating the effect of expanding access to asthma interventions. For example, if 20% of patients with asthma in a community with 20,540 annual asthma-related ED visits were offered asthma self-management education, ED visits would decrease by an estimated 1,643; and annual hospitalizations would decrease from 2,639 to 617. Conclusion Our method draws on the best available evidence from RCTs to estimate effects on rates of AAEs in the community of interest that result from expanding access to asthma interventions.


Introduction
The Centers for Disease Control and Prevention's Controlling Childhood Asthma and Reducing Emergencies initiative aims to prevent 500,000 emergency department (ED) visits and hospitalizations within 5 years among children with asthma through implementation of evidence-based interventions and policies. Methods are needed for calculating the anticipated effects of planned asthma programs and the estimated effects of existing asthma programs. We describe and illustrate a method of using results from randomized control trials (RCTs) to estimate changes in rates of adverse asthma events (AAEs) that result from expanding access to asthma interventions.

Methods
We use counterfactual arguments to justify a formula for the expected number of AAEs prevented by a given intervention. This formula employs a current rate of AAEs, a measure of the increase in access to the intervention, and the rate ratio estimated in an RCT.

Results
We justified a formula for estimating the effect of expanding access to asthma interventions. For example, if 20% of patients with asthma in a community with 20,540 annual asthma-related ED visits were offered asthma self-management education, ED visits would decrease by an estimated 1,643; and annual hospitalizations would decrease from 2,639 to 617. Introduction More than 4.2 million children in the US had current asthma in 2020; they had more than 0.79 million emergency department (ED) visits and 64,525 hospitalizations with a primary asthma diagnosis in 2019 (1,2). The health impact of pediatric asthma is accompanied by high costs (3). The Centers for Disease Control and Prevention's (CDC's) National Asthma Control Program developed and published the EXHALE Technical Package in 2018 (4). This resource comprises 6 evidence-based strategies that can be used to help people achieve better asthma control and decrease the number of asthma-related ED visits and hospitalizations (4). These 6 strategies are 1) education on asthma self-management ("E," hereinafter referred to as E1), 2) extinguishing smoking and exposure to secondhand smoke ("X"), 3) home visits for trigger reduction and asthma self-management education ("H"), 4) achievement of guidelines-based medical management ("A"), 5) linkages and coordination of care across settings ("L"), and 6) environmental practices to reduce asthma triggers from indoor, outdoor, or occupational settings ("E," hereinafter referred to as E2) (4). Implementing and expanding access to these strategies, in part by leveraging partnerships with public and private partners, is a current priority for National Asthma Control Program (4). In 2019, the Controlling Childhood Asthma and Reducing Emergencies initiative was launched with the goal of preventing 500,000 ED visits and hospitalizations nationwide within 5 years through implementation and expansion of access to EXHALE strategies in the population of children with asthma (5).
Evidence for effectiveness of EXHALE strategies differs from strategy to strategy in the type, number, and quality of relevant published studies (4). Four of these strategies -education on asthma self-management, extinguishing smoking, home visits for trigger reduction and asthma self-management education, and achieving guidelines-based medical management (6) -are implemented at an individual level (7). In contrast, the remaining 2 strategies -environmental policies or best practices to reduce asthma triggers from indoor, outdoor, and occupational sources and linkages and coordination of care across settings -address policies or regulations. Each strategy promotes various asthma interventions, which can be tailored to various groups of people with asthma (for example, by demographic and socioeconomic status, geographic location, place of intervention, and asthma severity) (4). For example, an asthma self-management education intervention could involve instructions on using medications correctly, recognizing and managing symptoms, and reducing exposure to asthma triggers and could take place in a school, clinic, home, or other setting. A full description of EXHALE strategies is available at www.cdc.gov/asthma/exhale/index.htm.
The primary objective of this study was to describe and illustrate a method of using results of randomized controlled trials (RCTs) to estimate changes in rates of adverse asthma events (AAEs) that result from expanding access to an EXHALE strategy. The method can also be applied to other study designs, such as cohort and case-control studies, if they generate valid effect estimates. It can be used to calculate anticipated effects of future Controlling Childhood Asthma and Reducing Emergencies interventions or estimate impacts of previously implemented programs. Expanding asthma interventions is a major public health approach to decreasing asthma-related illness and death, so it is important to estimate the effect of an expansion in access to these interventions on rates of AAEs.

Methods
We illustrate a method for using the results of RCTs to estimate the effect of asthma-related interventions on reducing AAEs among children. We used counterfactual reasoning to justify a formula for estimating the expected number of AAEs prevented by implementing selected EXHALE strategies. Asthma interventions are commonly offered to only a part of the population with asthma. First, we show how to predict or estimate effects in this situation. Second, we use data extracted from relevant RCTs identified in the literature to illustrate an application of the formula. Finally, we discuss the approach, results, and considerations for when the method should apply.

Selection of RCTs
We reviewed the literature to identify RCTs for each EXHALE strategy. We restricted the pool of eligible RCTs to those that were 1) included in any of 3 systematic reviews (8)(9)(10) or the EX-HALE technical package (4), 2) recently published (in 2000 or later), 3) included children with asthma, and 4) were conducted in North America (11). We excluded studies for a given outcome if they reported pre-intervention rates of AAEs that differed between the treatment and control arms by more than 30%, indicating potentially imbalanced randomization (12)(13)(14)(15).

Effects in the RCTs
In the RCTs considered, an EXHALE strategy was offered to all participants in the treatment arm but not to participants in the control arm, who generally continued to receive care as usual. Some participants in the treatment arm may not have complied with, taken advantage of, or used the intervention. All participants were followed during the RCTs to determine rates of AAEs. We denoted the observed rates of AAEs in the treatment arm as Rate T and in the control arm as Rate C and the rate ratio (RR) by (Equation 1) where subscript E refers to EXHALE strategy E1 (education on asthma self-management), which is made available by the intervention. In a well-conducted RCT, we expect to be a consistent estimator of the average causal effect of making E available to the study population (assumption 1, Table 1).

Effect in a community where a subpopulation is selected for an intervention
We considered the effects of a particular intervention offered to only a part of a population of interest (for example, all children with asthma in a county). We assumed that the study population was not selected from the community and that only proportion (Pr) was enrolled to receive the intervention (enrollees). We denoted the rates of AAEs among enrollees as Rate P and the rate in the community among those not enrolled (nonparticipants) as . We assumed that the asthma intervention offered to the enrollees was similar to the intervention evaluated in the RCT (assumption 2, Table 1). The intervention is made available to all enrollees, but not everyone will necessarily use or take advantage of its availability. This situation parallels the situation in an RCT, in which some participants in the intervention arm may not take advantage of or use the intervention offered (ie, less than complete compliance). Furthermore, before or during a community program, the intervention may have already been available as part of usual care to nonparticipants in the community. Again, this situation parallels the situation of imperfect compliance in an RCT wherein some participants in the control group receive the intervention as part of usual care outside the study.
The key assumption is that the effect on community enrollees approximates the situation in the RCT with effects measured on the appropriate scale (assumption 3, Table 1 [16]). The approximation need hold only within strata of covariates if stratum-specific estimates are available. Here, we use the rate ratio ( ) to measure effects. We equated the effect of the intervention in the RCT to that among community enrollees: where is the counterfactual rate among community enrollees if they (contrary to fact) had not been enrolled in the intervention. Equation 2 directly parallels a counterfactual formulation of the causal effect in those enrolled to receive the intervention: it is the ratio of the rate if all were enrolled compared with the counterfactual rate if none were enrolled (17). Equation 2 encodes the assumption that the causal effect among community enrollees, measured as a rate ratio, equals the rate ratio in the RCT without further adjustment (16,18). This assumption is related to transportability, a concept that holds if the causal effect of treatment in the study sample applies to the target population that is distinct from that sample.

Effect on adverse events among community enrollees
We derived expressions for the expected number of adverse events prevented, assuming that the program only affected enrollees. We let ΔEvents denote the expected change in the number of adverse events, in a community of size N in which the proportion Pr was enrolled in the program. We estimated the change in the annual number of events by the rate difference (rate per person per year if enrolled minus rate not enrolled) times the number enrolled, PrN: represents the expected number of events in the community, if the program had not been implemented. Which equation, 4a, 4b, or 4c, is best applicable depends on availability of additional information as described next.
If we observed the rate among community enrollees (Rate p ) and have a suitable rate ratio from one or more RCTs, we can apply Equation 4a to estimate the expected change in the number of events.
If the rate among community enrollees (Rate p ) is not available, it may be possible to estimate , the rate among enrollees, if they had not been enrolled ( ) and use an applicable rate ratio from an RCT in Equation 4b to estimate ΔEvents. For example, it may be reasonable to estimate as , the rate observed in the remainder of the community (or a relevant subgroup) where the intervention was not implemented. It could be plausible to assume that the observed satisfies where, denotes the (observed) rate in a subgroup or community that was not offered the intervention (Table 1, assumption 4b). This assumption should hold if the intervention had no effect on rates in parts of the community where it was not offered, and the enrollees were comparable to the rest of the community with respect to rates. In this case, the estimate of and of , if applicable to the community of interest, allows application of Equation 4b. Alternatively, it may be reasonable to use observed rates in another community that is not offered the intervention as an estimate of . Yet another alternative could be to use the observed rate among enrollees in the year before the intervention PREVENTING CHRONIC DISEASE was introduced as an estimate of . However, major disruptions in health and health care, such as a pandemic, can render this option inappropriate.
Alternatively, if the number of events expected in the community had the program not been implemented can be estimated, perhaps based on numbers in a prior year, then this number can be substituted for "events" in Equation 4c.
For completeness, if the rate among enrollees Rate P is measured and a direct estimate of the rate among enrollees had they not enrolled is also available, then the number of adverse events prevented can be estimated by direct substitution into Equation 3. This approach, however, would not incorporate results of the RCTs.

Implementation
Implementation of the approach for estimating the effect of asthma-related interventions on rates of AAEs involved the following steps. First, we reviewed the literature to identify relevant studies that provided a valid effect estimate for one or more interventions similar to the ones of interest (Assumptions 1 and 2, Table 1). We summarized characteristics of the identified studies, including the EXHALE strategy used, the type of intervention implemented, the demographic characteristics of the population studied (the number of participants, age group, percentage that was male, and race and ethnicity), the severity of asthma in the study population, the baseline rates in the hospital or ED or both, the study period, and the location of the study. Second, we evaluated each study's population and compared its characteristics with those of the community to assess the extent to which the effect in the RCT might validly estimate the effect among those in the community who were offered the program (Assumption 3, Table 1, transportability). The most appropriate RCT to use depends on these characteristics, comparability of the study population with the target community, and distributions of effect modifiers. Third, we evaluated available information to identify which combination of information ( , or ) was available in addition to (Assumption 4a, 4b, or 4c; Table 1). Finally, we calculated the number of prevented events by using Equations 4a, 4b, or 4c, where the choice was guided by the third step (evaluation of available information).
We developed 2 examples to illustrate how we implemented our approach. In Example 1, we considered the projected number of ED visits prevented by a community program if the program expanded access to a specific EXHALE strategy: E, A, X, or E + A + H + L. We first considered the number of ED visits prevented by expanding access to strategy E, education on asthma selfmanagement. We separately estimated the effect of a program in each relevant RCT. We estimated by assuming it approximately equaled the observed rate in the community of interest during the year before the planned intervention. We calculated the anticipated effect of expanding this strategy by offering a program to Pr = 20% of a community with 20,540 asthma-related ED visits among children; we assumed this number of ED visits on the basis of publicly available data from Los Angeles County (19). Similarly, we used the same approach to calculate the number of ED visits prevented by programs expanding access to strategies A, X, and E + A + H + L.
In Example 2, we considered the projected number of hospitalizations prevented by a community program if the program expanded access to strategy E, A, or X. We first considered strategy E. To apply Equation 4b, we must choose a study in which the children with asthma studied in the RCT were sufficiently similar to those in the community population for the rate ratios to be applicable. Here, we estimated by assuming that it approximately equaled the observed rate in the community of interest in the year before the planned intervention. To predict the number of hospitalizations prevented, we again presumed that Pr was equal to 0.2 (ie, the program provided access to self-management education for 20% of the community population). On the basis of data from Los Angeles County (19), 2,639 asthma-related hospitalizations occurred in 2015 among children. Next, we applied the rate ratio ( = 0.21) from the study by Bruzzese et al (20) and applied Equation 4c. For other studies, we similarly estimated the number of hospitalizations prevented when the rate ratio from the respective RCT was transportable to the population of interest.

Results
Our literature search identified 15 RCTs meeting inclusion criteria (Table 2). Study populations ranged in size from 56 to 937. Participant characteristics differed from study to study. For example, asthma severity varied across the studies; 1 study included only children with severe asthma, while other studies included children with any asthma diagnosis. We found only 1 RCT for strategy X (29) and no RCTs met our selection criteria for strategy H or for strategy E2.

Example 1: Number of ED visits prevented
Ten RCTs satisfied our inclusion criteria for ED visits among children with asthma (Table 3A). Although each RCT evaluated an EXHALE strategy, the interventions described differed in the number of sessions, setting, population studied and overall content.
On the basis of the effect estimate for strategy E, education on asthma self-management, the intervention described by Brown et

Example 2: Number of hospitalizations prevented
Eleven RCTs satisfied our criteria for hospitalizations among children with asthma (Table 3B). Each RCT had a unique set of features such as study population, demographic characteristics, distribution of asthma severity, setting, and content. On the basis of the effect estimate for strategy E, education on asthma selfmanagement, the intervention described by Bruzzese et al (20), for example, would have prevented 417 hospitalizations annually. For strategy A, the intervention described by Szefler et al (28) would have prevented 293 hospitalizations annually; and for strategy X, the intervention described by Wilson et al (29) would have prevented 107 hospitalizations annually.

Discussion
We illustrated how to use effect estimates in the published literature to predict the effect of expanding access to interventions in the population of children with asthma. We demonstrated a practical method for converting effect estimates into rates of AAEs prevented by expanding EXHALE strategies. We used 15 recent RCTs to illustrate the method, but the method can also be applied to other study designs. These RCTs were conducted in various settings with different study populations, degrees of asthma severity, and intervention types. The treatment and control arms of RCTs are often not random samples of the population of interest. Public health practitioners can use the most appropriate RCT on the basis of considerations of comparability and transportability.
In the studies selected for our analysis, participants were evaluated to determine the effect of the program on rates of AAEs in treatment and control arms. However, the intervention might have already been available to some members of the study population as part of usual care. The comparison of the rates in the treatment and the control arms estimates the effect of the program compared with no program or usual care. Such a lack of compliance in an RCT or the number of dropouts in an observational study parallels the situation in the community where not all enrollees necessarily participated, received, or completed the intervention (30,31).
Two key challenges in using results of RCTs or other studies to estimate events prevented in a community are generalizability and transportability. Researchers distinguish between these 2 related concepts by using the term "generalizability" when study groups are selected from the community of interest and the term "transportability" when the study groups are not selected from the community of interest. In our context, transportability concerns the extent to which rates and effects from the selected studies can be used directly or with certain transport equations (32,33) to estimate effects in the community of interest. Pearl and Bareinboim argued in 2014 that the issues that determine whether and how to transport effect estimates are causal issues that reflect differences between populations and the mechanisms by which effects occur, rather than just differences in statistical distributions (32). In a theoretic approach and building on extensive work in causal inference, they formally represented causal relationships in "selection diagrams" and offered several transport formulas and guidance on when they apply to account for types of differences (32). Several other methods have recently been developed for transporting estimates from RCTs and other studies to a population of interest (33-36). These techniques often require information on certain covariate-specific effects (effect modifiers) and the covariate distribution in the population of interest. One study used in our analysis assessed effect modification and allowed us to illustrate our approach in that context (20).
Emphasizing practical considerations, Spiegelman et al argued that the choice of effect measure (eg, rate ratio or rate difference) for analyzing studies should be driven by model fit (16,18,37). They also cited anecdotal evidence to suggest that a rate ratio or risk ratio model might often fit well (16). They suggested transforming the effect measure to the scale of public health interest and standardizing to the distribution of risk factors (effect modifiers) in the population of interest. They cautioned that the required distributions of risk factors could be unavailable, consistent with the paucity of such information that we found. (18), we used a ratio scale for effect estimates. The validity of our estimates depends on the assumption that the causal effect in the population of interest, when measured on the ratio scale ( ), is reasonably approximated by the effect estimate from an RCT. Considerations for selection of studies that could make the assumption more plausible include the following: evaluation of the same intervention, similarity of the study population and community of interest with respect to baseline levels of asthma intervention, asthma severity, access to care, socioeconomic levels, health insurance, and demographic characteristics. One should also consider the existence and distribution of important effect modifiers. Similarity of baseline rates might also be considered, but it is not necessary if effects are not heterogeneous on the chosen scale (eg, ratio).

Limitations
We found only 1 RCT for strategy X (29) and no RCTs met our selection criteria separately for strategy H or for strategy E2. Furthermore, certain combinations of strategies have been recommended. For example, strategies A and E1 together and A, E1, and H together were recommended as part of a stepwise approach for children according to levels of asthma control (38), but we found only 1 RCT for the strategies A and E1 together, making it difficult to evaluate the effect of this combination (27). In addition, the characteristics of the study populations in the RCTs included in our analysis differed in terms of race and ethnicity, age, asthma severity, and urbanicity. Some populations were not well represented in the selected RCTs. For example, only 1 RCT (14) focused on a rural population and only one (21) evaluated effect modification, which can be important for estimating effects in the community. Although we illustrated our methods by using RCTs, our approach can be used with other study designs that provide valid effect estimates for these strategies, combinations of strategies, effect modifications, and population subgroups.

Conclusion
Expanding access to the evidence-based EXHALE strategies can contribute to reducing the number of asthma-related ED visits and hospitalizations. The method we illustrated draws on the best available evidence from RCTs to estimate effects on rates of AAEs in a community that result from expanding access to an EX-HALE strategy. This method can be used to calculate the anticipated effects of future asthma interventions or to calculate the effects of programs that have been implemented. Public health practitioners can use the method outlined in this article and the current base of RCT literature in their planning and evaluation of interventions toward the overall goal of reducing the number of asthma deaths.

Acknowledgments
The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention. No copyrighted materials were used in this article or research. The authors received no external financial support for the research, authorship, or publication of this article. The authors declared no potential conflicts of interest with respect to the research, authorship, or publication of this article.
Our formula for estimating the effect of asthma-related interventions on reducing adverse events among children uses rate ratios, but some RCTs report risks or risk ratios. A description of how to convert risks to rates is available from the corresponding author upon request.

Author Information
Corresponding Author: CDR Kanta Sircar, PhD, MPH, Asthma and Community Health Branch, Centers for Disease Control and Prevention, 4770 Buford Hwy, MS 106-6, Atlanta, GA 30329 (ksircar@cdc.gov).     Abbreviation: ED, emergency department; ETS, environmental tobacco smoke; NAEPP, National Asthma Education and Prevention Program; RCT, randomized controlled trial. a EXHALE comprises a set of 6 strategies that contribute to asthma control: 1) education on asthma self-management, 2) extinguishing smoking and exposure to secondhand smoke, 3) home visits for trigger reduction and asthma self-management education, 4) achievement of guidelines-based medical management, 5) linkages and coordination of care across settings, and 6) environmental practices to reduce asthma triggers from indoor, outdoor, or occupational settings (4 Abbreviation: ED, emergency department; ETS, environmental tobacco smoke; NAEPP, National Asthma Education and Prevention Program; RCT, randomized controlled trial. a EXHALE comprises a set of 6 strategies that contribute to asthma control: 1) education on asthma self-management, 2) extinguishing smoking and exposure to secondhand smoke, 3) home visits for trigger reduction and asthma self-management education, 4) achievement of guidelines-based medical management, 5) linkages and coordination of care across settings, and 6) environmental practices to reduce asthma triggers from indoor, outdoor, or occupational settings (4). b Estimated incidence rate (per person per year). c Study population included adults and children. d Compared baseline rates with post-study rates for a single intervention arm.  Abbreviations: ED, emergency department; RCT, randomized control trial; RR, rate ratio. a Estimated number from Los Angeles County, California (19). b EXHALE comprises 6 strategies that contribute to asthma control: 1) education on asthma self-management, 2) extinguishing smoking and exposure to secondhand smoke, 3) home visits for trigger reduction and asthma self-management education, 4) achievement of guidelines-based medical management, 5) linkages and coordination of care across settings, and 6) environmental practices to reduce asthma triggers from indoor, outdoor, or occupational settings (4). c RRs were calculated from group-specific (ie, control and intervention) ED rates provided by each RCT. For studies reporting risks, we estimated the group-specific rates before calculating the estimated study RR. d Estimates are for illustrative purposes only. Parameter Pr depends on the scale of the intervention. e Numbers in parentheses were calculated by using the upper and lower bound of the 95% CIs for . f Negative number for prevented ED visits reflects an RR >1 and that the intervention was associated with an increase in ED visits.