Grading of Recommendations, Assessment, Development, and Evaluation (GRADE): Moderna COVID-19 Vaccine

Grading of Recommendations, Assessment, Development, and Evaluation

Overview

A Grading of Recommendations, Assessment, Development and Evaluation (GRADE) review of the evidence for benefits and harms for Moderna COVID-19 vaccine was presented to the Advisory Committee for Immunization Practices (ACIP) on February 4, 2022. GRADE evidence type indicates the certainty of estimates from the available body of evidence. Evidence certainty ranges from type 1 (high certainty) to type 4 (very low certainty) [1].

The policy question was, “Should vaccination with the Moderna COVID-19 vaccine (Spikevax, 2-dose primary series) be recommended for persons 18 years of age and older?” The potential benefits pre-specified by the ACIP COVID-19 Vaccines Work Group included prevention of symptomatic laboratory-confirmed COVID-19 (critical), hospitalization due to COVID-19 (critical), death due to COVID-19 (important), and asymptomatic SARS-CoV-2 infection (important). The two pre-specified harms were serious adverse events (SAEs) (including myocarditis and anaphylaxis) (critical) and reactogenicity (severe, grade ≥3) (important).

A systematic review of evidence on the benefits and harms of a two-dose regimen of Moderna COVID-19 vaccine among persons aged ≥18 years was conducted, based on data available as of December 10, 2021. The evidence from two Phase I open label trials [2, 3], one Phase II randomized controlled trial (RCT) [4], one Phase III RCT [5, 6], 26 vaccine effectiveness studies [7-32], and two vaccine safety surveillance systems [33, 34] were assessed using a modified GRADE approach [1]. Pooled efficacy and effectiveness estimates were calculated when multiple sources had data on an outcome.

In terms of benefits, the available data from RCTs demonstrated that, compared with placebo, vaccination was associated with a lower risk of symptomatic laboratory-confirmed COVID-19 (relative risk [RR] 0.07, 95% confidence interval [CI] 0.05–0.09; evidence type 1), hospitalization due to COVID-19 (RR 0.04; 95% CI 0.01–0.31; evidence type 2), death due to COVID-19 (RR 0.14, 95% CI 0.01–2.79; evidence type 3), and asymptomatic SARS-CoV-2 infection (RR 0.43; 95% CI 0.36–0.50; evidence type 1). The certainty of estimates regarding hospitalization and death due to COVID-19 was reduced due to imprecision.

The pooled vaccine effectiveness estimates from observational studies were consistent with these findings. Compared with no vaccination, vaccination with Moderna COVID-19 vaccine was associated with a decreased risk of symptomatic laboratory-confirmed COVID-19 (RR 0.11, 95% CI 0.06–0.18; evidence type 2), hospitalization (RR 0.05, 95% CI 0.04–0.07; evidence type 2), and death due to COVID-19 (RR 0.06, 95% CI 0.05–0.08; evidence type 2). The certainty of each of these estimates was increased for a strong association. Vaccination was also associated with a decreased risk of asymptomatic SARS-CoV-2 infection (RR 0.30, 95% CI 0.23–0.39; evidence type 4); the evidence certainty type was downgraded for inconsistency.

In terms of harms, the available data from RCTs indicated that serious adverse events were balanced between the vaccine and placebo arms (RR 0.92; 95% CI 0.78–1.08, evidence type 2), and 15 serious adverse events reported for 12 persons were judged to be related to vaccination among more than 15,000 persons vaccinated. The certainty of this estimate was reduced due to imprecision. Reactogenicity grade ≥3 was associated with vaccination (RR 5.03; 95% CI 4.65–5.45, evidence type 1). About 21% of vaccine recipients versus 5% of placebo recipients reported grade ≥3 reactions. Two rare but serious adverse events, anaphylaxis and myocarditis, have been associated with vaccination in post-authorization safety surveillance (see results section and Table 3e).

Introduction

On January 31, 2022, the U.S. Food and Drug Administration (FDA) approved the Biologics License Application (BLA) for Moderna COVID-19 Vaccine (Spikevax) for the prevention of COVID-19 in persons aged ≥18 years [35]. As part of the process employed by the Advisory Committee for Immunization Practices (ACIP), a systematic review and Grading of Recommendations, Assessment, Development and Evaluation (GRADE) assessment of the evidence for Moderna COVID-19 vaccine was conducted and presented to ACIP [1]. There were no conflicts of interest reported by CDC and ACIP COVID-19 Vaccines Work Group members involved in the GRADE analysis.

ACIP adopted a modified GRADE approach in 2010 as the framework for evaluating the scientific evidence that informs recommendations for vaccine use. ACIP has made modifications to the GRADE approach by presenting assessed evidence as type 1, 2, 3, and 4, which corresponds to high, moderate, low, and very low certainty, whereas standard GRADE has high as level 4 and very low as level 1. Additionally, instead of presenting the overall certainty of evidence across all outcomes, ACIP presents the certainty of evidence for the benefits and harms separately. ACIP includes an option “ACIP recommends the intervention for individuals based on shared clinical decision-making” instead of providing a conditional recommendation for or against an intervention. GRADE was used to evaluate the efficacy and safety of a two-dose regimen of Moderna COVID-19 vaccine among persons aged ≥18 years. Evidence of benefits and harms were reviewed based on the modified GRADE approach [1].

The policy question was, “Should vaccination with Moderna COVID-19 vaccine (Spikevax, 2-dose primary series) be recommended for persons 18 years of age and older?” (Table 1).

Methods

We conducted a systematic review of evidence on the benefits and harms of a two-dose regimen of Moderna COVID-19 vaccine (see Appendix 2 for databases and search strategies). We assessed outcomes and evaluated the quality of evidence using the GRADE approach. Patient-important outcomes (including benefits and harms) for assessment were selected by the Work Group during Work Group calls and via online surveys where members were asked to rate and rank the importance of relevant outcomes.

We identified RCTs through clinicaltrials.gov. Relevant Phase I, II, or III RCTs of Moderna COVID-19 vaccine were included if they: 1) involved human subjects; 2) reported primary data; 3) included adults (aged ≥18 years) at risk for SARS-CoV-2 infection; 4) included data relevant to the efficacy and safety outcomes being measured; 5) included data for the dosage being recommended (100 μg, 2 doses at 0 and 28 days). We identified relevant observational studies through an ongoing systematic review conducted by the International Vaccine Access Center (IVAC) and the World Health Organization (WHO) [36]. Relevant observational studies, using case-control, test-negative, or cohort designs, were restricted to the defined population, intervention, comparison, and outcome outlined in the policy question. Outcomes were assessed starting at least 7 days after 2^nd dose. We included only 2-dose Moderna vaccine effectiveness estimates, with combined mRNA vaccine effectiveness estimates excluded. We included studies of general populations and special populations. In addition, efforts were made to obtain unpublished and other relevant data by hand-searching reference lists, and consulting with vaccine manufacturers and subject matter experts. We included observational safety data from two vaccine safety surveillance systems based on input from ACIP’s COVID-19 Vaccines Safety Technical (VaST) Work Group: Vaccine Safety Datalink (VSD) and Vaccine Adverse Event Reporting System (VAERS). Characteristics of all included studies and surveillance systems are shown in Appendix 1 [2-33].

Two reviewers evaluated all studies for study limitations (risk of bias) using the Cochrane Risk of Bias (RoB) tool for RCTs and the Newcastle-Ottawa Scale (NOS) for observational studies. RoB comprises a series of questions structured into domains focusing on different aspects of trial design, conduct, and reporting. Based on question responses, judgement can be “low”, “moderate”, or “high” risk of bias. NOS is a 9-point scale which assesses study limitations related to participant selection and comparability, and assessment of outcome (cohort studies) or ascertainment of exposure (case-control studies). Studies with NOS scores <7 were considered to have serious study limitations.

From the RCT data, RR were calculated from numerators and denominators available in the body of evidence. Vaccine efficacy estimates were defined as 100% x (1-RR). Vaccine effectiveness estimates and 95% CIs were taken from the published/preprint studies, as defined by the authors using a variety of study designs and analytical approaches; adjusted estimates were used when available. When multiple studies were available, pooled estimates were calculated using random effects (>3 studies) or fixed effects (≤3 studies) meta-analysis (R meta package). When multiple studies provided estimates based on overlapping study populations, the study with the most comprehensive population and follow-up time was selected for inclusion in the pooled estimate. Because there was a relatively large body of evidence from vaccine effectiveness studies, with many available only in the preprint literature, an a priori decision was made to exclude studies judged to have serious study limitations from the main pooled estimate used for GRADE. Sensitivity analyses were performed to assess the influence of study characteristics (e.g., special populations vs. full population, preprint vs. peer-reviewed, standard vs. extended dosing interval, cohort vs. case-control/test-negative study design, study limitations, and circulating variants). The evidence certainty assessment for randomized and observational studies addressed risk of bias, inconsistency, indirectness, imprecision, and other characteristics. The GRADE assessment across the body of evidence for each outcome was presented in an evidence profile.

Results

The results of the GRADE assessment were presented to ACIP on February 4, 2022.

Outcomes of interest included individual benefits and harms. Indirect effects of vaccination (e.g., societal benefits) were not considered as part of GRADE. Benefits of interest deemed critical were prevention of symptomatic laboratory-confirmed COVID-19 and prevention of hospitalization due to COVID-19 (Table 2). Other important benefits included prevention of death due to COVID-19 and prevention of asymptomatic SARS-CoV-2 infection. The critical harm of interest was serious adverse events (SAEs), including myocarditis and anaphylaxis; reactogenicity grade ≥3 was deemed an important harm.

After screening 134 publications, 96 were excluded from full-text review because they were a different intervention (e.g., a different vaccine or a different dose, n=91), or a different outcome (e.g., any infection instead of symptomatic COVID-19 or asymptomatic infection, n=5). Of the 38 publications that were deemed eligible for full-text review, four were excluded because they assessed a different intervention, and one was excluded because it assessed a different outcome. The remaining 33 publications, which reported data on 31 studies or surveillance systems, were included in the evidence synthesis and GRADE evidence assessment (Appendix 1) [2-33]. Data were reviewed from five RCT publications, including two publications from the Phase I trial, one publication from a Phase II trial, and two publications from the Phase III trial [2-6]. Data were reviewed from 26 vaccine effectiveness studies. Two vaccine safety surveillance systems, VSD and VAERS, included data for SAEs [33, 34].

In the Phase III RCT, using data on all blinded follow-up (median follow up of 5 months), the Moderna COVID-19 vaccine reduced symptomatic laboratory-confirmed COVID-19 when compared to placebo (vaccine efficacy: 92.7%, 95% CI 90.4–94.4%) (Table 3a). For hospitalization due to COVID-19, 25 events occurred, 24 in the placebo group and one in the vaccine group. Vaccine efficacy against hospitalization due to COVID-19 was 95.9% (95% CI 69.5–99.4%) (Table 3b). Deaths due to COVID-19 were uncommon, zero in the vaccine group and three in the placebo group (vaccine efficacy: 100%) (Table 3c). Numbers of SAEs were comparable between the vaccine group and the placebo group across the two RCTs (Phase III: 268/15,184 (1.8%) vs. 292/15,164 (1.9%); Phase II: 0/200 (0.0%) vs. 0/200 (0.0%)); there were no cases of vaccine-associated enhanced disease or vaccine-related deaths (Table 3e). Grade ≥3 reactions generally were not uncommon and occurred more frequently in the vaccine than placebo groups (Table 3f).

Fourteen vaccine effectiveness studies reported data on symptomatic laboratory-confirmed COVID-19 (Table 3a), 19 reported data on hospitalization due to COVID-19 (Table 3b), five reported data on death due to COVID-19 (Table 3c), and three reported data on asymptomatic SARS-CoV-2 infection (Table 3d). The pooled vaccine effectiveness estimates from the observational studies demonstrated that the Moderna COVID-19 vaccine reduced symptomatic laboratory-confirmed COVID-19 when compared to no vaccination (pooled vaccine effectiveness: 89.2%, 95% CI 82.0–93.6%; based on 11 studies) [7, 10, 12, 13, 16, 19, 20, 22, 24, 28, 31]. The pooled vaccine effectiveness against hospitalization due to COVID-19 was 94.8% (95% CI 93.1–96.1%), based on 14 studies [8-10, 12, 16-20, 25-28, 31]. The pooled vaccine effectiveness for prevention of death due to COVID-19 was 93.8% (95% CI 91.5–95.4%), based on five studies [10, 17, 18, 25, 32]. The pooled vaccine effectiveness against asymptomatic SARS-CoV-2 infection was 69.8% (95% CI 60.9–76.7%), based on three studies [10, 12, 28].

Observational data on serious adverse events were reviewed. A rapid cycle analysis from VSD evaluated chart-reviewed cases of myocarditis and pericarditis occurring among persons aged 18–39 years following dose 2 of the Moderna COVID-19 vaccine (Table 3e) [34]. The rate of myocarditis and pericarditis was 33.0 cases per million doses in the 0–7-day risk interval. Data from VAERS showed an elevated ratio of observed to expected myocarditis cases in the 7-day interval following vaccination among females aged 18–29 years and among males aged 18–49 years, with higher observed/expected ratios in males than females [34]. A rapid cycle analysis of data from VSD evaluated chart-reviewed cases of anaphylaxis among all vaccinated persons aged ≥18 years. Based on events occurring in a 0–1 day risk interval after vaccination, the estimated incidence of confirmed anaphylaxis was 5.1 (95% CI 3.3–7.6) per million doses [33].

GRADE Summary

The initial GRADE evidence level was type 1 (high) for randomized controlled trials and type 3 (low) for the observational data (Table 4). In terms of benefits, the RCT data indicated that the vaccine reduces the risk of symptomatic laboratory-confirmed COVID-19, and no serious concerns impacting certainty were identified for this outcome (type 1, high). Observational data for symptomatic laboratory-confirmed COVID-19 indicated a similar risk reduction with vaccination, and the certainty was upgraded one point for a strong association (type 2, moderate). The certainty of the evidence from one RCT for hospitalization due to COVID-19 was downgraded one point for serious concern of imprecision (type 2, moderate). Observational data for hospitalization due to COVID-19 indicated a similar risk reduction with vaccination, and the certainty was upgraded one point for a strong association (type 2, moderate). The certainty of the evidence for death due to COVID-19 was downgraded two points for very serious concern of imprecision (type 3, low). Observational data for death due to COVID-19 concurred with a strong risk reduction with vaccination, and the certainty was upgraded one point for a strong association (type 2, moderate). RCT data indicated that the vaccine reduces the risk for asymptomatic SARS-CoV-2 infection, and no serious concerns impacting certainty were identified for this outcome (type 1, high). Observational data for asymptomatic SARS-CoV-2 infection indicated a similar risk reduction with vaccination, and the certainty was downgraded one point for serious concern of inconsistency (type 4, very low). The certainty of evidence for serious adverse events was downgraded one point for serious concern of imprecision (type 2, moderate). Observational data on specific serious adverse events (i.e., myocarditis among persons aged 18–39 years and anaphylaxis among persons aged 12 years and older) demonstrated these events were rare (evidence type 3, low). No serious concerns impacted the certainty of estimates of reactogenicity from RCTs (type 1, high).

The summary of evidence types is shown in Table 5. The final evidence types were type 1 for symptomatic laboratory-confirmed COVID-19, type 2 for hospitalization due to COVID-19 and death due to COVID-19, type 1 for asymptomatic SARS-CoV-2 infection, type 2 for serious adverse events, and type 1 for reactogenicity.

References

1. Ahmed, F., et al., Methods for developing evidence-based recommendations by the Advisory Committee on Immunization Practices (ACIP) of the U.S. Centers for Disease Control and Prevention (CDC). Vaccine, 2011. 29(49): p. 9171-6.
2. Anderson, E.J., et al., Safety and Immunogenicity of SARS-CoV-2 mRNA-1273 Vaccine in Older Adults. N Engl J Med, 2020. 383(25): p. 2427-2438.
3. Jackson, L.A., et al., An mRNA Vaccine against SARS-CoV-2 – Preliminary Report. N Engl J Med, 2020. 383(20): p. 1920-1931.
4. Chu, L., et al., A preliminary report of a randomized controlled phase 2 trial of the safety and immunogenicity of mRNA-1273 SARS-CoV-2 vaccine. Vaccine, 2021. 39(20): p. 2791-2799.
5. El Sahly, H.M., et al., Efficacy of the mRNA-1273 SARS-CoV-2 Vaccine at Completion of Blinded Phase. N Engl J Med, 2021. 385(19): p. 1774-1785.
6. Baden, L.R., et al., Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine. N Engl J Med, 2021. 384(5): p. 403-416.
7. Andrews, N.T., E.; Stowe, J.; Gower, C.; Kirsebom, F.; Simmons, R.; Gallagher, E.; Chand, M.; Brown, K.; Ladhani, S.N.; Ramsay, M.; Lopez Bernal, J., Vaccine effectiveness and duration of protection of Comirnaty, Vaxzevria and Spikevax against mild and severe COVID-19  in the UK. preprint, 2021.
8. Bajema, K.L., et al., Effectiveness of COVID-19 mRNA Vaccines Against COVID-19-Associated Hospitalization – Five Veterans Affairs Medical Centers, United States, February 1-August 6, 2021. MMWR Morb Mortal Wkly Rep, 2021. 70(37): p. 1294-1299.
9. Bruxvoort, K.J., et al., Effectiveness of mRNA-1273 against delta, mu, and other emerging variants of SARS-CoV-2: test negative case-control study. BMJ, 2021. 375: p. e068848.
10. Bruxvoort, K.J., et al., Real-world effectiveness of the mRNA-1273 vaccine against COVID-19: Interim results from a prospective observational cohort study. Lancet Reg Health Am, 2021: p. 100134.
11. Carazo, S., et al., Single-dose mRNA vaccine effectiveness against SARS-CoV-2 in healthcare workers extending 16 weeks post-vaccination: a test-negative design from Quebec, Canada. Clin Infect Dis, 2021.
12. Chemaitelly, H., et al., mRNA-1273 COVID-19 vaccine effectiveness against the B.1.1.7 and B.1.351 variants and severe COVID-19 disease in Qatar. Nat Med, 2021. 27(9): p. 1614-1621.
13. Chin, E.T., et al., Effectiveness of the mRNA-1273 Vaccine during a SARS-CoV-2 Delta Outbreak in a Prison. N Engl J Med, 2021. 385(24): p. 2300-2301.
14. Chung, H., et al., Effectiveness of BNT162b2 and mRNA-1273 covid-19 vaccines against symptomatic SARS-CoV-2 infection and severe covid-19 outcomes in Ontario, Canada: test negative design study. BMJ, 2021. 374: p. n1943.
15. Embi, P.J., et al., Effectiveness of 2-Dose Vaccination with mRNA COVID-19 Vaccines Against COVID-19-Associated Hospitalizations Among Immunocompromised Adults – Nine States, January-September 2021. MMWR Morb Mortal Wkly Rep, 2021. 70(44): p. 1553-1559.
16. Grannis, S.J., et al., Interim Estimates of COVID-19 Vaccine Effectiveness Against COVID-19-Associated Emergency Department or Urgent Care Clinic Encounters and Hospitalizations Among Adults During SARS-CoV-2 B.1.617.2 (Delta) Variant Predominance – Nine States, June-August 2021. MMWR Morb Mortal Wkly Rep, 2021. 70(37): p. 1291-1293.
17. Irizarry, R.A.R.-F., M. M.; Nieves, E.G.; Cardona-Gerena, I., Time-Varying Effectiveness of Three COVID-19 Vaccines in Puerto Rico. SSRN, 2021.
18. Lin, D.Y., et al., Effectiveness of Covid-19 Vaccines over a 9-Month Period in North Carolina. N Engl J Med, 2022.
19. Martinez-Baz, I., et al., Product-specific COVID-19 vaccine effectiveness against secondary infection in close contacts, Navarre, Spain, April to August 2021. Euro Surveill, 2021. 26(39).
20. Nasreen, S.C., H.; He, S.; Brown, K. A.; Gubbay, J.B.; Buchan, S.A.; Fell, D.; Austin, P.C.; Schwartz, K.L.; Sundaram, M.E.; Calzavara, A.; Chen, B.; Tadrous, M.; Wilson, K.; Wilson, S.E.; Kwong, J.C., Effectiveness of mRNA and ChAdOx1 COVID-19 vaccines against symptomatic SARS-CoV-2 infection and severe outcomes with variants of concern in Ontario. medRxiv, 2021.
21. Nordström, P., M. Ballin, and A. Nordström, Effectiveness of heterologous ChAdOx1 nCoV-19 and mRNA prime-boost vaccination against symptomatic Covid-19 infection in Sweden: A nationwide cohort study. Lancet Reg Health Eur, 2021. 11: p. 100249.
22. Nordström, P.B., M.; Nordström, A., Effectiveness of Covid-19 Vaccination Against Risk of Symptomatic Infection, Hospitalization, and Death Up to 9 Months: A Swedish Total-Population Cohort Study. SSRN, 2021.
23. Pawlowski, C., et al., FDA-authorized mRNA COVID-19 vaccines are effective per real-world evidence synthesized across a multi-state health system. Med (N Y), 2021. 2(8): p. 979-992 e8.
24. Pilishvili, T., et al., Effectiveness of mRNA Covid-19 Vaccine among U.S. Health Care Personnel. N Engl J Med, 2021. 385(25): p. e90.
25. Puranik, A.L., P.J.; Silvert, E.; Niesen, M.; Corchado-Garcia, J.; O’Horo, J.C.; Virk, A.; Swift, M.D.; Halamka, J.; Badley, A.D; Venkatakrishnan, A.J.; Soundararajan, V., Comparison of two highly-effective mRNA vaccines for COVID-19 during periods of Alpha and Delta variant prevalence. med Rxiv, 2021.
26. Self, W.H., et al., Comparative Effectiveness of Moderna, Pfizer-BioNTech, and Janssen (Johnson & Johnson) Vaccines in Preventing COVID-19 Hospitalizations Among Adults Without Immunocompromising Conditions – United States, March-August 2021. MMWR Morb Mortal Wkly Rep, 2021. 70(38): p. 1337-1343.
27. Skowronski, D.M., et al., Two-dose SARS-CoV-2 vaccine effectiveness with mixed schedules and extended dosing intervals: test-negative design studies from British Columbia and Quebec, Canada. medRxiv, 2021: p. 2021.10.26.21265397.
28. Tang, P., et al., BNT162b2 and mRNA-1273 COVID-19 vaccine effectiveness against the SARS-CoV-2 Delta variant in Qatar. Nat Med, 2021. 27(12): p. 2136-2143.
29. Tenforde, M.W., et al., Effectiveness of SARS-CoV-2 mRNA Vaccines for Preventing Covid-19 Hospitalizations in the United States. Clin Infect Dis, 2021.
30. Tenforde, M.W., et al., Association Between mRNA Vaccination and COVID-19 Hospitalization and Disease Severity. JAMA, 2021. 326(20): p. 2043-2054.
31. Thompson, M.G., et al., Effectiveness of Covid-19 Vaccines in Ambulatory and Inpatient Care Settings. N Engl J Med, 2021. 385(15): p. 1355-1371.
32. Voko, Z., et al., Nationwide effectiveness of five SARS-CoV-2 vaccines in Hungary-the HUN-VE study. Clin Microbiol Infect, 2021.
33. Klein, N.P., et al., Surveillance for Adverse Events After COVID-19 mRNA Vaccination. JAMA, 2021. 326(14): p. 1390-1399.
34. Shimabukuro, T.T., Updates on myocarditis and pericarditis following Moderna COVID-19 vaccination. Presentation to ACIP, 2022.
35. U.S. Food and Drug Administration, January 31, 2022 Approval Letter – Spikevax. 2022.
36. International Vaccine Access Center (IVAC). VIEW-hub. 2021; Available from: www.view-hub.org.

Table 1: Policy Question and PICO

Abbreviations: IM: intramuscular; PICO: population, intervention, comparison, outcomes.

Table 2: Outcomes and Rankings

^aThree options: 1. Critical; 2. Important but not critical; 3. Not important for decision making

Table 3a: Summary of Studies Reporting Symptomatic Laboratory-confirmed COVID-19

Table 3b: Summary of Studies Reporting Hospitalization due to COVID-19

Table 3c: Summary of Studies Reporting Death due to COVID-19

Table 3d: Summary of Studies Reporting Asymptomatic SARS-CoV-2 infection (assessed bAB levels against SARS-CoV-2)

^aAdditional data provided by study sponsor

^bVaccine effectiveness estimate included in main pooled analysis used for GRADE.

Table 3e: Summary of Studies Reporting Serious Adverse Events^a

^aAdditional data provided by sponsor

^bRisk of bias was not formally assessed for these small studies with no comparator; these were not included in the quantitative estimate used for GRADE.

^c15 serious adverse events reported in 12 participants were deemed by blinded investigators to be related to vaccination. These included: B-cell small lymphocytic lymphoma, Basedow’s disease, autonomic nervous system imbalance, cerebrovascular accident, multiple sclerosis, pericardial effusion, pericarditis, pleural effusion, nausea, vomiting, alopecia areata, angioedema, rheumatoid arthritis, and two reports of facial swelling.

^dRisk evaluated in a 7-day interval following vaccination

^eRisk evaluated in a 0–1 day risk interval after vaccination

^fRisk evaluated in a 7-day interval following dose 2

Table 3f: Summary of Studies Reporting Reactogenicity^a

Table 4. Grade Summary of Findings Table

CI: Confidence interval; RR: Risk ratio

Explanations 

1. Risk of bias related to blinding of participants and personnel was present. Although participants and study staff were blinded to intervention assignments, they may have inferred receipt of vaccine or placebo based on reactogenicity. This was deemed unlikely to overestimate efficacy or underestimate risk of serious adverse events, therefore the risk of bias was rated as not serious.
2. The effects noted are from a per protocol analysis with outcomes assessed at least 14 days post dose 2 among persons who received two doses and had no evidence of prior SARS-CoV-2 infection. In an analysis using the full analysis set (persons with or without evidence of prior SARS-CoV-2 infection), there were 58 cases among 15180 persons in the vaccine arm and 754 cases among 15166 persons in the placebo arm (RR = 0.08 (0.06 to 0.10)).
3. The RCT excluded persons with prior COVID-19 diagnosis, pregnant or breastfeeding women, and persons who were in an immunosuppressive or immunodeficient state, had asplenia or recurrent severe infections (HIV-positive participants on stable antiretroviral therapy were not excluded). The population included in the RCT may not represent all persons aged >=18 years.
4. Absolute risk was calculated using the observed risk among placebo recipients in the available body of evidence from randomized controlled trials. Absolute risk estimates should be interpreted in this context.
5. 14 studies were available in the body of evidence. 2 were excluded because the study population was already represented, and 1 was excluded because the confidence interval for the effect estimate was provided.
6. The body of evidence includes preprints.
7. Although I2 value was high (96.6%), no serious concern for inconsistency was judged because all studies showed a high degree of vaccine effectiveness, with point estimates ranging from 79% to 92%.
8. Two included studies measured COVID-19 vaccine effectiveness against laboratory-confirmed COVID-19–associated emergency department and urgent care clinic encounters and hospitalizations. In a sensitivity analysis excluding these studies the pooled RR was 0.11 (95% CI: 0.06, 0.22).
9. Data on numerators and denominators were not consistently reported in the available body of evidence. Seven test negative studies contributed 404,823 cases and 1,083,774 controls. In addition to the test negative study data, four cohort studies contributed data to the pooled estimate.
10. Pooled RR based on a random effects meta-analysis, using adjusted vaccine effectiveness estimates on a log scale.
11. Hospitalization due to COVID-19 was not a protocol defined outcome and was provided in a post-hoc analysis, which may be subject to bias. However, severe COVID-19, defined per FDA guidance, was a protocol defined outcome. There were 2 severe COVID-19 cases among 14164 persons in the vaccine arm and 106 severe COVID-19 cases among 14287 persons in the placebo arm (RR = 0.02 (0.00 to 0.08)). Due to the similarity of these estimates, we deemed the risk of bias as not serious.
12. Serious concerns of imprecision due to fragility in the estimate was present because there were only 25 events observed from a single RCT.
13. 19 studies were available in the body of evidence. Four were excluded because the study population was already represented, and one was excluded because no confidence interval for the effect estimate was provided. One included study provided two VE estimates from distinct study locations.
14. Although I2 value was high (77.6%), no serious concern for inconsistency was judged because all studies showed a high degree of vaccine effectiveness, with point estimates ranging from 93% to 97%.
15. Definitions varied by study. Indirectness was considered given COVID-19 was not necessarily confirmed as the cause of hospitalizations, but this was deemed not serious.
16. Data on numerators and denominators were not consistently reported in the available body of evidence. Ten test negative studies contributed 9,059 cases and 457,807 controls. In addition to the test negative study data, five cohort studies contributed data to the pooled estimate.
17. Very serious concern for imprecision was present due to the small number of events that were observed. In addition to a 95% confidence interval crossing the line of no effect, there was concern for fragility in the estimate due to the small number of events.
18. RR calculated using a standard continuity correction of 0.5.
19. All available studies in the body of evidence were included in the pooled estimate.
20. The relative risk shown is from a pooled analysis of 5 cohort studies conducted in different populations. I2 was 0%.
21. Definitions varied by study. Indirectness was considered given COVID-19 was not necessarily confirmed as the cause of deaths, but this was deemed not serious.
22. Data on numerators and denominators were not consistently reported in the available body of evidence. The n is not included because studies reported rates and did not report the number of cases. The N is not included because studies variously provided person-time.
23. Serious concern for inconsistency was present (I2= 90.3%). The magnitude of the relative risks from the three studies in the body of evidence varied widely, possibly reflecting different prevalence of circulating SARS-CoV-2 variants at the time of data collection or differences in study design.
24. Data on numerators and denominators were not consistently reported in the available body of evidence. Two test negative studies contributed 38,304 cases and 40,099 controls. In addition to the test negative study data, one cohort study contributed data to the pooled estimate.
25. Pooled RR based on a fixed effects meta-analysis, using adjusted vaccine effectiveness estimates on a log scale. Fixed effects model was used for this analysis due to imprecise estimates of the between-studies variance.
26. Two small Phase 1 observational studies with no comparison group were not included in the GRADE analysis. In a total of 35 vaccinated persons, 0 serious adverse events were observed.
27. Serious concern for imprecision was present. The confidence interval indicates that both reduced and increased risk of serious adverse events are possible.
28. Pooled RR based on a fixed effects meta-analysis. Fixed effects model was appropriate for this analysis because these RCTs used the same protocol and were conducted in similar populations.
29. For the outcome of myocarditis evaluated in Vaccine Safety Datalink, data are shown for the age groups 18-39, therefore these were not completely generalizable to the age groups of all persons aged >=18 years as defined in the PICO question. This was deemed not serious.
30. A rapid cycle analysis from Vaccine Safety Datalink (VSD) evaluated chart-reviewed cases of myocarditis occurring among persons aged 18–39 years following dose 2. Based on events occurring in a 7-day risk interval after vaccination vs. a comparison interval in vaccinated individuals, the adjusted rate ratio was 18.75 (95% CI 6.73–64.94). The excess cases of myocarditis in the risk interval were 31.2 per million doses.
31. Data from the national Vaccine Adverse Event Reporting System (VAERS) showed an elevated ratio of observed to expected myocarditis cases in the 7-day interval following vaccination among females in age groups 18-29 years, and among males in age groups 16-49 years, with higher observed/expected ratios in males than females. Although VAERS data are subject to the limitations of a passive surveillance system, the elevated risk of myocarditis following Moderna vaccination is consistent with that observed in VSD.
32. A rapid cycle analysis of data from VSD evaluated chart-reviewed cases of anaphalaxis among all vaccinated persons aged 18 years and older. Based on events occurring in a 0–1 day risk interval after vaccination, the estimated incidence of confirmed anaphalaxis was 5.1 (95% CI 3.3–7.6) per million doses.

Table 5: Summary of Evidence for Outcomes of Interest

Appendix 1. Studies Included in the Review of Evidence

Randomized Controlled Trial

Observational Cohort Studies

Observational Case-Control Studies

Safety Surveillance

^aPre-print

^bAdditional data provided by sponsor

^cThis was a primary outcome of the RCT, defined as SARS-CoV-2 RT-PCR-positive symptomatic illness, in seronegative persons aged ≥18 years, ≥7 days post second dose. In a secondary analysis among seronegative and seropositive persons, the efficacy was Grade 3 or worse.

^dGrade 3 local reactions include pain at injection site that prevents daily activity, redness > 10 cm, and swelling > 10 cm. Grade 3 systemic events include vomiting that requires IV hydration, diarrhea of 6 or more loose stools in 24 hours, or headache, fatigue/tiredness, chills, new or worsened muscle pain, or new or worsened joint pain that prevent daily routine activity.

^eNot reported

Appendix 2. Databases and strategies used for systematic review