How CDC Measures COVID-19 Vaccine Effectiveness
After the U.S. Food and Drug Administration (FDA) approves a vaccine or authorizes a vaccine for emergency use, experts continue to assess vaccine effectiveness—or how a vaccine works in real-world conditions. The goal is to understand how a vaccine protects people outside of strict clinical trial settings.
Evaluating COVID-19 Vaccine Effectiveness in the Real World
Experts are working on many different types of assessments to learn how COVID-19 vaccines work in real-world conditions. Each assessment uses a different method, which are described below.
- Case-control studies: These assessments include case-patients (people who have the virus that causes COVID-19) and controls (people who do not have the virus that causes COVID-19). The people who agree to participate in a case-control assessment will provide information on whether they received a COVID-19 vaccine or not. Experts look to see if the case-patients were less likely to have received the vaccine than controls, which would show that the vaccine is working.
- Test-negative design: This is a special type of case-control study. These assessments enroll people who are seeking medical care for symptoms that could be due to COVID-19. Experts then compare the COVID-19 vaccination status of those who test positive (meaning they have COVID-19) to those who test negative (meaning they do not have COVID-19).
- Cohort studies: These assessments follow people who have and haven’t had a COVID-19 vaccine for several months to see if getting vaccinated protects them from getting the disease. This can be done in real time (prospectively) or by looking back in time (retrospectively) using data that were already collected, such as information in participants’ electronic health records (medical records).
- Screening method: These assessments look at vaccination coverage among a group of people with COVID-19 (for example, people with cases of COVID-19 detected through ongoing COVID-19 monitoring) and compares it with vaccination coverage among the overall population where those people with COVID-19 come from (for example, people from the same state). By comparing coverage among these two groups, researchers can get an early estimate of whether a vaccine is working as expected.
- Ecologic analyses: These assessments look at groups of people—such as those in different geographic locations or at different times—and find out how many people were vaccinated and how many people were diagnosed with COVID-19. These analyses may be hard to interpret because the number of COVID-19 illnesses has changed rapidly over time and in different places.
CDC is using several methods as they can all contribute different information about how a vaccine is working.
Cross-Agency Collaboration and Evaluating COVID-19 Vaccine Effectiveness
CDC coordinates with several other federal agencies to assess how COVID-19 vaccines work under real-world conditions after FDA approval or authorization for emergency use. These agencies include:
- Centers for Medicare and Medicaid Services (CMS)external icon
- Department of Defense (DoD)external icon
- Food and Drug Administration (FDA)external icon
- Indian Health Service (IHS)external icon
- Veterans Health Administration (VHA)external icon
Some examples of how these agencies are working together include:
- CDC, FDA, and VHA working groups are discussing how each is using electronic health records (medical records) to see if people who received a COVID-19 vaccine are protected against developing COVID-19.
- FDA and CMS are assessing how COVID-19 vaccines protect against developing COVID-19 among older adults, including those living in nursing homes and long-term care facilities.
What groups of people are included in the real-world vaccine assessments?
CDC makes sure real-world assessments include diverse groups of people, such as healthcare personnel, essential workers, older adults and those living in nursing homes, people with underlying medical conditions, racial and ethnic minority groups, and tribal nations. It is important to measure how COVID-19 vaccines work in groups of people who are at increased risk of getting COVID-19, as well as in those who are at increased risk of getting seriously ill if they get COVID-19.
CDC is conducting additional evaluations to rapidly assess vaccine effectiveness among healthcare personnel working in all healthcare settings. These studies will be informed by the experiences of healthcare personnel working in hospitals or nursing homes who were among those recommended to be vaccinated first against COVID-19. Healthcare personnel are among the groups that will provide a first look at how COVID-19 vaccines work in real-world conditions. Some of the things CDC will look at are how well the vaccines prevent people from getting sick or severely ill with COVID-19.
Essential non-healthcare workers are people who are essential to maintaining critical infrastructure, services, and functions. Essential workers may be at increased risk of getting COVID-19 because they are unable to physically distance and/or are exposed to unvaccinated persons at their jobs. Many essential workers are part of racial and ethnic minority groups disproportionately affected by COVID-19.
Experts are working to understand how COVID-19 vaccines work in real-world conditions among essential workers.
Older adults and those living in nursing homes
Making sure COVID-19 vaccines protect older adults is critical because the risk for severe illness from COVID-19 increases with age. People living in nursing homes and other long-term care facilities are at especially high risk of getting COVID-19 and having a severe illness. FDA and CMS are using CMS Medicare billing data to assess COVID-19 vaccine effectiveness among older adults, including those living in nursing homes and other long-term care facilities. These data include information about whether people got a COVID-19 vaccine, whether they got sick with COVID-19, and if they needed to receive care in a hospital for COVID-19. This information will help inform how the vaccine works in real-world conditions in preventing COVID-19, including severe illness, among older adults.
Experts are also conducting a case-control assessment using data from CDC and CMS. Experts will identify older adults hospitalized for COVID-19 and older adults hospitalized for other reasons. To estimate vaccine effectiveness, they will then compare how many in each of these groups received a COVID-19 vaccine.
Underlying medical conditions
Adults of any age with certain underlying medical conditions are at increased risk for severe illness from COVID-19. CDC is looking at how COVID-19 vaccines protect people who have heart conditions, obesity, diabetes, and other underlying medical conditions that place them at higher risk for severe COVID-19 outcomes. Experts are also working to make sure these real-world vaccine assessments include groups of adults who are racially and ethnically diverse.
Racial and ethnic minority groups
The percent of people who become ill, are hospitalized, or die from COVID-19 is higher among Hispanic or Latino, non-Hispanic Black, and non-Hispanic American Indian or Alaska Native than among non-Hispanic White people. Vaccine uptake among racial and ethnic minority groups is also lower than among non-Hispanic White people and this may be attributed to mistrust, or differences in healthcare access. These vaccines have been safely tested in persons of diverse racial and ethnic backgrounds.
CDC also is working with the Indian Health Service (IHS), tribal nations, and other partners to ensure real-world COVID-19 vaccine effectiveness assessments include American Indian and Alaska Native populations, who have been disproportionately affected by COVID-19. This is important to ensure that COVID-19 vaccines can help achieve health equity so that everyone has a fair opportunity to be as healthy as possible. Learn more about COVID-19 and health equity considerations.
CDC has recommended that some groups of people get COVID-19 vaccination first because of their increased risk of exposure from their jobs or because they are at higher risk for severe illness. Learn more about how CDC is making COVID-19 vaccination recommendations.
- Vaccine Effectiveness Presentation at the Oct. 22, 2020, FDA Vaccines and Related Biological Products Advisory Committee Meetingexternal icon
- FDA’s Center for Biologics Evaluation and Research Plans for Monitoring COVID-19 Vaccine Safety and Effectiveness [399 KB, 27 pages]external icon
- Food and Drug Administration COVID-19 Vaccinesexternal icon
- Combat COVID: Information about Clinical Trialsexternal icon