How CDC Monitors COVID-19 Vaccine Effectiveness
After the U.S. Food and Drug Administration (FDA) authorizes a vaccine for emergency use or approves a vaccine, experts continue to assess how well the vaccine is working in real-world conditions. Clinical trials of the available COVID-19 vaccines are important for authorization and approval, but their results may not reflect how effective the vaccines are once administered beyond the clinical trial. CDC has established methods to monitor vaccine effectiveness in real-world conditions and address ongoing and future questions around how vaccines perform.
CDC uses several methods to evaluate COVID-19 vaccine effectiveness. These methods can contribute different information and build a base of evidence about how COVID-19 vaccines are working. The information collected through this combined approach enables CDC to monitor vaccine effectiveness over time. This can help inform public health action to continue to best protect the population.
Vaccine Effectiveness Studies
Experts are working on many types of assessments to determine COVID-19 vaccine effectiveness in real-world conditions. These are known as vaccine effectiveness (“VE”) studies. The goal is to understand how a vaccine protects people outside of strict clinical trial settings, where storage and delivery are monitored and participants are usually healthy. Vaccine effectiveness measures vaccine protection in observational studies, which:
- Include people with underlying medical conditions
- Include people who have been given vaccines by different types of healthcare professionals under real-world conditions
- Occur in settings where people may make different decisions about vaccination
There are several kinds of observational studies, which use different methods:
- Case-control studies include cases (people who have COVID-19) and controls (people who do not have COVID-19). People who agree to participate in a case-control study provide information on whether they received a COVID-19 vaccine. Experts look back to see if cases were less likely to be vaccinated than controls, which would show the vaccine is working.
- Some case-control studies use a test-negative design. These studies enroll people who are seeking medical care for symptoms that could be due to COVID-19. In this special type of case-control study, experts compare the COVID-19 vaccination status of those who test positive, meaning they have COVID-19, to those who test negative, who do not have COVID-19).
- Cohort studies observe groups of people, both vaccinated and unvaccinated, then follow them for a period of time to see if they get COVID-19. Experts compare COVID-19 illness in the vaccinated and unvaccinated groups to measure how well COVID-19 vaccines protected against the disease. This can be done in real time (prospectively) or by looking back in time (retrospectively) using data already collected, such as information in participants’ medical records.
- Screening method assessments look at vaccination status among a group of people infected with the virus that causes COVID-19 (for example, those detected through ongoing COVID-19 surveillance) and compare that with the vaccination coverage in the overall population where those cases arise (for example, people from the same state). By comparing vaccination coverage between these two groups, researchers can get an estimate of whether a vaccine is working as expected.
- Ecologic analysis assessments look at groups of people—such as those in different locations or at different times—using different sources of data to see if there is an overall association between how many were vaccinated against COVID-19 and how many were diagnosed with COVID-19. These analyses may be hard to interpret broadly because an association that might appear at the group level might not exist on an individual level. Also, it is hard to control for differences in the different sources of data.
Cross-Agency Collaboration and Evaluating COVID-19 Vaccine Effectiveness
CDC coordinates with several other federal agencies to assess how COVID-19 vaccines work under real-world conditions after FDA authorization for emergency use or approval. These agencies include:
- Centers for Medicare and Medicaid Services (CMS) external icon
- Department of Defense (DoD) external icon
- Food and Drug Administration (FDA) external icon
- Indian Health Service (IHS) external icon
- Veterans Health Administration (VHA) external icon
What Groups of People Are Included in the Real-World Vaccine Assessments?
CDC is looking at how well COVID-19 vaccines work in different groups of people, such as, essential workers, racial and ethnic minority groups, and tribal nations. It is important to measure how COVID-19 vaccines work in groups of people who are more likely to get COVID-19, as well as in those who are more likely to get seriously ill if they get COVID-19. Through these studies in various populations, locations, and settings, CDC can obtain more representative, scientifically valid, and complete information about vaccine effectiveness.
CDC is rapidly assessing vaccine effectiveness among healthcare personnel working in healthcare settings. Healthcare personnel are more likely to get COVID-19 while taking care of patients. Healthcare personnel are among the groups that are providing a first look at how COVID-19 vaccines work in real-world conditions.
Essential non-healthcare workers are needed to maintain critical infrastructure, services, and functions. Essential workers may be more likely to get COVID-19 because they unable to work from home, are unable to keep at least 6 feet from others or are exposed to people with COVID-19 at their jobs, and often lack paid sick leave. Many essential workers are disproportionately affected by COVID-19.
Older adults and those living in nursing homes
Making sure COVID-19 vaccines protect older adults is critical, because people are more likely to get severely ill from COVID-19 as they get older. People living in nursing homes and other long-term care facilities are much more likely to get COVID-19 and to have a severe illness. FDA and CMS are using CMS Medicare billing data to measure COVID-19 vaccine effectiveness among older adults, including those living in nursing homes and other long-term care facilities. These data include information about whether people are vaccinated against COVID-19, whether they got sick with COVID-19, and if they needed to receive care in a hospital for COVID-19. Experts are also conducting a case-control assessment using data from CDC and CMS. Experts will identify older adults hospitalized for COVID-19 and older adults hospitalized for other reasons. To estimate vaccine effectiveness, they will then compare how many in each of these groups received a COVID-19 vaccine.
Underlying medical conditions
Adults of any age with certain underlying medical conditions are more likely to get severely ill from COVID-19. CDC is looking at how COVID-19 vaccines protect people who have heart conditions, obesity, diabetes, and other underlying medical conditions that place them at increased risk of severe illness.
Racial and ethnic minority groups
The proportion of people who become ill, are hospitalized, or die from COVID-19 is higher among Hispanic or Latino, non-Hispanic Black, and non-Hispanic American Indian or Alaska Native people than among non-Hispanic White people. Vaccine uptake among some racial and ethnic minority groups is also lower than among non-Hispanic White people. Experts are working to make sure real-world vaccine assessments include groups of adults from racial and ethnic minority groups. CDC also is working with the Indian Health Service, tribal nations, and other partners to ensure real-world COVID-19 vaccine effectiveness assessments include American Indian and Alaska Native populations. It is important real-world vaccine effectiveness studies include people from different racial and ethnic minority groups to ensure COVID-19 vaccines help achieve health equity.
Investigating Breakthrough Infections
COVID-19 vaccines are effective and are a critical tool to help bring the pandemic under control. However, no vaccines are 100% effective at preventing illness in vaccinated people. There will be a small percentage of fully vaccinated people who still get sick, be hospitalized, or die from COVID-19. As with other vaccines, this is expected. As the number of people who are vaccinated goes up, the number of breakthrough cases is also expected to increase.
Vaccine breakthrough cases may occur for many reasons:
- Variants of the virus that causes COVID-19 could emerge that are less vulnerable to available vaccines.
- Protection (immunity) in vaccinated people could lessen over time.
- Specific people may develop less immunity after vaccination.
CDC will work with state, tribal, local and territorial authorities to identify and investigate specific institutional and facility outbreaks, for example specific outbreaks in shared or congregate housing. They will assess whether any of the reasons listed above play a role in why vaccinated people might still get COVID-19.
CDC is also working with state and territorial health departments to investigate vaccine breakthrough cases. This includes monitoring trends in case characteristics, such as age or vaccination status, and virus variants identified from people with these infections. Investigations of breakthrough cases help support the vaccine effectiveness studies under way. Cases of vaccine breakthrough that occur within vaccine effectiveness studies are also examined.
National Surveillance of Vaccine Breakthrough Cases
When COVID-19 vaccines were being introduced, CDC created a vaccine breakthrough surveillance system to allow state and local health departments to voluntarily report vaccine breakthrough cases to CDC. State and local health departments continue to report breakthrough cases to CDC to identify and investigate patterns or trends among hospitalized or fatal vaccine breakthrough cases. CDC will continue this activity until more sustainable systems for monitoring breakthrough cases are fully established.
CDC is also working with the Emerging Infections Program (EIP) sites in nine states to obtain genetic sequence data (like a DNA fingerprint) for the virus that causes COVID-19 from vaccine breakthrough cases. Those data will include asymptomatic and mild vaccine breakthrough cases and compare those to COVID-19 cases among unvaccinated people.
See the latest data on hospitalized or fatal vaccine breakthrough cases.
- Vaccine Effectiveness Presentation at the Oct. 22, 2020, FDA Vaccines and Related Biological Products Advisory Committee Meetingexternal icon
- FDA’s Center for Biologics Evaluation and Research Plans for Monitoring COVID-19 Vaccine Safety and Effectiveness [399 KB, 27 pages]external icon
- Food and Drug Administration COVID-19 Vaccinesexternal icon
- Combat COVID: Information about Clinical Trialsexternal icon