Novel COVID-19 survey takes nation’s social, mental “Pulse”
The effects of COVID-19 go beyond numbers of cases and deaths. How many people are struggling under the stresses of the pandemic?
Answers from the Household Pulse Survey are tracking
our mental, social, and economic health during the
pandemic.
Is mental health suffering as Americans try to manage isolation, worries about jobs, and a constant stream of anxiety-producing headlines? Are they putting their future health at risk by delaying trips to the doctor or avoiding the emergency room when needed?
The Household Pulse Survey is helping to answer these questions by innovating and capturing data in new ways. The experimental survey aims to measure these other costs – and get answers before it’s too late to act.
Uncovering American experiences
Spearheaded by the US Census Bureau, the Household Pulse Survey is an experimental data project that taps into the full knowledge and resources of CDC and five other large federal agencies, each doing what they do best: collecting data on important issues that affect our everyday lives. The data collected will uncover what people are experiencing during the pandemic in terms of employment status, spending patterns, physical and mental health, access to health care, food security, housing, and educational disruption.
Initially, Pulse will reach out to a total of 14 million people nationwide, through emails and texts asking them to go online and complete a 20-minute questionnaire. With about 100,000 responses every week – and only about 8 days between collecting those responses and sharing them with the public – the survey provides a near real-time snapshot of what’s happening.
As part of the project, CDC’s National Center for Health Statistics (NCHS) is gathering data to monitor symptoms of anxiety and depression, whether people are delaying medical care, and if their health insurance coverage has changed.
Federal agencies rise to the challenge
“Our nation’s federal statistical system is set up to collect and deliver survey information in a way no one else can,” says NCHS Director Dr. Brian Moyer. “Statistical agencies have the resources to do things right. We know which questions to ask. We collect high-quality data and get high response rates. We minimize biases.”
However, ensuring this high level of reliability takes time. The federal statistical system was designed to prioritize quality over speed, with the data normally undergoing months of careful analysis and verification before it gets released.
“But pandemics don’t wait,” says Dr. Moyer. “So we had to reinvent how we do things.”
Agencies needed to rapidly deliver data that policymakers could trust to make immediate decisions. As the largest of the nation’s federal statistical agencies, the Census Bureau was uniquely poised to make this happen. They sprang into action to create a new data system that could work faster while using a high-quality, probability-based sampling frame to get more reliable results.
Within about a month, they had pulled together with five other federal agencies – NCHS, the Bureau of Labor Statistics, USDA Economic Research Service, the National Center for Education Statistics, and the Department of Housing and Urban Development – to design the Household Pulse Survey questionnaire and begin sending it out. The first invitations to participate in Pulse were sent out on April 23.
Delivering data in record time
NCHS had to rethink its systems to handle the rapid response. Within the questionnaire are eight questions specifically chosen by NCHS to identify outcomes that might be affected by the pandemic and, wherever possible, for which CDC had previous data to use as a benchmark to measure change.
“We had to act quickly to make ourselves ready to process all this incoming information,” says Emily Terlizzi, a health statistician at NCHS and lead analyst for this project. In an extraordinary effort, NCHS’s analysts and programmers developed an automated system that can turn the survey data around and publish it in record time – within hours after receiving it.
The new system captures changes happening week by week, resulting in a national picture across all 50 states that can be broken down by area, age, gender, race and ethnicity, and educational level.
An eye on quality results
Because the Household Pulse Survey is the first of its kind, federal statisticians are keeping an eye on the quality of the new data being collected.
“We’ll be able to know something about the quality of the data NCHS is receiving because we have high-quality baseline data from our National Health Interview Survey (NHIS),” says Dr. Stephen Blumberg, who leads that program within NCHS.
Every year, NCHS and Census Bureau staff knock on about 60,000 doors to conduct hour-long interviews with people about their health. Ongoing since 1957, NHIS is the oldest continuously fielded health survey in the United States.
“Because of NHIS, we have data from previous years on the same topics we’re now assessing through the Pulse Survey, and we can compare the Pulse data against them,” says Blumberg. “We’ll also be able to use this year’s NHIS data to assess the quality of the data we’re gathering from Pulse.”
Experimenting toward the future of health data
The Household Pulse Survey demonstrates what the federal statistical system can do in emergencies, while creating even more potential for how data can work in the future.
“Having a survey like Pulse in the field producing weekly estimates not only gives us important information for policies and decisions related to COVID-19 but is also an asset for examining the impact of other major events,” Blumberg notes. For example, Pulse data released for the last week of May showed a statistically significant increase in symptoms of anxiety and depression for black adults when compared to the previous week.
While fast-turnaround surveys like Pulse will never replace the careful and deliberative surveys the US government has relied on for decades, they add a critical tool to the federal arsenal, creating the ability to deliver near real-time insights on our nation’s experiences when it matters most.