BRFSS Data Quality Research
Research on Item Non-Response
Item nonresponse, or respondents' refusal to answer specific questions on a survey, can jeopardize data quality in all survey-based research. Item nonresponse may suggest a survey has a flawed design, that there are questions of a sensitive nature that respondents deem too personal to answer (such as questions about household income), or that there are questions too complex for respondents to answer. Item non- response research is focused on whether respondents on landline or cellular telephones are more or less likely to refuse to answer questions, or whether the mode of interview is likely to affect item nonresponse. Data from surveys are analyzed to assess patterns of nonresponse, including refusal to answer questions and/or patterns of "do not know" responses. Understanding some of the reasons why inadequate responses were given to BRFSS questions can help determine whether particular questions should be reworded or if respondents need more explanation in order to understand a question and answer it accurately. Item nonresponse research is ongoing at the BRFSS and will continue through the foreseeable future.
Research on Response Rate Variation among States
Factors that influence whether an individual chooses to participate in a survey is well documented in the scientific literature. In most instances, researchers have focused on personal demographic characteristics or household information. More research is needed, however, to determine the associations between respondents' community or state-level characteristics and survey response. To address this gap in the knowledge base, research is being conducted to review response rate variation among states and territories participating in the BRFSS. This research focuses on community-level factors that may help to explain variation in response rates, including demographic characteristics about respondents, social characteristics, and state culture. Data from the American Community Survey and the Current Population Survey—as well as calling outcomes from the BRFSS—are combined to help to explain variations seen in response rates. This project is scheduled to be completed in 2013.
Research on Changes in Marginal Costs of Cellular Telephone Surveys
This research, which was completed in 2011, used data from the BRFSS in 33 states to calculate marginal changes in costs after the introduction of cellular telephone interviews. BRFSS data showed that cell phone surveys cost approximately 1.5- to 3-times more than completed landline surveys. Data related to the age of the respondents (mean age estimates) for each state were taken from the landline and cellular telephone portions of the survey and were compared with corresponding data from the U.S. Census. The researchers also made calculations to assess changes in cost after cellular telephones were included in the data collection, to determine the change in cost required to complete a survey. Results illustrated that the introduction of cellular telephone interviews, although more costly than landline telephone interviews, reduced the error of some estimates, thereby making them more valid. This research was presented at the American Association of Public Opinion Research (AAPOR) conference in May 2011.
Research on Response Latency
The BRFSS is conducting research on response latency, or the time it takes for respondents to answer specific questions on surveys. Published research indicates that longer time lapses—or response latency—between questions and responses may suggest a false response was given or that the respondent is confused about what the question is asking or how to answer it. Response-latency differences may be explained in part by demographic characteristics of respondents or differences among interviewers. Latency may also be associated with respondents' states of mind while completing a survey. The BRFSS is conducting research to determine whether respondents who report they have experienced symptoms of anxiety and/or depression may take longer to respond, especially when surveys include sensitive questions. Analyses are being conducted that use statistical and other techniques that help researchers account for many different respondent characteristics as well as the types of questions they are answering. Response latency research will continue through 2013.
Research on Cellular Telephone Calling Outcomes
BRFSS is researching the results of contacting potential respondents who use cellular telephones. This research is currently focused on the calling outcomes for cellular telephone numbers included in the BRFSS samples from 2008–2011. Survey completion trends are being examined over time for the pooled data collected across several states as well as for the data from selected individual states with large numbers of cellular telephone users. Patterns of response rates, refusal rates, contact rates, out-of-sample numbers, terminations, and partial completes are being examined. This research will be completed in 2013 and will be presented at the 2013 American Association of Public Opinion Research (AAPOR) conference.