Skip Navigation LinksSkip Navigation Links
Centers for Disease Control and Prevention
Safer Healthier People
Blue White
Blue White
bottom curve
CDC Home Search Health Topics A-Z spacer spacer
spacer
Blue curve MMWR spacer
spacer
spacer

The content, links, and pdfs are no longer maintained and might be outdated.

  • The content on this page is being archived for historic and reference purposes only.
  • For current, updated information see the MMWR website.

Review of the 2003 National Syndromic Surveillance
Conference --- Lessons Learned and Questions To Be Answered

James W. Buehler
Emory University, Atlanta, Georgia

Corresponding author: James W. Buehler, Rollins School of Public Health, Rm. 416, Emory University, 1518 Clifton Rd., NE, Atlanta, Georgia 30322. Telephone: 404-712-8345; E-mail: jbuehle@sph.emory.edu.

Abstract

Syndromic surveillance is a rapidly evolving field within public health practice. Substantial experience has been gained in learning how to conduct syndromic surveillance, informed by a growing body of research and practice, including refinement of surveillance methods, development of new tools for analysis and evaluation, findings from statistical models and applied evaluations, and expansion of syndromic surveillance to uses beyond preparedness for biologic terrorism. Despite these advances, additional evaluation is needed to help health departments determine whether to conduct syndromic surveillance. This paper summarizes the lessons learned from the 2003 National Conference on Syndromic Surveillance, which provided a foundation for defining a research and evaluation agenda and for developing preliminary guidance for public health agencies planning to implement syndromic surveillance.

Introduction

Participants in the 2003 National Syndromic Surveillance Conference were junior- and senior-level professionals from multiple disciplines, including epidemiology, statistics, informatics, health care, and public health practice. Conference presentations outlined the substantial progress that has been made in understanding how to conduct syndromic surveillance. Methods are being refined, and additional health departments are gaining experience with syndromic surveillance. However, additional evaluation is needed before guidelines can be developed to help other health departments decide whether to conduct syndromic surveillance. This paper follows the outline used by the summary of the 2002 conference (1) to summarize the lessons learned at the 2003 conference and make recommendations for the future.

What Is Syndromic Surveillance?

The term syndromic surveillance describes the growing array of surveillance methods aimed at early detection of epidemics related to biologic terrorism. Although syndromic surveillance originated before 2001, the field grew substantially after the terrorist attacks of 2001 generated fears of future attacks. The word syndromic has been applied because the majority of such systems monitor different syndromes that might herald the early stages of epidemics (2). Other syndromic surveillance systems monitor health indicators of different actions persons might take or consequences they might suffer (e.g., miss work, use outpatient services, purchase medications, or require ambulance transport for emergency care) from the early stages of illness until death. Although certain syndromic surveillance systems depend on manual data collection, the 2003 conference emphasized systems that use automated methods to harvest data stored electronically and then transmit and analyze these data. The majority of presenters described ongoing surveillance, not systems designed to operate only during specific high-profile events.

The 2003 conference focused on describing the utility of syndromic surveillance, which remains, primarily, the early detection of an epidemic caused by deliberate release of a biologic agent. Syndromic surveillance also enables public health officials to provide reassurance that terrorism-related or other epidemics are not occurring, to detect the onset of expected seasonal upswings in viral respiratory and gastrointestinal infections, to detect common epidemics, and to conduct surveillance for a growing spectrum of health-related events.

Data Sources

Multiple data sources are being used for syndromic surveillance, limited only by the imagination of investigators. These sources can be classified into two broad categories: 1) clinical data arising from the use of health-care services (e.g., emergency department visits, clinic visits, or ambulance trip logs), and 2) all other indicators (e.g., pharmacy sales, calls to emergency numbers or information hotlines, and work or school absentee rates). Multiple health departments use a combination of data sources that complement one another.

The benefits of clinical data are twofold. First, productive relationships can arise between public health staff and clinicians as they establish and conduct syndromic surveillance. Second, the majority of clinical data sources enable investigators to follow up with individual patients when surveillance detects an unusual trend. Nonclinical data can complement clinical information by providing indicators of events (e.g., purchase of over-the-counter medications) that might occur before persons seek health care, by describing groups not represented at selected clinical facilities, or by validating trends observed in clinical data. One disadvantage of nonclinical data sources is that they typically do not readily allow for follow-up with affected persons.

Analytic Methods

Although various analytic methods are being used, two utilities are emerging as the statistical workhorses of syndromic surveillance: CDC's Early Aberration Reporting System (EARS), which detects unusual trends by time (3), and SaTScan (4), a program originally developed for detection of cancer clusters that identifies clustering by time and geographic location. As described elsewhere in these proceedings, substantial work is under way to develop new statistical methods for aberration detection and to refine syndrome categories.

Evaluation of Syndromic Surveillance Systems

After the 2002 conference, at which draft guidelines for evaluating syndromic surveillance systems were introduced, CDC engaged a panel to assist in revising these guidelines. A revised draft was distributed to participants at the 2003 conference, and the final version was published in Morbidity and Mortality Weekly Report (5). The guidelines rely on established CDC recommendations for evaluating surveillance systems but emphasize detection of epidemics rather than cases of illness. Presenters at the conference used the guidelines to describe surveillance systems and assess the balance between predictive value (i.e., the likelihood that a statistical alert represents a problem of public health importance) and sensitivity and timeliness (i.e., the likelihood that all epidemics are detected at the earliest possible stages).

Investigation of Signals

After being established, a syndromic surveillance system will inevitably generate alerts, indicating that a monitored indicator has surpassed a statistical threshold. When this happens, someone (typically an epidemiologist working in a local public health department) must decide whether, or to what extent, an investigation is warranted. Multiple conference presenters described their experiences with responding to signals, illustrating both the science and art of syndromic surveillance. Practitioners are developing graduated approaches to follow-up, ranging from closer examination of surveillance data to aggressive field investigation. They also report developing a sense of when signals merit more or less aggressive reactions. Certain practitioners wait to see whether aberrant trends persist for >1 day; others wait until more than one data source yields a signal before responding more aggressively. These varying approaches highlight the hard-to-quantify local rules that are evolving to maximize predictive value while minimizing losses in timeliness or sensitivity. What is known is that statistical alerts are common, certain alerts represent true public health emergencies, and substantial work is needed to characterize and quantify the relation between the presence or absence of an alert and the presence or absence of an outbreak.

Protecting Confidentiality

Protecting confidentiality while maximizing the usefulness of surveillance raises concerns regarding public health law, surveillance procedures, and relationships with the public. In the arena of public health law, one of the most important events of 2003 was the implementation of the Privacy Rule of the Health Insurance Portability and Accountability Act of 1996 (HIPAA). HIPAA governs the ways that health-care providers can share patient information but provides specific exemptions that allow for reporting of confidential health information to public health agencies for surveillance and other authorized disease prevention and control purposes (6). Clinicians, health-care managers, public health officials, and their attorneys are struggling to achieve an understanding of HIPAA, including how the provisions for reporting to public health agencies apply to syndromic surveillance. The distinction between syndromic surveillance, which is a public health practice and thus exempt from certain HIPAA privacy provisions, and research, which is governed differently under HIPAA, has emerged as a key concern. Presenters at the 2003 conference described certain successes in conducting surveillance in the HIPAA era but also reported difficulties.

Virtually all syndromic surveillance systems shun the collection of names or other identifiable information to ensure that privacy and confidentiality are not violated in the event of a security lapse. Systems also use multiple methods to encrypt data and ensure secure transmission and storage. Certain clinical systems assign numbers to patient-level surveillance records and provide only those numbers in reports to health departments so that identifying information is retained by the individual health-care facility. For those systems, any follow-up investigation is conducted through and with the assent of reporting site staff, who control access to identifying information. Other systems limit the detail reported to decrease the likelihood that patients will be identified inappropriately. Together, such measures reflect adherence to two principles in public health surveillance: collect information judiciously, and collect and retain identifying information as locally as possible. When describing syndromic surveillance systems based on automatic medical record systems, conference presenters referred to this practice as "the distributive data model" (7) because access to data is distributed in a manner commensurate with the respective roles of care providers and public health staff. The result is that epidemiologists have information needed to monitor community-level trends in selected syndromes. If surveillance indicates that further investigation is warranted, including review of individual patient records, then access can be requested from the health-care providers.

Long-term public support for syndromic surveillance will depend on both the public's perception that public health agencies are responsible stewards of any information with which they are entrusted, and on the perception that syndromic surveillance serves a useful public good. Thus, public health agencies must be diligent in communicating with the public about the utility of syndromic surveillance and about their strategies for protecting health information.

National and Local Data

Health departments seeking to establish syndromic surveillance can either develop data sources locally or tap national systems that provide local information. The question is no longer one of selecting one source versus another but of determining the right mix of local and national sources (e.g., the systems offered by the Real-Time Outbreak Disease Surveillance System group at the University of Pittsburgh [8] or the resources being developed by CDC under its BioSense program [9]). A critical question concerning these national sources is whether they will allow for rapid local follow-up with facilities or patients when they yield an aberrant signal that merits investigation.

Who Owns Syndromic Surveillance?

The question of who "owns" syndromic surveillance was raised at the 2002 conference because the leadership roles of different governmental, academic, and private participants were unclear (1). As demonstrated by presenters at the 2003 conference, innovative projects are being conducted or supported by multiple entities, including local, state, and national agencies; the U.S. Department of Defense; the U.S. Department of Homeland Security; and the U.S. Department of Health and Human Services. National coordination is increasingly being provided by CDC, as evident from its role in coordinating the development of evaluation guidelines and syndrome definitions, implementing BioSense, supporting national pilot projects, and providing state funding for surveillance under its terrorism-preparedness program.

Multiple Uses for Syndromic Surveillance

Compared with the 2002 conference, the 2003 meeting included considerably less discussion of whether syndromic surveillance, traditional surveillance, or astute clinicians would most likely be the first to detect an epidemic. Instead, the emphasis was on interactions among different epidemic-detection strategies, including how syndromic surveillance can alert clinicians to community trends and improve their diagnostic assessments (10). Syndromic surveillance and the usefulness of the resulting information can foster better relations among health departments, clinicians, and laboratorians, thereby enhancing the reporting of notifiable diseases or suspected clusters.

Another difference during the 2003 conference was that greater attention was given to nonterrorism-related applications of syndromic surveillance, for multiple reasons. In 2002, the events of 2001 were much fresher in our minds. Since 2001, the United States has not suffered another domestic terrorist attack, and the public's fears about domestic terrorism as the nation prepared for war in Iraq have not been realized. When the Federal government directed resources toward terrorism preparedness, public health officials recognized immediately that, to justify their expense, these efforts must extend beyond surveillance of terrorism-related syndromes. Furthermore, every naturally occurring outbreak is a limited rehearsal for responding to a terrorist attack. The emergence of severe acute respiratory syndrome (SARS) in 2003 demonstrated the nation's vulnerability to new infectious diseases and their potential for epidemic spread. Presenters at the 2003 conference discussed the feasibility of adapting syndromic surveillance for SARS detection, particularly emergency-department--based systems (11).

Finally, those who conduct syndromic surveillance are exploring other innovative uses of this new tool. For example, New York City used its pharmacy system to assess the impact of smoking cessation interventions by tracking sales of nicotine patches (12), and the U.S. Department of Defense examined the mental health effects of the terrorist attack on the Pentagon (13).

Importance of Partnerships

Those in the vanguard of this field represent successful partnerships between public health practitioners and academics. Syndromic surveillance is more complex than traditional surveillance and benefits from expertise in informatics, statistics, and advanced epidemiologic methods --- skills that health departments might not be able to maintain as a result of budget and mission constraints but that are readily available in universities. In turn, public health departments bring a familiarity with community resources, their relations with health-care providers, and their expertise in conducting surveillance and applying it to meet public health objectives.

Relation Between Surveillance and Disease Epidemiology

One theme that was less prominent at the 2003 conference was the epidemiology of potential agents of biologic terrorism. Usually, the conduct of surveillance is shaped by the epidemiology of the condition under surveillance, including how it is diagnosed, treated, or prevented. Relevant questions regarding the detection of terrorist-related epidemics include the following:

  • What is the likely shape of an epidemic curve?
  • How rapidly will different stages of illness occur?
  • How will the spectrum of illness become manifest with respect to different surveillance indicators?
  • How will these patterns vary among the potential agents of biologic terrorism?

In the absence of terrorist attacks, the answers will likely come from epidemiologic models that simulate a range of hypothetical scenarios and that test the usefulness of data sources and aberration-detection methods. Critical groundwork for conducting such investigations was described at this meeting (14).

Next Steps

Evaluation

The syndromic surveillance evaluation criteria developed by CDC (5) should be used in multiple ways. First, the criteria should be used to describe the field's rapidly growing experience in conducting syndromic surveillance. For example, how frequently do different syndromic surveillance methods generate statistical alerts, and what is learned when alerts are investigated? Conversely, how frequently are epidemics detected through other means also identified by syndromic surveillance? How does timeliness of detection compare with timeliness of other detection methods? CDC might request grantees conducting syndromic surveillance to add this information to required periodic reports. Aggregating, summarizing, and disseminating such reports will allow for a more comprehensive assessment of the usefulness of syndromic surveillance. Second, more in-depth evaluations of syndromic surveillance should be conducted in partnership with those states or localities that have the capacity to conduct such evaluations. Third, historic data should be used to test the utility of different detection algorithms; the work presented by the Defense Advanced Research Projects Agency and its collaborators illustrates the benefits of this approach (15). Fourth, epidemiologic models should be constructed to test the timeliness, sensitivity, and predictive value of detection strategies under different hypothetical scenarios; progress is being made in model development (14).

Research and Evaluation Funding

During the 2003 conference, representatives from three federal agencies --- CDC, the Agency for Healthcare Quality and Research, and the U.S. Department of Homeland Security --- described the research and evaluation activities they have funded or plan to fund. These funding agencies should take guidance from this conference to define a research and evaluation agenda for syndromic surveillance and, if necessary, update their funding priorities and clarify their roles accordingly. This would help applicants by clarifying practice and evaluation objectives and increase the likelihood that investigations funded by different agencies complement one another. Federal agencies should promote government and academic partnerships by making evidence of such collaboration part of funding criteria. One strategy might be to create centers of excellence in syndromic surveillance that would focus on methods development and evaluation and provide technical assistance to health departments.

Guidelines

Despite the advances highlighted during this conference, considerable questions remain to be answered, particularly for those agencies that have not yet initiated syndromic surveillance:

  • Where should syndromic surveillance be conducted? Should all states conduct a form of syndromic surveillance?
  • Within a state, should syndromic surveillance be conducted in only the largest cities or in medium-sized cities and rural areas as well?
  • If syndromic surveillance is conducted, what are the minimum standards for the selection or number of data sources?
  • What are the recommended methods for data analysis?

These questions are difficult to answer because experience and evaluation thus far are insufficient and because quantifying the risk of a terrorist attack for a given locality is impossible. As the field gains experience with syndromic surveillance, such decisions might ultimately be based on the usefulness of syndromic surveillance in detecting outbreaks not related to terrorism, with potential detection of terrorist-related events becoming a secondary use.

In the meantime, health department officials should feel assured that a decision not to conduct syndromic surveillance is justifiable. For those who have decided to implement syndromic surveillance, expecting definitive answers to the preceding questions is premature, but preliminary guidance can be developed. Because of its increasing role in coordinating syndromic surveillance and its history of leadership in public health surveillance, CDC is the logical agency to take the lead in developing such guidance, which should include articulation of the following:

  • planning steps, including whom to involve;
  • advantages and disadvantages of different data sources and commonly used or readily available statistical tools;
  • strategies for responding to alerts;
  • what utility to expect, and what is unknown; and
  • a plan to document experience and evaluate performance.

Partnerships with Community Representatives

The 2003 conference revealed a mix of partnerships involving public health professionals, clinicians, health-care administrators, emergency responders, legal experts, law enforcement, and companies that provide data and other surveillance resources. Thus far, however, the perspective of community representatives has not been prominent in deliberations about syndromic surveillance. For the majority of health problems, risk is not distributed proportionately among diverse populations. Biologic terrorism might not be an equal opportunity threat; the consequences of a terrorist attack are likely to affect most severely those populations that have long suffered the adverse consequences of health disparities. Involving community advocates is not always easy for public health professionals because advocates sometimes ask questions that are difficult to answer. However, they often have good questions, and their perspectives help ensure that surveillance meets community needs.

Conclusion

The field of syndromic surveillance has advanced considerably. An urgent need remains for continued evaluation of syndromic surveillance to define its utility and develop recommendations for its practice. Evaluation criteria developed by CDC should be used to the extent possible to guide assessments of syndromic surveillance based on both experience and hypothetical scenarios. The 2003 conference provided a basis for defining a comprehensive research and evaluation agenda. Although developing definitive guidelines on syndromic surveillance is premature, experience to date should enable the development of preliminary guidance to help those interested in stepping into this arena.

References

  1. Mostashari F, Hartman J. Syndromic surveillance: a local perspective. J Urban Health 2003;80(2 Suppl 1):i1--7.
  2. Henning K. What is syndromic surveillance? MMWR 2004;53 (Suppl):7--11.
  3. Hutwagner L, Thompson W, Seeman GM, Treadwell T. The bioterrorism preparedness and response early aberration reporting system (EARS). J Urban Health 2003;80(2 Suppl 1):i89--96.
  4. Kulldorff M and Information Management Services, Inc. SaTScan verion 4.0: software for the spatial and space-time scan statistics, 2004. Available at http://www.satscan.org.
  5. CDC. Framework for evaluating public health surveillance systems for early detection of outbreaks: recommendations from the CDC working group. MMWR 2004;53(No. RR-5).
  6. CDC. HIPAA Privacy Rule and public health: guidance from CDC and the US Department of Health and Human Services. MMWR 2003;52(Suppl):1--20.
  7. Yih WK, Caldwell B, Harmon R, et al. National Bioterrorism Syndromic Surveillance Demonstration Program. MMWR 2004;53(Suppl):43--6.
  8. Wagner MM, Espino J, Tsui F-C, et al. Syndrome and outbreak detection using chief-complaint data: experience of the Real-Time Outbreak and Disease Surveillance Project. MMWR 2004;53(Suppl):28--31.
  9. Loonsk J. BioSense---a national initiative for early detection and quantification of public health emergencies. MMWR 2004;53(Suppl): 53--5.
  10. Foldy SL. Linking better surveillance to better outcomes. MMWR 2004;53(Suppl):12--7.
  11. Foldy SL, Barthell E, Silva J, et al. SARS Surveillance Project---Internet-enabled multiregion surveillance for rapidly emerging disease. MMWR 2004;53(Suppl):215--20.
  12. Metzger K, Mostashari F, Heffernan R, Das D. Use of pharmacy data to evaluate impact of smoking regulations on sales of nicotine replacement therapies in New York City. Presentation, 2003 National Syndromic Surveillance Conference, New York, New York, October 2003. Available at http://www.syndromic.org/pdf/con2-KM-4.pdf.
  13. Pavlin J. Conducting population behavioral health surveillance using automated diagnostic and pharmacy data systems. MMWR 2004; 53(Suppl):166--72.
  14. Mostashari F. Analytic methods track: a summary. Presentation at the 2003 National Syndromic Surveillance Conference, New York, New York, October 2003. Available at http://www.syndromic.org/pdf/con2-FM-9.pdf.
  15. Siegrist D. Bio-ALIRT biosurveillance detection algorithm evaluation. MMWR 2004;53(Suppl):152--8.

Use of trade names and commercial sources is for identification only and does not imply endorsement by the U.S. Department of Health and Human Services.


References to non-CDC sites on the Internet are provided as a service to MMWR readers and do not constitute or imply endorsement of these organizations or their programs by CDC or the U.S. Department of Health and Human Services. CDC is not responsible for the content of pages found at these sites. URL addresses listed in MMWR were current as of the date of publication.

Disclaimer   All MMWR HTML versions of articles are electronic conversions from ASCII text into HTML. This conversion may have resulted in character translation or format errors in the HTML version. Users should not rely on this HTML document, but are referred to the electronic PDF version and/or the original MMWR paper copy for the official text, figures, and tables. An original paper copy of this issue can be obtained from the Superintendent of Documents, U.S. Government Printing Office (GPO), Washington, DC 20402-9371; telephone: (202) 512-1800. Contact GPO for current prices.

**Questions or messages regarding errors in formatting should be addressed to mmwrq@cdc.gov.

Page converted: 9/14/2004

HOME  |  ABOUT MMWR  |  MMWR SEARCH  |  DOWNLOADS  |  RSSCONTACT
POLICY  |  DISCLAIMER  |  ACCESSIBILITY

Safer, Healthier People

Morbidity and Mortality Weekly Report
Centers for Disease Control and Prevention
1600 Clifton Rd, MailStop E-90, Atlanta, GA 30333, U.S.A

USA.GovDHHS

Department of Health
and Human Services

This page last reviewed 9/14/2004