Three Years of Emergency Department Gastrointestinal Syndromic Surveillance in New York City: What Have we Found?

Sharon Balter, D. Weiss, H. Hanson, V. Reddy, D. Das, R. Heffernan
New York City Department of Health and Mental Hygiene, New York, New York

Corresponding author: Sharon Balter, New York City Department of Health and Mental Hygiene, 125 Worth Street, Box 22A, New York, NY 10013. Telephone: 212-788-9662; Fax: 212-676-6091; E-mail: sbalter@health.nyc.gov.

Disclosure of relationship: The contributors of this report have disclosed that they have no financial interest, relationship, affiliation, or other association with any organization that might represent a conflict of interest. In addition, this report does not contain any discussion of unlabeled use of commercial products or products for investigational use.

Abstract

Background: Use of syndromic surveillance as a tool to detect outbreaks and potential biologic or chemical terrorist attacks is increasing. Evaluating health departments' use of syndromic surveillance is necessary to determine the value of this methodology.

Methods: Syndromic surveillance signals detected by the New York City Department of Health and Mental Hygiene (DOHMH) during November 2001--August 2004 were reviewed for diarrhea and vomiting syndromes, the methods used to investigate such signals, and results of these investigations to determine if any unreported outbreaks were detected. Gastrointestinal (GI) outbreaks reported to DOHMH also were reviewed to understand why they were not detected by DOHMH's Emergency Department (ED) syndromic surveillance system.

Results: During the study period, ED surveillance generated 98 citywide and 138 spatial GI signals. Multiple outbreaks suspected to be caused by norovirus and rotavirus were identified, as well as a citywide increase in diarrheal illness. Of 98 citywide signals detected, 73 (75%) occurred during seasonal outbreaks. During the same period, 49 GI outbreaks were reported to DOHMH; none was detected simultaneously by ED surveillance.

Conclusion: Only substantial, citywide syndromic signals were identified as outbreaks and routinely reported. GI outbreaks did not generate syndromic signals. Syndromic surveillance signals occur frequently, are difficult to investigate satisfactorily, and should be viewed as a supplement to, rather than a replacement for, well-maintained traditional surveillance systems that rely on strong ties between clinicians and public health authorities.

Background

Syndromic surveillance is increasingly used as a tool to detect both naturally occurring outbreaks and potential biologic or chemical terrorist attacks (1). In the absence of etiologic information, these systems use constellations of symptoms, complaints, or diagnostic codes to group patients into syndrome categories. Data can be gathered from emergency department (ED) logs (2--5), hospital admissions records (6), ambulatory care center records (7,8), ambulance dispatch records (9), or clinical laboratory submissions (10). Other data sources can include over-the-counter (OTC) medication sales (10--13), nurse hotline calls (14,15), and work and school absenteeism records (16) to identify patients who have not sought medical care. Although syndrome-based surveillance has long been used to detect and track diseases for which etiologic diagnoses are made infrequently (e.g., influenza and polio-myelitis) (17--20), since the 2001 anthrax attacks, such systems have been used as early warning systems to detect biologic or chemical terrorism (21--25). Syndromic surveillance is based on the concept that illnesses caused by agents likely to be used in a biologic or chemical terrorist attack (e.g., plague or anthrax) will first manifest with nonspecific (prodromal) symptoms (25). In principle, syndromic surveillance systems should detect outbreaks of naturally occurring illness and those caused by intentional attacks. The theoretic ability of these systems to detect such attacks has been described (1,2,25); however, to date, these systems have been useful primarily to detect and monitor substantial seasonal outbreaks of influenza, rotavirus, and norovirus (7,8). Whether syndromic surveillance systems also can detect smaller, more localized outbreaks or identify outbreaks that are not reported through traditional surveillance is not known, and, despite their increasing use, few systems have been evaluated (24). Having a better understanding of the experiences of health departments that use syndromic surveillance systems might help to improve the usefulness of this methodology.

Methods

Since November 2001, the New York City Department of Health and Mental Hygiene (DOHMH) has operated an ED syndromic surveillance system (2). Every day, EDs transfer electronic data to DOHMH regarding the age, sex, home ZIP code, date and time of visit, and chief complaint of patients examined the previous day. A computer algorithm codes chief complaints into four syndromes: vomiting, diarrhea, fever, and respiratory; complaints that do not fit these categories are coded as "other." Data are analyzed daily for aberrations in time and space, which are reported as either citywide or spatial signals. Spatial signals indicate clustering in syndrome visits by either hospital or patient home ZIP code.

For this report, DOHMH reviewed ED gastrointestinal (GI) syndromic surveillance data collected during November 15, 2001--August 15, 2004, to determine whether GI syndromic signals represented real disease clusters and whether syndromic surveillance detected known GI outbreaks. During the study period, the data collection system increased from 28 (42%) of 67 EDs, representing approximately 57% of ED visits in NYC, to 48 (73%) of 66 EDs, representing approximately 90% of ED visits (one ED had closed during that period).

To determine whether GI syndromic surveillance signals represented real outbreaks, DOHMH reviewed 236 GI signals (e.g., vomiting or diarrhea) detected during November 15, 2001--August 15, 2004, together with any documented signal investigations conducted during this period. An analyst and physician jointly decided whether to begin, and how far to pursue, an investigation.

Multiple possible steps are involved in an investigation of citywide or spatial signals. For a citywide signal, hospital-level data are evaluated to determine whether one or multiple hospitals account for the majority of cases to focus the investigation, and, if so, ED clinical staff at these hospitals are asked whether they have noticed anything unusual and whether the trend is continuing. They are also asked to be aware of new patients reporting with the syndrome of concern and to notify DOHMH if they notice clusters of patients with similar symptoms or young and otherwise healthy patients with severe symptoms. For a spatial signal, the ED patient line list is reviewed to determine the age and chief complaints of patients in the cluster before the ED is called. For both citywide and spatial signals, other syndromic surveillance data sources (e.g., records of sales of OTC medications) are reviewed for corroboration. If concern persists after the line list has been reviewed and clinicians have been contacted, the midday 12-hour chief complaint log is requested from hospitals in the signal to determine if the trend is continuing. Eight hospitals can send midday logs electronically; other hospitals photocopy their paper logbooks for the period from midnight to midday, black out identifying information, and send them to DOHMH by facsimile. Faxed logs are then hand coded and the proportion of syndrome visits compared with the signal and 7day baseline. Depending on the size and timing of the signal, whether or not it is sustained, or other information suggests that the signal indicates a true increase in illness, the signals might raise greater concern. For such signals, medical charts are abstracted by either hospital or DOHMH staff. Occasionally, patients have been called and asked whether they have improved; on one occasion, after a blackout in August 2003, a case-control study was conducted (26). Because a common microbial pathogen suggests a link among patients, an attempt was made to identify an etiologic diagnosis for signals of greater concern. However, obtaining specimens was challenging because the patient usually had been discharged from the ED by the time a signal was detected. Efforts to persuade EDs to augment their specimen collections have not succeeded because these laboratory studies typically do not affect clinical care and incur added effort, cost, and burden of tracking results. DOHMH staff members have occasionally gone onsite to collect specimens, but this activity is resource intensive, and despite arranging specimen transport, compliance has been low.

A total of 49 GI outbreaks investigated by DOHMH during November 15, 2001--August 15, 2004, were reviewed to determine whether syndromic surveillance can detect known GI outbreaks. All outbreaks involved >10 persons with vomiting or diarrhea symptoms. Outbreaks were reported to DOHMH from multiple sources (Table).

Results

Review of Syndromic Surveillance Signal Investigations

During the study period, investigations detected 236 GI signals, including 98 citywide and 138 spatial signals. Of these, 20 (8.4%) were documented in writing, including all investigations that determined microbial etiology. Signal investigations determined that annual citywide outbreaks of diarrheal illness were likely attributable to norovirus (typically during fall and winter) and rotavirus (typically during spring). Although no etiology was determined, one citywide increase in diarrhea after the August 2003 blackout was believed to have represented a true increase in diarrheal illness (26). No other citywide GI signals were linked to disease outbreaks. A total of 73 (75%) signals occurred during annual seasonal outbreaks of norovirus and rotavirus (Figure 1). No spatial signal was linked to an outbreak. However, chart reviews are often unrevealing because medical histories are only briefly documented, especially with regard to risk exposures, and laboratory workups performed during a typical ED visit are minimal, especially for GI illness. No onsite signal investigations have demonstrated a connection among cases.

On November 13, 2002, as a result of citywide diarrheal signals during October 27--28 and November 3-- 5 and citywide vomiting signals during November 7--12, a health alert was issued to physicians in the community. This alert noted that in addition to the citywide signals, stool specimens collected during October 2002, before the first citywide signal, were positive for calicivirus. The alert asked physicians to increase diagnostic testing to help DOHMH better understand these trends and prevent illness. Physicians also were asked to emphasize hand hygiene, proper cleaning of vomitus, and the need for persons to stay home when ill. A similar alert was issued on August 17, 2003, after the postblackout signal. This alert also reminded providers to advise patients to discard perishable food purchased before the blackout. In addition, a press release with this message was issued.

During February 16--17, 2004, a citywide signal occurred involving 1,803 observed cases of vomiting and 942 observed cases of diarrhea, compared with an expected 1,487 and 729 cases, respectively. DOHMH piloted having a chain of primary care clinics assist in the specimen collection. Because any citywide outbreak of diarrheal illness would likely have been detected in outpatient clinics, DOHMH sent specimen collection kits to five outpatient clinics with supplies for four children with a chief complaint of vomiting or diarrhea. Three specimens were requested per child: two rectal swabs* obtained in the clinic for ova and parasite and for culture and sensitivity testing, and a stool collection cup for viral pathogens, which was sent home with the family. Specimen transport from the patient home was arranged for viral specimens. Of the 20 distributed kits, specimens on 10 children were returned to DOHMH; however, a substantial proportion (percentages varied by test) were inadequate for testing. Three were tested for ova and parasites and were negative, nine were tested for culture and sensitivity, and five were forwarded to the New York State Department of Health's Wadsworth Laboratory for viral testing. Four tested positive for calicivirus, a norovirus. On March 24, 2004, results were received by DOHMH, 6 weeks after the citywide signal investigation began. Evidence from other outbreaks investigated at the time in schools, restaurants, and institutional settings suggested that norovirus was circulating in the community.

Review of GI Outbreak Investigations

Of 49 GI outbreaks investigated during November 15, 2001--August 15, 2004, none was detected by the ED syndromic surveillance system. In 36 outbreaks, few or no patients went to the ED, and in two outbreaks, a substantial proportion of the patients were visitors to NYC who returned to other jurisdictions before the onset of symptoms. In 11 outbreaks, patients reported to NYC EDs, but, for multiple reasons, no signal occurred. In three outbreaks, patients reported to NYC EDs not in DOHMH's system; in three outbreaks, patients reported over several days or weeks; in two outbreaks, patients reported as a group and were coded in the triage log by a group code (e.g., "school incident") that did not translate to a syndrome (27); and in two outbreaks, a combination of these problems occurred. To further understand why outbreaks might not have been detected by the syndromic surveillance system, DOHMH conducted a detailed retrospective examination of an outbreak at a grade school that was reported by traditional means by the ED physician the day it occurred. A traditional outbreak investigation indicated that the outbreak involved 150 children, 65 of whom reported to the same hospital ED in which the reporting physician worked. However, this ED was not included in the DOHMH system. Of 79 case-patients who were interviewed, all reported vomiting, and 75% reported diarrhea. One stool culture grew a norovirus.

Data were obtained from this ED for a 2-week period before and during the outbreak, and routine daily analyses were run again to explore whether the outbreak could have been detected by the system. On the first day of the outbreak, a significant ZIP code signal (six observed compared with 0.2 expected; p<0.001) was detected for diarrhea syndrome in all age groups in the ZIP code in which the school is located. Borderline clustering also was observed at two hospitals in the hospital-based analysis (29 observed compared with 16 expected; p = 0.08). As this hospital cluster was of borderline significance, it likely would not have been investigated. The significant ZIP code cluster is also unlikely to have triggered an investigation because the number (six) of excess cases was not unusual (ranked 25 of 138 spatial clusters). No signals occurred the following day. More robust signals would have been generated if a different set of analyses had been employed. For routine analyses, ED visit records are examined separately for vomiting and diarrhea; for each syndrome, all ages are analyzed together. An analysis that combined diarrhea and vomiting into one syndrome would have been more appropriate for this outbreak because certain children were coded as having diarrhea, and others were coded as having vomiting. Examining diarrhea and vomiting separately diluted the signal into two smaller signals. In addition, analyses focused on children aged 5--17 years generated a stronger series of signals on the day of this grade school outbreak (hospital diarrhea signal: seven observed, compared with one expected [p = 0.002]; ZIP code diarrhea signal: eight observed, compared with one expected [p = 0.009]) and on the following day (hospital diarrhea signal: nine observed, compared with two expected [p = 0.01]; ZIP code diarrhea signal: nine observed, compared with two expected [p = 0.007]; and ZIP code vomiting signal: 13 observed, compared with four expected [p = 0.01]) (Figure 2). However, each analysis added to the daily routine would increase the total numbers of signals observed. For example, when daily analyses were simulated for four age categories (0--4, 5--17, 18-- 59, and >60 years) and three GI syndrome categories (diarrhea and vomiting alone or in combination), an additional 296 GI signals were generated annually.

Discussion

In NYC, syndromic surveillance has proven useful primarily for detecting and monitoring annual citywide outbreaks of norovirus, rotavirus, and influenza. However the utility of this information for preventing illness is uncertain (28). For norovirus and rotavirus, the only public health intervention that can be offered is outreach to child-care and school settings regarding the importance of hand washing and of ensuring that children stay home when ill. For norovirus, the etiologic diagnosis can be difficult to make because commercial testing is not readily available, so syndromic surveillance might permit a better description of the disease epidemiology. For influenza, syndromic surveillance might allow public health officials to recognize the start of the season in a more timely manner so vaccine prevention messages can be emphasized. However, surveillance systems that also include a laboratory component are paramount to confirm that influenza has arrived and to identify circulating strains.

Syndromic surveillance systems have also provided reassurance during times of concern (e.g., the 2001 anthrax attacks) and states of elevated security alerts (e.g., during the 2004 Republican National Convention) that an excess number of patients citywide has not sought ED care for acute illnesses. Although envisioned as an early warning system, syndromic surveillance has thus far functioned more as a back-up system to traditional reporting. Constructing a syndromic surveillance system that detects statistical aberrations in the number of citywide ED visits has not been technically difficult. What has proven difficult is determining a rational, timely and resource-efficient response to signal investigation. By the time an increase in citywide ED visits is investigated by using existing methods and the etiology is determined to be either a natural or an intentional outbreak, the problem is likely to be widespread.

The NYC syndromic surveillance system originated in part from the need to perform enhanced surveillance for cryptosporidiosis because the NYC water supply is not filtered. The 1993 cryptosporidiosis outbreak in Milwaukee was detected by reports to the city health department of widespread absenteeism and substantial increases in sales of OTC antidiarrheal medications (29,30). Delay in detection of this outbreak has been attributed to multiple shortcomings of disease surveillance. Cryptosporidiosis was not a reportable disease at the time. Patients with mild symptoms, especially those who are immunocompetent, usually do not seek medical care for diarrhea, and the majority of persons affected recover without treatment. Diagnostic tests are seldom ordered for those who do seek medical care, or the diagnosis is delayed because Cryptosporidium is not considered in the differential diagnosis, and the specific test is not included in standard ova and parasite examinations (30). Considering these issues, retrospective analyses of syndromic surveillance systems, including surveillance of OTC medication sales, clinical lab submissions for any test on a stool specimens, nursing home surveillance, and ED surveillance have suggested that aberrations would have been noticeable weeks before detection of the waterborne outbreaks of cryptosporidiosis (10,30,31). Because no such waterborne outbreaks have occurred in NYC, whether such aberrations would have led to early detection and intervention cannot be determined.

Syndromic surveillance has not been useful in detecting acute localized GI outbreaks in NYC, in part because signal investigations to determine etiologic and epidemiologic links among patients are difficult and time consuming. The primary problem with using syndromic surveillance to prospectively detect outbreaks is that analyses that are sensitive enough to detect smaller outbreaks signal falsely so often that they generate too many signals from which to distinguish genuine outbreaks. Without diagnostic or epidemiologic data, whether the apparent cluster represents patients who are linked or even have the same etiologic cause for their symptoms cannot be determined easily. Multiple studies have analyzed whether ICD-9--coded discharge diagnoses yield better results for syndromic surveillance analyses (32--34), but little diagnostic work-up is performed on ED patients. Although less a problem during large-scale citywide outbreaks, misclassification obscures limited, localized signals caused by real outbreaks and can cause spurious signals composed of unrelated cases. Enhancing existing syndromic surveillance systems might improve their usefulness by increasing the specificity and positive predictive value of signal detection and investigation. Data streams containing laboratory and radiologic findings on patients could help rapidly determine if a cluster is more concerning to help prioritize signal investigations. The development of rapid, multiplex, point-of-care diagnostic assays that allow clinicians to rapidly include a natural cause such as influenza or exclude potential biologic terrorist agents would greatly improve the ability to determine whether an outbreak is occurring and its cause. Such improvements will only help if ED clinicians order the diagnostic tests; however, such tests might not be ordered for persons who have mild or moderate illness.

Health departments receive reports of disease clusters from multiple sources and set priorities regarding use of limited staff resources. If insufficient information exists to initiate an investigation, the decision is often made to observe whether the signal continues the next day, thereby losing syndromic surveillance's theoretical advantage of timeliness. Regardless of whether an investigation is begun immediately, the decision to launch a public health intervention (e.g., vaccination or antibiotic prophylaxis) requires that an etiologic diagnosis be determined. Syndromic surveillance might prove useful for detecting a problem and quantifying its magnitude but, by its very design, cannot determine the true etiology.

This evaluation is subject to at least two limitations. First, the analysis reviewed only the ED syndromic surveillance system and was focused on the system's ability to detect GI outbreaks. As localized respiratory outbreaks are reported less commonly by traditional surveillance systems, experience with fever and respiratory signals could not be evaluated. Second, other systems using different data might have a greater ability to detect outbreaks. A system using outpatient data is being piloted in NYC.

Conclusion

Concerns about biologic terrorism have generated substantial financial support for development of syndromic surveillance detection systems (28,35) at the local, state, and national levels. However, the utility of these systems has not been demonstrated. The rareness of biologic terrorism means that syndromic surveillance systems can be evaluated only by their ability to detect naturally occurring outbreaks in a timely manner. Given the increasing investments being made in syndromic surveillance, communities should examine their systems and report their findings. Critical evaluations are needed to determine whether the resources spent by public health agencies conducting signal investigations, which cannot then be used elsewhere, are worth the theoretical benefits of detecting outbreaks more quickly.

References

1. Henning KJ. What is syndromic surveillance? In: Syndromic surveillance: reports from a national conference, 2003. MMWR 2004;53 (Suppl):5--11.
2. Heffernan R, Mostashari F, Das D, Karpati A, Kulldorf M, Weiss D. Syndromic surveillance in public health practice, NYC. Emerg Infect Dis 2004;10:858--4.
3. Paladini M. Daily emergency department surveillance system---Bergen County, New Jersey. In: Syndromic surveillance: reports from a national conference, 2003. MMWR 2004;53(Suppl):47--9.
4. Wagner M, Espino J, Tsui F-C, et al. Syndrome and outbreak detection using chief-complaint data---experience of the real- time outbreak and disease surveillance project. In: Syndromic surveillance: reports from a national conference, 2003. MMWR 2004;53(Suppl):28--31.
5. Yuan CM, Love S, Wilson M. Syndromic surveillance at hospital emergency departments---Southeastern Virginia. In: Syndromic surveillance: reports from a national conference, 2003. MMWR 2004;53(Suppl):56--8.
6. Dembek ZF, Carley K, Siniscalchi A, Hadler J. Hospital admissions syndromic surveillance---Connecticut, September 2001-- November 2003. In: Syndromic surveillance: reports from a national conference, 2003. MMWR 2004;53(Suppl):50--2.
7. Lazarus R, Kleinman K, Dashevsky I, et al. Use of automated ambulatory-care encounter records for detection of acute illness clusters, including potential bioterrorism events. Emerg Infect Dis 2002;8:753--60.
8. Lewis MD, Pavlin JA, Mansfield JL, et al. Disease outbreak detection system using syndromic data in the greater Washington, DC area. Am J Prev Med 2002;23:180--6.
9. Mostashari F, Fine A, Das D, Adams J, Layton M. Use of ambulance dispatch data as an early warning system for community wide influenza-like illness, New York City. J Urban Health 2003;80(2 Suppl):i143--9.
10. Miller JR, Mikol Y. Surveillance for diarrheal disease in New York City. Urban Health 1999;76:388--90.
11. Das D, Mostashari F, Weiss D, Balter S, Heffernan R. Monitoring over-the-counter pharmacy sales for early outbreak detection---New York City, August 2001--September 2003. In: Syndromic surveillance: reports from a national conference, 2003. MMWR 2003;53(Suppl):235.
12. Rodman JS, Frost F, Davis-Burchat L, Fraser D, Langer J, Jakubowski W. Pharmaceutical sales---a method of disease surveillance? Environmental Health 1997;60:8--14.
13. Wagner M, Tsui F-C, Hogmn W, et al. National retail data monitor for public health surveillance. In: Syndromic surveillance: reports from a national conference, 2003. MMWR 2004;53(Suppl):40--2.
14. Rodman JS, Frost F, Jakubowski W. Using nurse hot line calls for disease surveillance. Emerg Infect Dis 1998;4:329--32.
15. Cooper DL, Smith G, Baker M, et al. National symptom surveillance using calls to a telephone health advice service. In: Syndromic surveillance: reports from a national conference, 2003. MMWR 2004;53(Suppl):179--83.
16. Besculides M, Heffernan R, Mostashari F, Weiss D. Evaluation of school absenteeism data for early outbreak detection--- New York City, 2001--2002 [Abstract]. In: Syndromic surveillance: reports from a national conference, 2003. MMWR 2004;53(Suppl):230.
17. CDC. Update: influenza activity---United States, January18--24, 2004. MMWR 2004;53:63--5.
18. Robertson SE, Suleiman AJM, Mehta FR, Al-Dhahry SHS, El-Bualy MS. Poliomyelitis in Oman: acute flaccid paralysis surveillance leading to early detection and rapid response to a type 3 outbreak. Bull World Health Organ 1994;72:907--14.
19. Welliver RC, Cherry JD, Boyer KM, et al. Sales of nonprescription cold remedies: a unique method of influenza surveillance. Pediat Res 1979;13:1015--7.
20. Lenaway DD, Ambler A. Evaluation of a school-based influenza surveillance system. Public Health Rep 1995;110:333--7.
21. Loonsk JW. BioSense---a national initiative for early detection and quantification of public health emergencies. In: Syndromic surveillance: reports from a national conference, 2003. MMWR 2004;53(Suppl):53--5.
22. Hutwagner L, Thompson W, Seeman M, Treadwell T. The bioterrorism preparedness and response early aberration reporting system (EARS). J Urban Health 2003;80(2 Suppl 1):i89--96.
23. Lober WB, Bryant TK, Wagner MW, et al. Roundtable on bioterrorism detection: information system--based surveillance. J Am Med Inform Assoc 2002;9:105--15.
24. Bravata DM, McDonald KM, Smith WM, et al. Systematic review: surveillance systems for early detection of bioterrorism- related diseases. Ann Int Med 2004;140:910--22.
25. Buehler JW, Berkelman RL, Hartley DM, Peters CJ. Syndromic surveillance and bioterrorism-related epidemics. Emerg Infect Dis 2003; 9:1197--204.
26. Marx MA, Rodriguez C, Greenko J, et al. Investigation of diarrheal illness detected through syndromic surveillance after a massive blackout---New York City, August 2003. Am J Public Health; In press.
27. Steiner-Sichel L, Greenko J, Heffernan R, Layton M, Weiss D. Field investigations of emergency department syndromic surveillance signals---New York City. In: Syndromic surveillance: reports from a national conference, 2003. MMWR 2004;53(Suppl):184--9.
28. Reingold A. If syndromic surveillance is the answer, what is the question? Biosecurity and Bioterrorism: Biodefense Strategy, Practice, and Science 2003;1:77--81.
29. Mac Kenzie W, Hoxie NJ, Proctor ME, et al. A massive outbreak in Milwaukee of Cryptosporidium infection transmitted through the public water supply. N Engl J Med 1994;331:161--7.
30. Proctor ME, Blair KA, Davis JP. Surveillance data for waterborne illness detection: an assessment following a massive waterborne outbreak of Cryptosporidium infection. Epidemiol Infect 1998;120:43--54.
31. Edge VL, Pollari F, Lim G, et al. Syndromic surveillance of gastrointestinal pharmacy over-the-counter sales. Can J Public Health 2004;95:446--50.
32. Begier EM, Sockwell D, Branch LM, et al. The national capitol region's emergency department syndromic surveillance system: do chief complaint and discharge diagnosis yield different results? Emerg Infect Dis 2003;9:393--6.
33. Fleischauer AT. The validity of chief complaint and discharge diagnosis in emergency department-based syndromic surveillance. Acad Emerg Med 2004;11:1262--7.
34. Ivanov O, Wagner MM, Chapmman WW, Olszewski RT. Accuracy of three classifiers of acute gastrointestinal syndrome for syndromic surveillance. Proc AMIA Symp 2002:345--9.
35. Sosin DM. Syndromic surveillance: the case for skillful investment. Biosecurity and Bioterrorism: Biodefense Strategy, Practice, and Science 2003;1:247--53.

Figure 1


Return to top.
Figure 2


Return to top.
Table


Return to top.

Use of trade names and commercial sources is for identification only and does not imply endorsement by the U.S. Department of Health and Human Services. References to non-CDC sites on the Internet are provided as a service to MMWR readers and do not constitute or imply endorsement of these organizations or their programs by CDC or the U.S. Department of Health and Human Services. CDC is not responsible for the content of pages found at these sites. URL addresses listed in MMWR were current as of the date of publication.

Disclaimer   All MMWR HTML versions of articles are electronic conversions from ASCII text into HTML. This conversion may have resulted in character translation or format errors in the HTML version. Users should not rely on this HTML document, but are referred to the electronic PDF version and/or the original MMWR paper copy for the official text, figures, and tables. An original paper copy of this issue can be obtained from the Superintendent of Documents, U.S. Government Printing Office (GPO), Washington, DC 20402-9371; telephone: (202) 512-1800. Contact GPO for current prices.

Date last reviewed: 8/5/2005