Skip Navigation LinksSkip Navigation Links
Centers for Disease Control and Prevention
Safer Healthier People
Blue White
Blue White
bottom curve
CDC Home Search Health Topics A-Z spacer spacer
Blue curve MMWR spacer

Persons using assistive technology might not be able to fully access information in this file. For assistance, please send e-mail to: Type 508 Accommodation and the title of the report in the subject line of e-mail.

Appendix. Operations Checklist

A. System-wide Issues

  • Describe the political, administrative, and geographic context for the system
  • Provide a process model that describes the data flow of the system:
    • Who inputs the data into the system
    • Who can view the data
    • Who can manipulate the data
    • Indicate where processing occurs centrally and where at distributed sites
    • Indicate where steps are automated and where manual
    • Indicate which steps are managed on-site and which can be done remotely
    • Estimate the time required for each step of the data flow
    • Indicate whether source data are produced in the course of routine workflow or specifically for the purpose of syndromic surveillance
  • Describe data and messaging standards:
    • Identify standards used to facilitate interoperability
    • Identify standards used to facilitate data sharing
    • Describe how the system interfaces with other surveillance systems from the same sites to limit reporting burden
    • Cite relevant PHIN standards and ability to meet them (
    • Provide legal documentation allowing data sharing
  • Describe procedures to maintain security:
    • Indicate security procedures employed for transmission of data between sites and for data management at the central repository
    • Describe security measures to protect data integrity at the central repository
    • Cite relevant PHIN standards and ability to meet them (
  • Describe procedures to assure privacy and confidentiality:
    • Identify the legal authority under which the surveillance activity is being conducted
    • Indicate the rules, procedures, and tools used to assure privacy and confidentiality, including methods for de-identification and re-identification, if used, and the points in the data flow where statistical disclosure limitation methods are applied

B. Data Sources

  • Describe the following:
    • Data producing facility
    • Data type
    • Data format
    • Data element definitions
    • Code sets (e.g., International Classification of Diseases (ICD) codes) used to describe the response categories
    • Data captured for geographic location (e.g., zipcode, geocode)
  • Provide a data model describing the relationship between data elements and the code sets (The architecture of the National Electronic Disease Surveillance System (NEDSS and the Public Health Conceptual Data Model ( can serve as illustrations of comprehensive data models.)
    • Indicate which data standards are used and whether they are proprietary
    • Identify the standards used for assembling data documentation (i.e., metadata)

C. Data Preprocessing

  •  Indicate the steps taken to share data between information systems and indicate the responsible organization for assuring each step (e.g., clinical facility, data clearinghouse, local health department, state health department)
  •  Indicate the frequency of data collection
  •  Indicate the volume of data (e.g., average number of records per day)
  •  Indicate how the accumulation of data is handled
  •  Describe how different data streams or data elements are assembled, subset, and manipulated to prepare them for analysis
  •  Indicate whether a relational database is formed to link datasets and the unique identifier(s) used for linkage
  •  Indicate the health-related events, syndromes, or constellation of findings under surveillance, including the derivation of the case-definitions
  •  Identify who has authority to determine the criteria for case definitions and how case criteria are applied to the data
  •  Provide a description of any algorithms used to establish the status of a potential case
  •  Indicate the frequency of editing and updating the electronic file
  •  Indicate how incomplete records are handled in analysis and reports
  •  Describe how data archiving and disposal is managed
  •  Describe how new data sources or necessary changes in data sources are identified and incorporated in the system.

D. Statistical Analysis

  • Describe how the health outcome baseline is established:
    • Describe the population under surveillance
    • Describe the source, the criteria, and the methods for establishing the background frequencies used to detect aberrations
    • How much baseline data are managed in the analysis database
  • Describe analytic methods used in automated analyses (i.e., aberration detection):
    • Describe in mathematical and statistical detail the algorithms intended to signal an event requiring further investigation
    • Describe adaptations in analytic methods to account for different outbreak patterns that might be anticipated in different data sources and types and for different outbreak scenarios
    • Indicate how reporting delays are corrected for in the analysis.
    • Describe the method of adjusting results for potential confounding factors
    • Describe how the system adapts over time and the empirical basis for modifications in the methods
  • Describe the detection process:
    • The frequency of data analysis
    • How an alarm is generated
    • Where the alarm goes
    • The type of alarms generated by the system
    • What is done to ensure that signals are not being missed
  • Describe the report generation process:
    • What routine reports are generated
    • Whether data are presented graphically or in tables
    • Whether data can be manipulated to get a specific table/chart
    • How often charts and tables are refreshed with new data
  • Indicate training level of personnel needed to manage the detection methods.

E. Epidemiologic Analysis, Interpretation, and Investigation

  • Describe the process for managing system alarms:
    • Describe the special procedures instituted when the alarm is generated (e.g., review for data errors, in-depth manual analysis of the specific conditions within the syndrome category, manual epidemiological analysis to identify subgroups responsible for an alarm, examining data from other systems, increasing the frequency of reporting from affected sites)
    • Estimate the person-hours that are devoted to review and analysis each day and the interval at which data are analyzed
    • Indicate documented procedures for managing system alarms.
    • Indicate communication method that staff is alerted of alarms (e.g., whether they get paged at home, receive an automated e-mail, etc.?)
    • Indicate the expectations and schedule of staff to actively check the system and schedule, including nights and weekends
    • Indicate the response options to an alarm and the factors that influence the choice (e.g., wait for an alarm in another system, initiate an onsite investigation, alert clinicians to gather information)
  • Describe the process for identifying cases for investigation when the data analyzed routinely are unidentified
  • Describe how independent data types are integrated in the analysis for improved decision making
  • Describe the rules, procedures, and tools for communication
    • Indicate the mechanisms used and content guidance provided for sharing results with 1) reporting sources, 2) response community, and 3) the public;
    • Describe how decisions are made for sending urgent communications and the methods for sending urgent communications
    • Indicate whether receipt of a communication is acknowledged and how unacknowledged receipt is managed
    • Indicate how often urgent communications and routine reports are sent
  • Describe the protocol for conducting surveillance during outbreak management, if one exists
    • Indicate how often data will be updated and analyzed
    • Describe how the system can be modified or customized to meet special data needs
  • Describe how the system will monitor the impact of prevention and control measures
    • Describe how and how often system components are tested for operational readiness (e.g., `spiked' data or modeling exercises)

Use of trade names and commercial sources is for identification only and does not imply endorsement by the U.S. Department of Health and Human Services.

References to non-CDC sites on the Internet are provided as a service to MMWR readers and do not constitute or imply endorsement of these organizations or their programs by CDC or the U.S. Department of Health and Human Services. CDC is not responsible for the content of pages found at these sites. URL addresses listed in MMWR were current as of the date of publication.

Disclaimer   All MMWR HTML versions of articles are electronic conversions from ASCII text into HTML. This conversion may have resulted in character translation or format errors in the HTML version. Users should not rely on this HTML document, but are referred to the electronic PDF version and/or the original MMWR paper copy for the official text, figures, and tables. An original paper copy of this issue can be obtained from the Superintendent of Documents, U.S. Government Printing Office (GPO), Washington, DC 20402-9371; telephone: (202) 512-1800. Contact GPO for current prices.

**Questions or messages regarding errors in formatting should be addressed to

Page converted: 5/5/2004


Safer, Healthier People

Morbidity and Mortality Weekly Report
Centers for Disease Control and Prevention
1600 Clifton Rd, MailStop E-90, Atlanta, GA 30333, U.S.A


Department of Health
and Human Services

This page last reviewed 5/5/2004