Persons using assistive technology might not be able to fully access information in this file. For assistance, please send e-mail to: mmwrq@cdc.gov. Type 508 Accommodation and the title of the report in the subject line of e-mail.
Appendix. Operations Checklist
A. System-wide Issues
Describe the political, administrative, and geographic context for the system
Provide a process model that describes the data flow of the system:
Who inputs the data into the system
Who can view the data
Who can manipulate the data
Indicate where processing occurs centrally and where at distributed sites
Indicate where steps are automated and where manual
Indicate which steps are managed on-site and which can be done remotely
Estimate the time required for each step of the data flow
Indicate whether source data are produced in the course of routine workflow or specifically for the purpose
of syndromic surveillance
Describe data and messaging standards:
Identify standards used to facilitate interoperability
Identify standards used to facilitate data sharing
Describe how the system interfaces with other
surveillance systems from the same sites to limit reporting burden
Describe procedures to assure privacy and confidentiality:
Identify the legal authority under which the
surveillance activity is being conducted
Indicate the rules, procedures, and tools used to
assure privacy and confidentiality, including methods for
de-identification and re-identification, if used, and the points in the data flow where statistical disclosure limitation methods are applied
B. Data Sources
Describe the following:
Data producing facility
Data type
Data format
Data element definitions
Code sets (e.g., International Classification of Diseases (ICD) codes) used to describe the
response categories
Data captured for geographic location (e.g., zipcode, geocode)
Provide a data model describing the relationship
between data elements and the code sets (The architecture of
the National Electronic Disease Surveillance System (NEDSS
http://www.cdc.gov/nedss/) and the Public Health Conceptual Data Model (http://www.cdc.gov/nedss/DataModels/phcdm.pdf) can serve as illustrations of comprehensive data models.)
Indicate which data standards are used and whether they are proprietary
Identify the standards used for assembling data documentation (i.e., metadata)
C. Data Preprocessing
Indicate the steps taken to share data between information systems and indicate the responsible organization
for assuring each step (e.g., clinical facility, data clearinghouse, local health department, state health department)
Indicate the frequency of data collection
Indicate the volume of data (e.g., average number of records per day)
Indicate how the accumulation of data is handled
Describe how different data streams or data elements are assembled, subset, and manipulated to prepare them
for analysis
Indicate whether a relational database is formed to
link datasets and the unique identifier(s) used for linkage
Indicate the health-related events, syndromes, or constellation of findings under surveillance, including the
derivation of the case-definitions
Identify who has authority to determine the criteria for case definitions and how case criteria are applied to the data
Provide a description of any algorithms used to
establish the status of a potential case
Indicate the frequency of editing and updating the electronic file
Indicate how incomplete records are handled in analysis and reports
Describe how data archiving and disposal is managed
Describe how new data sources or necessary changes in data sources are identified and incorporated in the system.
D. Statistical Analysis
Describe how the health outcome baseline is established:
Describe the population under surveillance
Describe the source, the criteria, and the methods for establishing the background frequencies used to
detect aberrations
How much baseline data are managed in the analysis database
Describe analytic methods used in automated analyses (i.e., aberration detection):
Describe in mathematical and statistical detail the algorithms intended to signal an event requiring
further investigation
Describe adaptations in analytic methods to account for different outbreak patterns that might be anticipated
in different data sources and types and for different outbreak scenarios
Indicate how reporting delays are corrected for in the analysis.
Describe the method of adjusting results for
potential confounding factors
Describe how the system adapts over time and
the empirical basis for modifications in the methods
Describe the detection process:
The frequency of data analysis
How an alarm is generated
Where the alarm goes
The type of alarms generated by the system
What is done to ensure that signals are not being missed
Describe the report generation process:
What routine reports are generated
Whether data are presented graphically or in tables
Whether data can be manipulated to get a
specific table/chart
How often charts and tables are refreshed with new data
Indicate training level of personnel needed to
manage the detection methods.
E. Epidemiologic Analysis, Interpretation, and Investigation
Describe the process for managing system alarms:
Describe the special procedures instituted when the alarm is generated (e.g., review for data
errors, in-depth manual analysis of the specific conditions within the syndrome category, manual
epidemiological analysis to identify subgroups responsible
for an alarm, examining data from other systems, increasing the frequency
of reporting from affected sites)
Estimate the person-hours that are devoted to review and analysis each day and the interval at which data
are analyzed
Indicate documented procedures for managing system alarms.
Indicate communication method that staff is alerted of alarms (e.g., whether they get paged at home, receive
an automated e-mail, etc.?)
Indicate the expectations and schedule of staff to actively check the system and schedule,
including nights and weekends
Indicate the response options to an alarm and the factors that influence the choice (e.g., wait for an alarm
in another system, initiate an onsite investigation, alert clinicians to gather information)
Describe the process for identifying cases for investigation when the data analyzed routinely are unidentified
Describe how independent data types are integrated in the analysis for improved decision making
Describe the rules, procedures, and tools for
communication
Indicate the mechanisms used and content guidance provided for sharing results with 1) reporting sources, 2)
response community, and 3) the public;
Describe how decisions are made for sending urgent communications and the methods for sending
urgent communications
Indicate whether receipt of a communication is acknowledged and how unacknowledged receipt is managed
Indicate how often urgent communications and routine reports are sent
Describe the protocol for conducting surveillance during outbreak management, if one exists
Indicate how often data will be updated and
analyzed
Describe how the system can be modified or
customized to meet special data needs
Describe how the system will monitor the impact of prevention and control measures
Describe how and how often system components are tested for operational readiness (e.g., `spiked' data
or modeling exercises)
Use of trade names and commercial sources is for identification only and does not imply endorsement by the U.S. Department of
Health and Human Services.References to non-CDC sites on the Internet are
provided as a service to MMWR readers and do not constitute or imply
endorsement of these organizations or their programs by CDC or the U.S.
Department of Health and Human Services. CDC is not responsible for the content
of pages found at these sites. URL addresses listed in MMWR were current as of
the date of publication.
Disclaimer
All MMWR HTML versions of articles are electronic conversions from ASCII text
into HTML. This conversion may have resulted in character translation or format errors in the HTML version.
Users should not rely on this HTML document, but are referred to the electronic PDF version and/or
the original MMWR paper copy for the official text, figures, and tables.
An original paper copy of this issue can be obtained from the Superintendent of Documents,
U.S. Government Printing Office (GPO), Washington, DC 20402-9371; telephone: (202) 512-1800.
Contact GPO for current prices.
**Questions or messages regarding errors in formatting should be addressed to
mmwrq@cdc.gov.
