OutbreakNet Enhanced Metrics

Print
Related Pages

OutbreakNet Enhanced sites incorporate performance metrics and model practices into their capacity building plans. OutbreakNet Enhanced performance metrics are a subset of FoodCORE and Council to Improve Foodborne Outbreak Response (CIFOR) measures used to document the burden, timeliness, and completeness of enteric disease surveillance and response. Performance metrics can also be used to identify areas for process improvements.

Metrics are reported separately for Salmonella, Shiga toxin-producing E. coli, Listeria, Shigella (optional), and Campylobacter (optional). Collectively, these pathogens are abbreviated as SSL(SC)

Metrics reporting time frame = 1 calendar year (Jan 1 – Dec 31)

On This Page

Isolate/Specimen-based Metrics

Rationale: The intent of these metrics is to evaluate the timeliness and completeness/availability of laboratory surveillance and subtyping data for Salmonella, Shiga toxin-producing Escherichia coli (STEC), Listeria, Shigella, and Campylobacter. These metrics can be used to determine if there are gaps in the laboratory isolate handling and testing processes. If gaps are identified, knowing the detailed circumstances around the gap will help develop targeted actions to address them specifically.

  1. OPTIONAL Total number of SSL(SC) isolates and isolate-yielding specimens submitted to or recovered at the Public Health Lab (PHL)
    • Intent: To allow evaluation of the burden of isolate submissions and testing at the PHL.
    • Note: This is the total number, not just primary isolates.
    • Note: This measure is optional as it can also be collected through the ELC Performance Measures
  2. Number of primary isolates and primary isolate-yielding specimens submitted to the PHL
    • Intent: To allow evaluation of laboratory testing associated with the first or representative isolate or sample for each clinical case non-human testing unit versus duplicate isolates from repeat testing or sampling protocols.
    • Note: If multiple samples are tested from the same source, the primary isolates from each of the samples for which you completed testing can be counted.
  1. Total number of preliminary positive clinical specimens or samples (i.e. positive result from CIDT or other preliminary testing) received at PHL as a specimen or sample (regardless of if isolate-yielding or not)
    • Intent: To allow evaluation of the submissions of preliminary positive clinical specimens (i.e. CIDTs) to the PHL.
  2. Total number of preliminary positive clinical specimens or samples received at the PHL that yielded isolates
    • Intent: To allow evaluation of the outcome of clinical specimen testing and to calculate the percent of isolate-yielding clinical specimens or samples received at PHL.
    • Note: Could be used to identify gaps in submission protocols if a high proportion of specimens or samples are not viable; this also indicates the utility of testing multiple clinical specimens to try to identify cases that might otherwise be missed if only using isolate submissions.
    • Note: This is a sub-measure of 2a and calculated with 2a as the denominator.
    • Example: 100 CIDT positive specimens or samples are received by your lab (not as isolates) and PHL is able to culture an isolate from 75; 2a = 100 2b =75 (75%)
  1. Percent of SSL(SC) primary isolates with WGS testing at PHL
    • Intent: To allow evaluation of the completeness of WGS subtyping of SSL(SC) isolates.
  1. Time from SSL(SC) isolate receipt (or recovery) or WGS request at PHL to sequence upload to PulseNet
    • Intent: To allow evaluation of the timeliness of WGS subtyping (e.g., sequencing) at the PHL.
    • Note: Time is measured in median days, measurements will exclude weekend days. For laboratory time measurements, only isolates tested at the PHL should be included.
    • Note: To reduce duplication, this measure is collected through the ELC Performance Measures. This measure will be included in annual summaries.
Top of Page

Case-based Metrics

Rationale: The intent of these metrics is to evaluate the timeliness and completeness/availability of epidemiologic data for reported cases. These metrics can be used to determine if there are gaps in the epidemiologic interviewing process. If gaps are identified, knowing the detailed circumstances around the gap will help develop targeted actions to address them specifically.

  1. Total number of SSL(SC) cases reported to epidemiology staff
    1. OPTIONAL Percent of SSL(SC) cases with isolates or specimens submitted to the PHL
  2. Total number of confirmed SSL(SC) cases reported to epidemiology staff
  3. Total number of probable SSL(SC) cases reported to epidemiology staff
  4. Total number of suspect SSL(SC) cases reported to epidemiology staff
    • Intent: To allow evaluation of the burden of cases reported to epidemiology staff.
    • Note: The number of confirmed cases reported to epidemiology staff may not be equivalent to laboratory isolate counts because of duplicate isolates submitted to the PHL, and/or because cases may be reported to epidemiology staff from outside the PHL in their health department (i.e., laboratory or clinical report of cases not submitted to the PHL).
    • Note: Definitions for confirmed, probable, and suspect cases will follow CSTE/NNDSS case definitions for each pathogen (https://ndc.services.cdc.gov/).
  1. Percent of SSL(SC) cases with attempted interview
    1. Percent of confirmed cases with attempted interview
    2. Percent of probable cases with attempted interview
    3. Percent of suspect cases with attempted interview
    4. Intent: To allow evaluation of interviewing capacity to try to reach cases.
    5. Note: To be based off #5 for calculation.
    6. Note: Definitions for confirmed, probable, and suspect cases will follow CSTE/NNDSS case definitions for each pathogen (https://ndc.services.cdc.gov/).
  2. Time from SSL(SC) case report to initial interview attempt
    • Intent: To allow evaluation of turnaround time for attempted interviews. Attempted interviews were used here because factors that impact completion of an interview may be outside the control of epidemiology staff (e.g., case refuses), whereas making an attempt to interview is a turnaround time that can be controlled through data exchange and capacity. Time is measured in median days; measurements will exclude weekend days. This metric should include all confirmed, probable, and suspect cases with attempted interviews.
  3. Percent of SSL(SC) cases with exposure history obtained
    • Intent: To evaluate proportion of cases with assessment of exposures prior to onset of illness.
    • Note: To be based off #6a for calculations. The intended evaluation could be made using either #5 or #6a, but using #6a has the added benefit of allowing assessment of how many attempted interviews are successful in obtaining exposure history.
      1. OPTIONAL Among SSL(SC) cases with an exposure history, percent with full shotgun or case exposure obtained
        • Intent: To evaluate the completeness of conducted interviews—e., how many interviews may have been a shorter interview and how many had collection of a full shotgun or exposure assessment.
        • Note: This is a sub-measure of #6c.
      2. OPTIONAL Among cases with an exposure history, time from case report to when an exposure history is obtained
        • Intent: To allow evaluation of turnaround time for obtaining exposure histories.
        • Note: This is a sub-metric for cases where an exposure history was obtained. Cases lost to follow-up and cases where no exposure history was obtained should not be included. Time is measured in median days; measurements will exclude weekend days.
Top of Page

Investigation-based Metrics

Rationale: The intent of these metrics is to evaluate epidemiologic activity related to cluster and outbreak monitoring, evaluation, and investigation. These metrics can be used to determine if there are gaps in cluster and outbreak investigation. If gaps are identified, knowing the detailed circumstances around the gap will help develop targeted actions to address them specifically.

  • Intent: To allow evaluation of the burden of clusters, outbreaks, and investigational needs.
  1. OPTIONAL Total number and percent of SSL(SC) cluster and outbreak investigations with routine interview of cases
    • Intent: To allow evaluation of completeness of cluster and outbreak investigation response activities.
    • Note: This metric would indicate that initial interviews were conducted with a case(s) in your jurisdiction.
  2. Total number and percent of SSL(SC) cluster and outbreak investigations where an analytic epidemiologic study was conducted
    • Intent: To allow the evaluation of conducting or participating in analytic epidemiologic investigations.
    • Note: This metric would indicate that your jurisdiction was responsible for (i.e., led) or participated in analytic hypothesis testing. There may be clusters that do not warrant analytic epidemiologic investigation based on the hypothesis generating data.
  1. Total number and percent of SSL(SC) cluster and outbreak investigations with confirmed vehicle/source identified
  2. OPTIONAL Total number and percent of SSL(SC) cluster and outbreak investigations with suspect vehicle/source identified
    • Intent: To allow the evaluation of how often cluster and outbreak investigations result in identifying confirmed and suspect vehicles/sources. These investigations can still contribute to the body of knowledge of risky foods, practices, or other gaps in the food safety system in order to inform prevention efforts.
    • Note: There is not always a relationship between the completeness and/or timeliness of an investigation and identification of a confirmed vehicle/source.
  • Intent: To allow the evaluation of how often food or environmental samples are collected for testing within your jurisdiction as part of a cluster or outbreak investigation.
  • Note: Not all investigations will yield evidence that support taking this kind of action.
  • Intent: To allow the evaluation of how often environmental health, agriculture, regulatory, or food safety program staff within your jurisdiction were engaged in cluster and outbreak investigation activities.
  • Note: Not all investigations will yield evidence that supports taking this kind of action. Additionally, contacting partners during an investigation does not necessarily imply that a regulatory action would be indicated or taken.
Top of Page

Outbreak-based Metrics

Rationale: The intent of these metrics is to evaluate outbreak reporting activity. These metrics can be used to determine if there are gaps in outbreak reporting. If gaps are identified, knowing the detailed circumstances around the gap will help develop targeted actions to address them specifically.

  1. Total number of SSL(SC) outbreaks, excluding multi-state outbreaks
  2. OPTIONAL Total number and percent of SSL(SC) outbreaks (excluding multi-state outbreaks) reported/submitted to National Outbreak Reporting System (NORS)
    1. Intent: To determine the burden and completeness of outbreak reporting through NORS.
    2. Note: It is understood that this value may not be 100% during specific reporting periods if an outbreak investigation is ongoing and therefore not ready to be submitted to NORS.
Top of Page
Definitions

Listed below are terms and definitions associated with the OutbreakNet Enhanced metrics.  Terms are listed in the order that they appear in the metrics. These definitions were adopted from the FoodCORE core definitions.

All measurements of time are in median days: Measurements will exclude weekend days. For laboratory time measurements, only isolates subtyped at the PHL should be included.

Isolate/Isolate-yielding specimen: This will include all SSL(SC) isolates (human, food, environmental, etc.) submitted to PHL and isolates recovered from specimen submitted to PHL. This can be further broken down to report total number of each category of isolates (human, food, environmental, etc.).

Primary isolate/isolate-yielding specimen: The first or representative SSL(SC) isolate/isolate-yielding specimen for each clinical case, or testing unit for any non-human isolates (i.e. isolates that are intended to be part of a subtyping workflow, such as WGS). Any samples that were collected but not processed or tested can be de-duplicated and not counted as a primary isolate. If multiple samples are from the same source, the primary isolates from each of the samples for which you completed testing can be counted.

Preliminary positive sample/specimen: A preliminary positive sample/specimen includes any submission received by your public health lab with a positive result from CIDT or other preliminary testing, whether or not it was then confirmed at the public health lab.

Confirmed SSL(SC) cases: Definitions for confirmed, probable, and suspect cases will follow CSTE/NNDSS case definitions for each pathogen (https://ndc.services.cdc.gov/).

Exposure history: To include an interview (of any format) that assesses exposures prior to onset of illness, via an open-ended exposure history, or via a list of potential exposures. The key factor to be considered an exposure history is an interview that goes beyond assessment of high-risk settings and prevention education to ascertain food consumption/preference, or other exposure data.

Cluster: Two or more cases of infection where the number of cases meets one or more of the following criteria:

  1. The number of isolates/cases represents an increase over the expected baseline (as determined by PulseNet definition or other subtyping criteria)
  2. Demographic or other epidemiologic characteristics among cases indicate some deviation from expected values for the region (and are consistent with subtyping information)
  3. There is a non-human isolate that would indicate a potential source of human infections
  4. In the absence of meeting the above criteria in a catchment area, a case-patient should be considered cluster-associated if the above criteria are met when including isolates from other jurisdictions or catchment areas.

In the absence of meeting any of the above criteria, ill persons should be considered cluster-associated if there are demographic or epidemiologic indications of a common source of infection even without laboratory subtyping data to link cases.

This definition also includes clusters that may be defined as outbreaks in your jurisdiction.

Outbreak: Definition for an outbreak will follow the NORS definition (https://www.cdc.gov/nors/forms.html).

Investigation: Any active epidemiologic follow-up resulting from the identification of a cluster. This could be initiating contact with a case (or the public health authority under whose jurisdiction a case falls) to ascertain direct case-based epidemiologic data, or active review of previously collected case-based data for cases later identified as cluster or outbreak-associated.

Analytic epidemiologic study: A systematic, statistical analysis against a comparison group or within a cohort to test a hypothesis.

Confirmed vehicle/source clusters: Clusters of infection where the etiologic agent has either been cultured from the vehicle or the vehicle has been statistically implicated in an analytic study.

Suspect vehicle/source clusters: Clusters of infection where investigational and/or laboratory data indicate a likely source/vehicle of infection without confirmation. For example: vehicle is a known risk factor, established errors in food preparation, or reported consumption by a high proportion of cluster-associated cases.

Top of Page