FoodCORE NOU Metrics

NOU metrics apply to Norovirus, other enteric disease pathogens, such as Cryptosporidium or Giardia, and outbreaks of unknown etiology. Collectively, these are referred to as the NOU metrics, for norovirus, other etiologies, and unknown etiologies. Other etiologies are enteric illnesses with determined etiology that are not Salmonella, Shiga toxin-producing Escherichia coli, Listeria, Shigella, Campylobacter, or norovirus. Unknown etiologies are enteric illness with no determined/identified etiology from case, product, or environmental testing to indicate the etiologic agent. This can be because no specimen or sample yielded an isolate or other positive result, and would also include investigations where no specimens or samples were collected.

The FoodCORE performance metrics are a list of measurable activities covering diverse aspects of outbreak response. These activities span from outbreak surveillance and detection through investigation, response, control, and prevention measures. Using the metrics, each center provides data about the burden, timeliness, and completeness of foodborne disease activities related to the key areas of activity. Data for all years of the FoodCORE program are available.

NOU Metrics

  1. Total number of investigations1
    • Intent: To allow the evaluation of the burden of clusters/outbreaks and investigational needs.
    • Note: This metric includes all investigations; it is not limited to foodborne or point-source but includes person-to-person investigations as well. This metric will be used to calculate #2a.1, #2a.2, #10.
  2. Total number of foodborne or point-source investigations1
    • Intent: To allow the evaluation of the burden of clusters and investigational needs.
    • Note: This metric includes investigations where the primary mode is determined to be foodborne or point-source (i.e. waterborne, zoonotic, environmental, or indeterminate/other/unknown) at the conclusion of the investigation. This metric will be used to calculate #3a, #3b, #4, #5, #6a-e, #7, #8, #9.
  3. Total number of person-to-person investigations1
    • Intent: To allow the evaluation of the burden of clusters and investigational needs.
    • Note: This metric includes investigations where the primary mode is determined to be person-to-person at the conclusion of the investigation.

Laboratory-based Metrics

Rationale: The intent of these metrics is to evaluate the timeliness and completeness/availability of laboratory surveillance and subtyping data for norovirus, other etiology, and unknown etiology investigations. These metrics include all NOU investigations; they are not limited to foodborne or point-source, but include person-to-person NOU investigations as well. These metrics can be used to determine if there are gaps in the diagnostic specimen handling and testing processes. If gaps are identified, knowing the detailed circumstances around the gap will help develop targeted actions to address them specifically.

Note: These laboratory-based metrics are for laboratory testing conducted at the reporting public health laboratory. For laboratories that are not equipped to perform a certain type of testing (e.g., testing for certain viruses or parasites), the measures related to those tests would not be applicable since the completeness and timeliness of the testing would not be within the control of the reporting laboratory.

  1. Measures of investigations with clinical specimens collected and submitted to lab
    1. Number and percent of all investigations with clinical specimens collected and submitted to any laboratory (including clinical labs)
      • Intent: To allow the evaluation of the burden and completeness of specimen submissions to the PHL that represent possible clusters or outbreaks of infection.
      • Note: To be based on #1a for calculation.
    2. Number and percent of all investigations with clinical specimens collected and submitted to the Public Health Lab (PHL)1
      • Intent: To allow the evaluation of the burden and completeness of specimen submissions to the PHL that represent possible clusters or outbreaks of infection.
      • Note: To be based on #1a for calculation. This metric will be used to calculate #2b, #2c, #2d.
  2. Number and percent of all investigations where submitted clinical specimens were tested for GI viruses at the PHL1
    • Intent: To allow the evaluation of the burden and completeness of viral specimen testing at the PHL.
    • Note: To be based on #2a.2 for calculation. This metric will be used to calculate #2b.1, #2b.2
    1. Number and percent of all investigations where GI-viral testing of specimens included testing for norovirus by real time RT-PCR at the PHL1
      • Intent: To allow the evaluation of the burden and completeness of testing specimens for the detection of norovirus at the PHL.
      • Note: This is a sub-measure to be based on #2b for calculation. This metric will be used to calculate #2b.1.1.
      1. Number and percent of all investigations with two or more specimens confirmed positive for norovirus by conventional RT-PCR at the PHL where at least one specimen was sequenced and uploaded to CaliciNet1
        • Intent: To allow the evaluation of the burden and completeness of sequencing for norovirus positive specimens.
        • Note: This is a sub-measure to be based on #2b.1 for calculation where the denominator is the number of NOU investigations in #2b.1 with at least two positive specimens based on RT-PCR.
        1. Median days from the first norovirus detection at the PHL via conventional RT-PCR to upload of sequence to CaliciNet1
          • Intent: To allow the evaluation of the timeliness of norovirus sequencing at the PHL.
          • Note: This is a sub-measure to be based on #2b.1.1 for calculation. Time is measured in median days, measurements will exclude weekend days. For laboratory time measurements, only isolates tested at the PHL should be included.
    2. Number and percent of all investigations where GI viral testing of specimens included testing for other viruses (e.g., sapovirus, astrovirus, rotavirus, adenovirus) at the PHL
      • Intent: To allow the evaluation of the burden and completeness of testing specimens for viruses other than norovirus (e.g., sapovirus, astrovirus, rotavirus, adenovirus) at the PHL.
      • Note: To be based on #2b for calculation.
  3. Number and percent of all investigations where clinical specimens were tested for pathogenic bacteria or their toxins, antigens, or specific antibodies at the PHL1
    • Intent: To allow the evaluation of the burden, breadth, and completeness of testing clinical specimens for pathogen identification.
    • Note: To be based on #2a.2 for calculation. This metric will be used to calculate #2c.1, #2c.2
    1. Number and percent of all investigations where bacterial testing of specimens included culture-based diagnostics at the PHL
      • Intent: To allow the evaluation of the burden, breadth, and completeness of culture-based testing of clinical specimens to confirm pathogen identification.
      • Note: To be based off #2c for calculation.
    2. Number and percent of all investigations where bacterial testing of specimens included non-culture-based diagnostics at the PHL
      • Intent: To allow the evaluation of the burden, breadth, and completeness of non-culture-based testing of clinical specimens to confirm pathogen identification.
      • Note: To be based off #2c for calculation.
  4. Number and percent of all investigations where clinical specimens were tested for parasites at the PHL1
    • Intent: To allow the evaluation of the burden, breadth, and completeness of testing clinical specimens for pathogen identification.
    • Note: To be based on #2a.2 for calculation.

Investigation-based Metrics

Rationale: The intent of these metrics is to evaluate response activity related to investigations. These metrics can be used to determine if there are gaps in investigational activities. If gaps are identified, knowing the detailed circumstances around the gap will help develop targeted actions to address them specifically.

  1. Number and percent of foodborne or point-source investigations with exposure assessments conducted1
    • Intent: To allow the evaluation of completeness of foodborne or point-source NOU investigation activities.
    • Note: This metric would indicate that initial interviews or exposure assessments were conducted with a case(s) in your jurisdiction. To be based on #1b for calculation.
  2. Number and percent of foodborne or point-source investigations where an analytic epidemiologic study was conducted1
    • Intent: To allow the evaluation of conducting or participating in analytic epidemiologic investigations.
    • Note: This metric would indicate that your jurisdiction was responsible for (i.e., led) or participated in analytic hypothesis testing. There may be investigations that do not warrant analytic epidemiologic studies based on hypothesis generating data. To be based on #1b for calculation.
  • Intent: To allow the evaluation of how often investigations result in identifying suspect vehicles or sources. The evaluation of suspect vehicles or sources is important because even without a confirmed source, these investigations can still contribute to the body of knowledge of risky foods, practices, or other gaps in the food safety system in order to inform prevention efforts.
  • Note: There is not always a relationship between the completeness and/or timeliness of an investigation and identification of a suspect vehicle/source. To be based on #1b for calculation.
  • Intent: To allow the evaluation of how often investigations result in identifying confirmed vehicles or sources. These investigations contribute to the body of knowledge of risky foods, practices, or other gaps in the food safety system in order to inform prevention efforts.
  • Note: There is not always a relationship between the completeness and/or timeliness of an investigation and identification of a confirmed vehicle/source. To be based on #1b for calculation.
  1. Number and percent of foodborne or point-source investigations with exclusion of an ill person(s) from a high-risk setting
    • Intent: To allow the evaluation of excluding an ill person(s) within your jurisdiction to help minimize the risk to others and mitigate ongoing transmission.
    • Note: High-risk settings may include, but are not limited to food handling, daycare attendance, or health care work. Not all investigations will yield evidence that support taking this kind of action. To be based on #1b for calculation.
  2. Number and percent of foodborne or point-source investigations with remediation or closure of an establishment linked to illness
    • Intent: To allow the evaluation of requiring remediation of an identified gap in food safety or even closure of an establishment within your jurisdiction to help minimize the risk to others and mitigate ongoing transmission.
    • Note: Not all investigations will yield evidence that support taking this kind of action. To be based on #1b for calculation.
  3. Number and percent of foodborne or point-source investigations with educations campaigns during outbreaks (beyond individual case education)
    • Intent: To allow the evaluation of conducting an educational campaign within your jurisdiction for at risk groups to help minimize the risk to others and mitigate ongoing transmission.
    • Note: Educational campaigns, beyond individual case education, may include but are not limited to hand washing education in a classroom or daycare or safe food handling and preparation practices. Not all investigations will yield evidence that support taking this kind of action. To be based on #1b for calculation.
  4. Number and percent of foodborne or point-source investigations with media or public messaging (web updates, press release, etc.)
    • Intent: To allow the evaluation of notifying the public about an NOU investigation to help minimize the risk to others and mitigate ongoing transmission. This applies to notifications that occurred within your jurisdiction, or that your jurisdiction participated in (e.g., confirmed or contributed information to).
    • Note: Media or public messaging includes but is not limited to web updates or press releases of materials that would be available beyond the population directly impacted by a cluster or outbreak. Not all investigations will yield evidence that support taking this kind of action. To be based on #1b for calculation.
  5. Number and percent of foodborne or point-source investigations with regulatory action (recall, hold, etc.)
    • Intent: To allow the evaluation of taking a regulatory action to prevent initial or further distribution of a product associated with illness or risk of illness. This applies to regulatory actions that occurred within your jurisdiction, or that your jurisdiction participated in (e.g., confirmed or contributed information to). To be based on #1b for calculation.
    • Note: Regulatory action includes but is not limited to product recalls, holding product from distribution, or initiating other restrictions of sale or production. Not all investigations will yield evidence that support taking this kind of action. To be based on #1b for calculation.
  • Intent: To allow the evaluation of how often environmental health assessments are conducted within your jurisdiction as part of an NOU investigation.
  • Note: Not all investigations will yield evidence that support taking this kind of action. To be based on #1b for calculation.
  • Intent: To allow the evaluation of how often food or environmental samples are collected for testing within your jurisdiction as part of an NOU investigation.
  • Note: Not all investigations will yield evidence that support taking this kind of action. To be based on #1b for calculation.
  • Intent: To allow the evaluation of how often environmental health, agriculture, regulatory, or food safety program staff within your jurisdiction were engaged in NOU investigation activities.
  • Note: Not all investigations will yield evidence that support taking this kind of action. Additionally, contacting partners during an investigation does not necessarily imply that a regulatory action would be indicated or taken. To be based on #1b for calculation.

Outbreak-based Metrics

Rationale: The intent of these metrics is to evaluate outbreak reporting activity. These metrics can be used to determine if there are gaps in outbreak reporting. If gaps are identified, knowing the detailed circumstances around the gap will help develop targeted actions to address them specifically.

  • Intent: To determine the burden and completeness of outbreak reporting through NORS.
  • Note: It is understood that this value may not be 100% during specific reporting periods if an outbreak investigation is ongoing and therefore not ready to be submitted to NORS. To be based on #1a for calculation.

NOTE: 1Minimum reporting requirement for FoodCORE Centers

Definitions

Other Etiology: Enteric illness with determined etiology that is not Salmonella, Shiga toxin-producing Escherichia coli, Listeria, Shigella, Campylobacter, or norovirus.

Unknown Etiology: Enteric illness with no determined/identified etiology from any of the case, product, or environmental testing to indicate the etiologic agent. This can be because no specimen or sample yielded an isolate or other positive result, and would also include investigations where no specimens or samples were collected.

Foodborne or Point-source Investigation: Epidemiologic activity in response to cases where there is indication of non-person-to-person transmission of a similar enteric illness. This includes investigations where there are demographic or epidemiologic indications of a common, point-source of infection even without laboratory subtyping data to link cases.

Specimens: This will include all clinical specimens submitted to PHL.

Samples: This will include all non-clinical, samples (food, environmental, etc.) submitted to PHL.

Exposure assessment: To include an interview (of any format) that assesses exposures prior to onset of illness, via an open-ended exposure history, or via a list of potential exposures. The key factor to be considered an exposure history is an interview that goes beyond assessment of high-risk settings and prevention education to ascertain food consumption/preference, or other exposure data.

Analytic epidemiologic study: A systematic, statistical analysis against a comparison group or within a cohort to test a hypothesis

Suspect norovirus or other etiology outbreak: If there are not sufficient specimens or samples to classify an outbreak as a confirmed norovirus or other etiology, but the available data (testing, epidemiologic data such as signs, symptoms, and incubation period) indicate an etiologic agent, an investigation should still be classified under norovirus or other etiology, as appropriate, for metrics reporting.

Confirmed norovirus outbreak: Outbreak with two or more clinical specimens positive for norovirus by RT-PCR, EM, or EIA.

Confirmed other etiology outbreak: For most etiologic agents, CDC considers an outbreak to have a confirmed etiology if there are two or more lab-confirmed cases. However, in botulism, marine toxins, and other chemical outbreaks, the definition of a confirmed etiology is not as strict. Since botulism, marine toxins, and other chemical outbreaks have such distinct clinical symptoms, a physician’s diagnosis is often sufficient and laboratory confirmation is not necessary. Therefore, for such outbreaks, CDC would consider it a confirmed etiology outbreak if there is at least 1 probable case (based on clinical symptoms). (See also: Other Etiology definition above)

All measurements of time are in median days: Measurements will exclude weekend days. For laboratory time measurements, only isolates subtyped at the PHL should be included.

Vehicle/source Identified:

SUSPECT vehicle/source investigations: Investigations of infection where investigational and/or laboratory data indicate a likely vehicle/source of infection without confirmation: vehicle/source is a known risk factor, established errors in food preparation, or reported consumption by a high proportion of cluster-associated cases.

CONFIRMED vehicle/source investigations: Investigations of infection where the etiologic agent has either been identified from the vehicle/source, or the vehicle/source has been statistically implicated in an analytic study.  

Control measure: to include interventions such as exclusion of an ill person(s) from high risk setting, remediation or closure of an establishment linked to illness, educational campaigns during daycare outbreaks, etc. To be considered a control measure, activities should extend beyond the routine educational component of an interview or exposure assessment.

Public health action: to include media, public messaging (web updates, press release, etc.), or regulatory action (recall, hold, etc.). To be considered a public health action, activities should extend beyond the routine investigation activities and reach at-risk individuals beyond identified cases. A public health action should be included in the metrics if the FoodCORE Center was directly involved in the action, or is aware that a public health action was taken during a multijurisdictional investigation. For example, if CDC produces public messaging during a multistate outbreak investigation that a FoodCORE Center is involved in, that investigation would be associated with a public health action for the purposes of the metrics.