FoodCORE SSL Metrics

SSL metrics apply to Salmonella, Shiga toxin-producing Escherichia coli (STEC), Listeria, Shigella, and Campylobacter (reporting for Shigella and Campylobacter is optional). The FoodCORE performance metrics are a list of measurable activities covering diverse aspects of outbreak response. These activities span from outbreak surveillance and detection through investigation, response, control, and prevention measures. Using the metrics, each center provides data about the burden, timeliness, and completeness of foodborne disease activities related to the key areas of activity. Data for all years of the FoodCORE program are available.

Isolate/Specimen-based Metrics

Rationale: The intent of these metrics is to evaluate the timeliness and completeness/availability of laboratory surveillance and subtyping data for Salmonella, Shiga toxin-producing Escherichia coli (STEC), Listeria, Shigella, and Campylobacter. These metrics can be used to determine if there are gaps in the laboratory isolate handling and testing processes. If gaps are identified, knowing the detailed circumstances around the gap will help develop targeted actions to address them specifically.

  1. Total number of isolates and isolate-yielding specimens submitted to or recovered at the public health laboratory (PHL)1
    • Intent: To allow evaluation of the burden of isolate submissions and testing at the PHL.
  2. Number of primary isolates/isolate-yielding specimens submitted to or recovered at the PHL
    • Intent: To allow evaluation of laboratory testing associated with the first or representative isolate or sample for each case or testing unit for non-human isolates versus duplicate isolates from repeat testing or sampling protocols.
  1. Total number of preliminary positive clinical specimens or samples received at PHL (regardless of if isolate-yielding or not).
    • Intent: To allow evaluation of the submissions of presumptive positive clinical specimens to the PHL.
  2. Number and percent of isolate-yielding clinical specimens or samples
    • Intent: To allow evaluation of the outcome of clinical specimen testing.
    • Note: Could be used to identify gaps in submission protocols if a high proportion of specimens or samples are not viable; this also indicates the utility of testing multiple clinical specimens to try to identify cases that might otherwise be missed if only using isolate submissions.
  • Intent: To allow evaluation of the timeliness of isolate submission to the PHL.
  • Note: Time is measured in median days, measurements will exclude weekend days. For laboratory time measurements, only isolates tested at the PHL should be included.
  • Intent: To allow assessment of the impact, if applicable within the PHL, of the time it takes to recover an isolate if an isolate-yielding specimen is received so that subtyping can begin.
  • Note: This may not be applicable in all PHLs, for those that only receive isolates this metric would be zero. Time is measured in median days, measurements will exclude weekend days. For laboratory time measurements, only isolates tested at the PHL should be included.
  • Intent: To allow evaluation of the completeness of serotyping (serogrouping for Shigella)
  • Note: Complete serotyping was included in the language to address the concern that it may be possible that serotyping cannot be completed to a final result because of some quirk or feature of the isolate, which would have to be sent elsewhere to complete serotyping. Therefore, the metric would include isolates that have complete/final serotyping results at the PHL.
  • Intent: To allow evaluation of the timeliness of serotyping at the PHL.
  • Note: Time is measured in median days, measurements will exclude weekend days. For laboratory time measurements, only isolates tested at the PHL should be included.
  1. Percent of primary isolates with PFGE results1
    • Intent: To allow evaluation of the completeness of PFGE subtyping of isolates.
  2. Percent of primary isolates with WGS results
    • Intent: To allow evaluation of the completeness of WGS sequencing of isolates at the PHL.
  1. Time from isolate receipt (or recovery) at PHL to PFGE upload to PulseNet1
    • Intent: To allow evaluation of the timeliness of PFGE subtyping at the PHL.
    • Note: Time is measured in median days, measurements will exclude weekend days. For laboratory time measurements, only isolates tested at the PHL should be included.
  2. Time from isolate receipt (or recovery) at PHL to WGS sequence being shared with national database
    • Intent: To allow evaluation of the timeliness of WGS sequencing at the PHL.
    • Note: Time is measured in median days, measurements will exclude weekend days. For laboratory time measurements, only isolates tested at the PHL should be included.
  3. Time from receipt (or recovery) at PFGE laboratory to upload to PulseNet
    • Intent: To allow evaluation of the timeliness of PFGE subtyping at the PHL. Other performance metrics (e.g., PHEP, CIFOR) measure the turn-around-time (TAT) from receipt at the PFGE lab to upload to PulseNet. The FoodCORE metrics use a wider definition for the TAT in metric #8a; this submetric allows for the distinction between the two time periods while still retaining the original FoodCORE definition in metric #8a, which has proved useful for process evaluation.
    • Note: Time is measured in median days, measurements will exclude weekend days. For laboratory time measurements, only isolates tested at the PHL should be included.
  4. Time from receipt (or recovery) at WGS laboratory to sequence being shared with national database
    • Intent: To allow evaluation of the timeliness of WGS sequencing at the PHL. The FoodCORE metrics use a wide definition for the TAT in metric #8b; this submetric allows for measurement of the time spent by the PHL conducting sequencing while still retaining the original FoodCORE definition in metric #8b, which has proved useful for process evaluation.
    • Note: Time is measured in median days, measurements will exclude weekend days. For laboratory time measurements, only isolates tested at the PHL should be included.

Case-based Metrics

Rationale: The intent of these metrics is to evaluate the timeliness and completeness/availability of epidemiologic data for reported cases. These metrics can be used to determine if there are gaps in the epidemiologic interviewing process. If gaps are identified, knowing the detailed circumstances around the gap will help develop targeted actions to address them specifically. 

  1. Number of laboratory confirmed cases reported to epidemiology staff1
    • Intent: To allow evaluation of the burden of cases reported to epidemiology staff.
    • Note: This number may not be equivalent to laboratory isolate counts because of duplicate isolates submitted to the PHL, and/or because cases may be reported to epidemiology staff from outside the PHL in their health department (i.e., laboratory or clinical report of cases not submitted to the PHL). This metric will be used to calculate #10a.1, #10d, #10f, #10g.
  2. Number of probable cases reported to epidemiology staff
    • Intent: To allow evaluation of the burden of cases reported to epidemiology staff. Epidemiology staff are notified of and respond to cases that are not laboratory confirmed. This metric allows for additional characterization of the burden of cases report to epidemiology staff. This metric will be used to calculate #10a.2.
  3. Number of suspect cases reported to epidemiology staff (n/a for Campylobacter)
    • Intent: To allow evaluation of the burden of cases reported to epidemiology staff. Epidemiology staff are notified of and respond to cases that are not laboratory confirmed. This metric allows for additional characterization of the burden of cases report to epidemiology staff. This metric will be used to calculate #10a.2.
  1. Measures for attempted interview completeness
    1. Percent of laboratory confirmed cases reported to epidemiology staff (#9a) with attempted interview1
      • Intent: To allow evaluation of interviewing capacity to try to reach laboratory confirmed cases.
      • Note: To be based off #9a for calculation. This metric will be used to calculate #10e.
    2. Percent of probable/suspect cases reported to epidemiology staff (#9b + #9c) with attempted interview
      • Intent: To allow evaluation of interviewing capacity to try to reach non-laboratory confirmed cases.
      • Note: To be based off #9b and #9c for calculation. This percent may be lower than 10a.1 if laboratory confirmed cases are prioritized above probable and suspect cases for time and resources.
  2. Time from confirmed/probable/suspect case report to initial interview attempt1
    • Intent: To allow evaluation of TAT for attempted interviews. Attempted interviews were used here because factors that impact completion of an interview may be outside the control of epidemiology staff (e.g., case refuses), whereas making an attempt to interview is a TAT that can be controlled through data exchange and capacity.
    • Note: Time is measured in median days, measurements will exclude weekdays.
  3. Time from confirmed/probable/suspect case report to completed interview
    • Intent: To allow evaluation of TAT for completed interviews.
    • Note: Time is measured in median days, measurements will exclude weekend days. This metric will only capture the TAT for cases that were able to be interviewed which is a subset of cases with an attempted interview.
  4. Percent of confirmed cases reported to epidemiology staff (#9a) with complete demographic data
    • Intent: To allow evaluation of minimum available data (demographic) even in the absence of exposure history.
    • Note: It is possible to get demographic data and have some information to report/utilize for cases without a completed interview, so while most cases with demographic data will have a completed interview, limiting this to only interviewed cases may under-represent the availability of these data. To be based off #9a for calculations.
  5. Percent of confirmed cases with an attempted interview (#10a.1) with exposure history obtained1
    • Intent: To evaluate proportion of cases with assessment of exposures prior to onset of illness.
    • Note: To be based off #10a.1 for calculations. This metric will be used to calculate #10e.1.
      – The intended evaluation could be made using either #9a or #10a.1, but using #10a.1 has the added benefit of allowing assessment of how many attempted interviews are being completed.
    1. Percent of confirmed cases with exposure history obtained (#10.e) with full shotgun or case exposure completed
      • Intent: To evaluate the completeness of conducted interviews, i.e., how many interviews may have been a shorter interview and how many had collection of a full shotgun or exposure assessment.
      • Note: This is a sub-measure of #10e.
  6. Percent of confirmed cases reported to epidemiology staff (#9a) with serotype (serogroup for Shigella) information (n/a for Listeria and Campylobacter)
    • Intent: To evaluate how many cases reported to epidemiology staff have serotype/serogroup information that could inform or impact interviewing.
    • Note: To be based off #9a for calculations.
  7. Percent of confirmed cases reported to epidemiology staff (#9a) with PFGE information
    • Intent: To evaluate how many cases reported to epidemiology staff have PFGE information that could inform or impact interviewing.
    • Note: To be based off #9a for calculations. This metric will be used to calculate #10g.1.
    1. Percent of cases with PFGE information with exposure history obtained
      • Intent: To evaluate if the availability of exposure history is impacted by PFGE subtyping availability (i.e., does PFGE subtyping data availability impact the availability of exposure history).
      • Note: To be based off #10.g for calculations.
  8. Reason for not interviewing cases (e.g., lost to follow-up/refused, time lag too long, other)
    • Intent: To evaluate any trends or gaps in the reasons why an interview cannot be conducted.
    • Note: This is reported in percentages, by category, for cases where an interview was not conducted.

Investigation-based Metrics

Rationale: The intent of these metrics is to evaluate epidemiologic activity related to cluster and outbreak monitoring, evaluation, and investigation. These metrics can be used to determine if there are gaps in cluster and outbreak investigation. If gaps are identified, knowing the detailed circumstances around the gap will help develop targeted actions to address them specifically.

  • Intent: To allow evaluation of the burden of clusters/outbreaks and investigational needs.
  • Note: This metric will be used to calculate #12a, #12b, #12c, #13, #14, #15a, #15b, #15c, #15d, #15e, #16, #17, #18.
  1. Number and percent of investigations with routine interview of cases1
    • Intent: To allow evaluation of completeness of cluster response activities.
    • Note: This metric would indicate that initial interviews were conducted with a case(s) in your jurisdiction. To be based off #11 for calculations.
  2. Number and percent of investigations with supplemental or targeted interviewing of cases
    • Intent: To allow evaluation of completeness of cluster response activities.
    • Note: This metric would indicate that additional interviews were conducted beyond the initial interview for further hypothesis-generating. To be based off #11 for calculations.
  3. Number and percent of investigations where an analytic epidemiologic study was conducted1
    • Intent: To allow the evaluation of conducting or participating in analytic epidemiologic investigations.
    • Note: This metric would indicate that your jurisdiction was responsible for (i.e. led) or participated in analytic hypothesis testing. There may be clusters that do not warrant analytic epidemiologic investigation based on the hypothesis generating data. To be based off #11 for calculations.
  • Intent: To allow the evaluation of how often cluster investigations result in identifying suspect vehicles or sources. The evaluation of suspect vehicles or sources is important because even without a confirmed source, these investigations can still contribute to the body of knowledge of risky foods, practices, or other gaps in the food safety system in order to inform prevention efforts.
  • Note: There is not always a relationship between the completeness and/or timeliness of an investigation and identification of a suspect vehicle/source. To be based off #11 for calculations.
  • Intent: To allow the evaluation of how often cluster investigations result in identifying confirmed vehicles or sources. These investigations can still contribute to the body of knowledge of risky foods, practices, or other gaps in the food safety system in order to inform prevention efforts.
  • Note: There is not always a relationship between the completeness and/or timeliness of an investigation and identification of a confirmed vehicle/source. To be based off #11 for calculations.
  1. Number and percent of investigations with exclusion of a(an) ill person(s) from high risk setting
    • Intent: To allow the evaluation of excluding an ill person(s) within your jurisdiction to help minimize the risk to others and mitigate ongoing transmission.
    • Note: High risk settings may include, but are not limited to food handling, daycare attendance, or healthcare work. Not all investigations will yield evidence that support taking this kind of action. To be based off #11 for calculations.
  2. Number and percent of investigations with remediation or closure of an establishment linked to illness
    • Intent: To allow the evaluation of requiring remediation of an identified gap in food safety or even closure of an establishment within your jurisdiction to help minimize the risk to others and mitigate ongoing transmission.
    • Note: Not all investigations will yield evidence that support taking this kind of action. To be based off #11 for calculations.
  3. Number and percent of investigations with educational campaigns during outbreaks (beyond individual case education)
    • Intent: To allow the evaluation of conducting an educational campaign within your jurisdiction for at-risk groups to help minimize the risk to others and mitigate ongoing transmission.
    • Note: Educational campaigns, beyond individual case education, may include but are not limited to hand washing education in a classroom or daycare or safe food handling and preparation practices. Not all investigations will yield evidence that support taking this kind of action. To be based off #11 for calculations.
  4. Number and percent of investigations with media or public messaging (e.g., web updates, press release, etc.)
    • Intent: To allow the evaluation of notifying the public about a cluster investigation to help minimize the risk to others and mitigate ongoing transmission. This applies to notifications that occurred within your jurisdiction, or that your jurisdiction participated in (e.g., confirmed or contributed information to).
    • Note: Media or public messaging includes but is not limited to web updates or press releases of materials that would be available beyond the population directly impacted by a cluster or outbreak. Not all investigations will yield evidence that support taking this kind of action. To be based off #11 for calculations.
  5. Number and percent of investigations with regulatory action (e.g., recall, hold, etc.)
    • Intent: To allow the evaluation of taking a regulatory action to prevent initial or further distribution of a product associated with illness or risk of illness. This applies to regulatory actions that occurred within your jurisdiction, or that your jurisdiction participated in (e.g., confirmed or contributed information to).
    • Note: Regulatory action includes but is not limited to product recalls, holding product from distribution, or initiating other restrictions of sale or production. Not all investigations will yield evidence that support taking this kind of action. To be based off #11 for calculations.
  • Intent: To allow the evaluation of how often environmental health assessments are conducted within your jurisdiction as part of a cluster investigation.
  • Note: Not all investigations will yield evidence that support taking this kind of action. To be based off #11 for calculations.
  • Intent: To allow the evaluation of how often food or environmental samples are collected for testing within your jurisdiction as part of a cluster investigation.
  • Note: Not all investigations will yield evidence that support taking this kind of action. To be based off #11 for calculations.
  • Intent: To allow the evaluation of how often environmental health, agriculture, regulatory, or food safety program staff within your jurisdiction were engaged in cluster investigation activities.
  • Note: Not all investigations will yield evidence that support taking this kind of action. Additionally, contacting partners during an investigation does not necessarily imply that a regulatory action would be indicated or taken. To be based off #11 for calculations.

Outbreak-based Metrics

Rationale: The intent of these metrics is to evaluate outbreak reporting activity. These metrics can be used to determine if there are gaps in outbreak reporting. If gaps are identified, knowing the detailed circumstances around the gap will help develop targeted actions to address them specifically.

  • Intent: To determine the burden and completeness of outbreak reporting through NORS.
  • Note: It is understood that this value may not be 100% during specific reporting periods if an outbreak investigation is ongoing and therefore not ready to be submitted to NORS.

NOTE: 1Minimum reporting requirement for FoodCORE Centers

 Top of Page

Definitions

All measurements of time are in median days: Measurements will exclude weekend days. For laboratory time measurements, only isolates subtyped at the PHL should be included.

Isolate/Isolate-yielding specimen: This will include all isolates (human, food, environmental, etc.) submitted to PHL and isolates recovered from specimen submitted to PHL. This can be further broken down to report total number of each category of isolates (human, food, environmental, etc.).

Primary isolates/isolate-yielding specimen: To be limited to the first or representative isolate or sample for each case or testing unit for non-human isolates.

Laboratory confirmed, probable, and suspect cases: Refer to NNDSS case definitions for each pathogen

Complete demographic data: To include State, County, Birth Month, Birth Year, Sex, Race

Exposure history: To include an interview (of any format) that assesses exposures prior to onset of illness, via an open-ended exposure history, or via a list of potential exposures. The key factor to be considered an exposure history is an interview that goes beyond assessment of high-risk settings and prevention education to ascertain food consumption/preference, or other exposure data.

Cluster: Two or more cases of Salmonella or STEC infection with an indistinguishable PFGE pattern in 60 days, or 120 days for Listeria infections, where the number of cases meets one or more of the following criteria:

  1. The number of PFGE-matched isolates represent an increase over the expected baseline
  2. Demographic or other epidemiologic characteristics among cases with PFGE-matched isolates indicate some deviation from expected values for the region
  3. There is a PFGE-matched non-human isolate that would indicate a potential source of human infections
  4. In the absence of meeting the above criteria in a catchment area, a case-patient should be considered cluster-associated if the above criteria are met when including isolates from other jurisdictions or catchment areas.

In the absence of meeting any of the above criteria, ill persons should be considered cluster-associated if there are demographic or epidemiologic indications of a common source of infection even without laboratory subtyping data to link cases.

This definition also includes clusters that may be defined as outbreaks in your jurisdiction.

Above baseline: A significant deviation (either in the number of isolates or a change in the demographic/temporal characteristics of cases) from expected values based on historical (laboratory) data for a specific serotype or PFGE pattern.

Investigation: Any active epidemiologic follow-up resulting from the identification of a cluster. This could be initiating contact with a case (or the public health authority under whose jurisdiction a case falls) to ascertain direct case-based epidemiologic data, or active review of previously collected case-based data for cases later identified as cluster-associated.

Notification: Report of a case or cluster (depending on the metric) to epidemiology staff, i.e., when epidemiology staff first were made aware of a specific case or an identified cluster. This could be via routine communication such as a laboratory report or accessing a database, or via direct complaints, reports from another health authority (local, other state, federal, etc.), media report, or other means of communication.

Analytic epidemiologic study: A systematic, statistical analysis against a comparison group or within a cohort to test a hypothesis

Vehicle/Source Identified:

SUSPECT vehicle/source clusters: Clusters of infection where investigational and/or laboratory data indicate a likely source/vehicle of infection without confirmation: vehicle is a known risk factor, established errors in food preparation, or reported consumption by a high proportion of cluster-associated cases.

CONFIRMED vehicle/source clusters: Clusters of infection where the etiologic agent has either been cultured from the vehicle or the vehicle has been statistically implicated in an analytic study.

Control measure: to include interventions such as exclusion of an ill person(s) from high risk setting, remediation or closure of an establishment linked to illness, educational campaigns during daycare outbreaks, etc. To be considered a control measure, activities should extend beyond the routine educational component of an interview or exposure assessment.

Public health action: to include media, public messaging (web updates, press release, etc.), or regulatory action (recall, hold, etc.). To be considered a public health action, activities should extend beyond the routine investigation activities and reach at-risk individuals beyond identified cases. A public health action should be included in the metrics if the FoodCORE Center was directly involved in the action, or is aware that a public health action was taken during a multijurisdictional investigation. For example, if CDC produces public messaging during a multistate outbreak investigation that a FoodCORE Center is involved in, that investigation would be associated with a public health action for the purposes of the metrics.