FAQs: Analysis
2015 Re-baseline
Q1: What can hospitals and other organizations expect to happen to their SIRs after the 2015 re-baseline?
The data included in the 2015 baseline serves as a new “reference point” for comparing progress. CDC expects that hospital SIRs will increase and shift closer to 1, especially for SIRs that will be calculated for 2015. In addition, hospitals may notice that the rules for inclusion and exclusion of data have changed from the original baseline.
For more information, please see “NHSN 2015 Rebaseline SIR Changes Applicable to CMS Quality Reporting Programs [PDF – 150 KB]” and the NHSN SIR Guide [PDF – 50 Pages].
Generating Datasets
Q2: If I generate datasets, will I be able to analyze the data that were entered by my colleagues? Or can I only analyze the data I’ve entered?
Each user has access to the data entered for their facility, per their data access rights. The dataset generation process essentially copies and freezes data from the live database, so you’re actually using a copy of your facility’s data, per the data rights assigned to you. Any changes you or another user makes to your facility’s data within NHSN after that point in time will not be represented in your reports until you generate datasets again.
Q3: Do I have to generate datasets in order for CMS, or a Group (e.g., state health department) to see my facility’s data?
No. Datasets are user-specific and do not impact other users in NHSN who have access to your facility’s data. In addition, applicable data shared with CMS are prepared using the data in the NHSN database and are independent of the dataset generation feature.
SIRs
Q4: I work in a small hospital and therefore, the number of predicted infections is most often less than 1.0. As a result, our SIR is not calculated. What is the justification for this?
NHSN withholds the calculation of the SIR when the number of predicted infections is < 1.0 in order to enforce precision of the estimate and comparisons to the national data. If the number predicted is less than 1.0, this means that the risk of patients is low enough (according to national baseline data) such that not even 1 infection or event of that type is predicted to occur in that group of patients. Here are other options to consider:
- Infection rates can be used to track internal HAI incidence over time.
- Analyze your facility’s SIRs over a longer period of time (e.g., six months, one year)
- Run the TAP Reports to review the CAD (cumulative attributable difference, which is the difference between the # observed and # predicted)
Q5: What is a 95% Confidence Interval and how do I interpret it for my SIRs?
A 95% confidence interval (CI) is an interval for which we have a high degree of confidence that it contains the true SIR. The upper and lower limits are used to determine the accuracy and precision of the SIR. If the 95% CI of the SIR includes the value of 1.0 – meaning, the lower bound is < 1.0 and the upper bound is > 1.0, then the SIR is not statistically significant. Example of a non-significant 95% CI: (0.674, 3.578) – notice that the lower bound, 0.674, is less than 1.0 and the upper bound, 3.578, is greater than 1.0. Since these two values are on the opposite side of the nominal value of 1.0, we say that the 95% CI “includes” 1.0 and is therefore not statistically significant.
For additional details regarding the interpretation of p-values and 95% CIs, please see the NHSN Analysis trainings.
Line Lists
Q6: Is there a way to see which records have been changed in NHSN and when?
Yes. Line lists available from the “Advanced” report options folder provide the date a record was first entered into NHSN (variable name = createDate) and the last date when a non-deleted record was modified (variable name = modifyDate). If a record has not been changed, the modify date and the create date will be the same. Note that this information will only list the date of the last saved change prior to generating datasets. Additionally, there is no record of exactly which data elements within the record have been changed.
For more information about how to view the Create and Modify dates [PDF – 342 KB] in NHSN.
SSI SIRs
Q7: I am running an SSI SIR and I noticed that there are a few procedures listed at the bottom of the report that are excluded from the SIR calculation. Why are they excluded?
Procedures (and any associated SSIs) can be excluded from the SIRs if they meet one or more of the defined exclusion criteria, as described in the Table 2 of the SSI section of the SIR Guide [PDF – 50 Pages].
A specialized line list that can be run in NHSN is the “Line Listing – Procedures Excluded from SIR [PDF – 366 KB]” that will include only those procedure records that are excluded from the SIRs for the reasons listed.
While these records may be technically complete, the data in those records are considered “outliers” or invalid (for example, extremely high procedure duration) and could indicate a potential data quality issue.
There are some instances when the record can be fixed (specifically, if you notice a data entry error). There are other instances, however, where it may be impossible to include the procedure in the SIR. For example, if a procedure has an extremely long procedure duration (what we call being greater than the interquartile range (IQR5) – as described in the above SIR Guide) – and that duration is accurate – there is nothing that can be fixed on the record and therefore, it will continue to be excluded from the SIRs.
Certain procedures may also be excluded from the SIR depending on the SSI SIR model you are using. The inclusion and exclusion criteria for each SSI model are stated in the SSI section of the SIR Guide [PDF – 50 Pages] as well as the SSI Protocol Chapter.
SSI Rates
Q8: Where can I find the most recently published SSI rates?
In the past, CDC has published SSI rates stratified by procedure category and basic risk index. NHSN has last published data in this way in 2009 – the report can be found here: 2009 NHSN Report [PDF – 6 MB].
Since the fall of 2010, we have progressed to the use of standardized infection ratios (SIRs), which utilize a different kind of risk adjustment and is an improvement over the risk adjustment afforded by the legacy basic risk index. In the past, we compared a hospital’s SSI rate to the national SSI rate (for a given strata). With the SIRs, we can now use the national baseline data to determine risk adjustment and the number of SSIs predicted based on those risk factors. For details regarding this risk adjustment used in the SIRs for all procedure categories, please see the following resources:
- SIR Guide for the 2015 baseline [PDF – 50 Pages]
- The model paper for the 2006-2008 baseline [PDF – 600 KB]
- Most recent HAI Progress Report
MRSA or C.difficile SIR
Q9: My line list shows 5 healthcare facility-onset (HO) C. difficile LabID events. Why are only 3 events showing in the SIR report?
Not all CDI LabID events are counted in the SIR. Only those events from an applicable inpatient location that are categorized as incident and healthcare-facility onset are included in the SIR. When running your CDI line list, look at the variable called “FWCDIF_facIncHoCount”; this variable will equal 1 for each event counted in the SIR. Rehab units within a hospital that are designated as CMS IRF units use a different algorithm for determining which events are counted in the Rehab unit’s SIR. More information about the algorithms used for all LabID Event SIRs, as well as general troubleshooting steps, can be found in the LabID SIR Troubleshooting Guide [PDF – 300 KB].
Q10: I am unable to get a C. difficile SIR for a single month. The SIR report shows a second table with months excluded from the SIR. Why?
The risk adjustment for CDI LabID events under the 2015 baseline requires that data from all 3 months of the quarter have been entered into NHSN before an SIR can be calculated. Several variables in the risk adjustment models, including the inpatient community-onset (CO) prevalence rate and CDI test type, are based on data that are entered for the entire quarter. If you run the LabID event SIR report before completing data entry for all 3 months of the quarter, you will see a table showing those months that have been entered into NHSN from an incomplete quarter; these months will continue to be excluded from the SIR until data entry is complete for the entire quarter. Once data from all 3 months of the quarter(s) have been entered, SIRs can be calculated for a quarter, half-year, or year. Monthly CDI and MRSA bacteremia data can be reviewed using the Rate Tables, Frequency Tables, and Line Lists.
Fiscal Year SIRs
Q11: I would like to obtain a fiscal-year SIR, however when I try to set the time period, I am unable to obtain a single SIR.
If you would like a cumulative SIR for a time period that you’ve defined on the Modify Screen – such as fiscal year, this can be obtained using the “Display Options” tab of the Modify Screen. Set Group by = Cumulative.
CLABSI ECMO and VAD Events
Q12: Are MBI-LCBI still included in the CLABSI SIR?
Are CLABSI events that are identified as ECMO and VAD included in the CLABSI SIR?
CLABSI events reported to NHSN as ECMO and VAD will be excluded from the numerator beginning with 2019 CLABSI data and forward. To identify these specific excluded events, please run your CLABSI Device-associated line list. For more information on modifying your reports, please see: How to Modify a Report [PDF – 375 KB]
Rehab Unit Data
Q13: Can I combine my inpatient rehab unit data with my ACH data?
With the re-baseline, only the Inpatient Rehabilitation Facilities (IRF) SIR reports will contain Rehab unit data. Acute Care Hospital (ACH) Reports will no longer contain Rehab unit data. This is due to varying risk factors and models that have been applied to the ACH and IRF SIRs. Therefore, the IRF unit data and the ACH data cannot be combined into the same report.
Previously Excluded Locations
Q14: With the re-baseline, we were told that Telemetry units and Mixed Acuity Units will be included in the SIR reports, however, I still don’t see these units in my CMS SIR reports. Why?
The CMS SIR reports were created in order to allow facilities to review those data that would be submitted to CMS on their behalf. These reports will only include the locations that meet the CMS reporting requirements. Telemetry Units and Mixed Acuity Units will not be included in the CMS reports. However, these units will be included in the non-CMS SIR reports. Please see the link below on more information regarding reportable locations and deadlines: Reporting Requirements and Deadlines in NHSN per CMS Current Rules [PDF – 853 KB]
PS Annual Survey
Q15: Does NHSN use the most recent PS annual survey for SIR Calculations?
Under the 2015 NHSN baseline, SIRs are risk adjusted using the corresponding annual survey from that same year. If the annual survey from that year hasn’t been completed, SIRs will be risk adjusted using the previous year’s survey.
TAP Reports
Q16: How can I use TAP Reports to review a subset of locations (for example, only ICU locations)?
This is not recommended. TAP Reports are designed to identify and prioritize locations where prevention efforts can yield the greatest impact. All locations must be included in the analysis for the TAP report to properly prioritize the facility’s locations for the TAP strategy.
Q17: Why would a location not be ranked in the TAP Report?
The Cumulative Attributable Difference (CAD) is the metric used to rank facilities or facility locations within the TAP Report. The CAD is based on the same rules and exclusions that influence the SIR calculation. Therefore data that is not included in the calculation of the SIR, will not be included in the calculation of CAD. Locations that are not included in the SIR calculation for a particular HAI will also be excluded from TAP Reports. For example, outpatient locations are excluded from the SIR. Outpatient locations may show up in the TAP Report as unranked locations with the location CAD, Location DUR, and location SIR all remaining blank.
Outpatient Procedures
Q18: Are outpatient procedures included in the new 2015 baseline models?
Under the 2015 baseline, hospital outpatient procedure department (HOPD) procedures were modeled separately from the inpatient procedures. This means, there are separate SIR reports for the inpatient procedures and HOPD procedures in NHSN.
Outpatient procedures from ambulatory surgery centers were also separated from the general acute care hospitals and included in a new NHSN component called the outpatient procedure component (OPC).
Please see the SSI section of the SIR guide [PDF – 50 Pages] for more information on the new risk-adjusted models, it starts on page 26.
Closure techniques of other than primary
Q19: Are procedures with other than primary closures included in the SIR?
All of the SSI reports will include procedures (and associated SSIs) that are reported with either primary or other than primary closure techniques. This is a change that was made during the 2015 rebaseline process and not related to a change in protocol or definition changes.
Under the previous risk models and baseline of 2006-2008, the SSI reports included only procedures (and associated SSIs) that were reported with primary closure technique.
Rates
Q20: Where can I find the most recent National HAI Rates
National, stratified rates will no longer be provided annually. This is due to the manner in which the HAI data are risk-adjusted – meaning, we will be using regression models to estimate the predicted number of infections, and using that information to calculate SIRs. In addition, CLABSI and CAUTI now use factors in addition to location in this risk-adjustment. A similar approach is applied for device use, such that facilities will have the ability to run standardized utilization ratios (SURs).
Note that, for both SIRs and SURs, these measures will continue to be available at the location-level. In contrast, the location-level rates and DURs will not have a national rate or DUR comparison.
