Effect of Cause-of-Death Training on Agreement Between Hospital Discharge Diagnoses and Cause of Death Reported, Inpatient Hospital Deaths, New York City, 2008–2010

Introduction Accurate cause-of-death reporting is required for mortality data to validly inform public health programming and evaluation. Research demonstrates overreporting of heart disease on New York City death certificates. We describe changes in reported causes of death following a New York City health department training conducted in 2009 to improve accuracy of cause-of-death reporting at 8 hospitals. The objective of our study was to assess the degree to which death certificates citing heart disease as cause of death agreed with hospital discharge data and the degree to which training improved accuracy of reporting. Methods We analyzed 74,373 death certificates for 2008 through 2010 that were linked with hospital discharge records for New York City inpatient deaths and calculated the proportion of discordant deaths, that is, death certificates reporting an underlying cause of heart disease with no corresponding discharge record diagnosis. We also summarized top principal diagnoses among discordant reports and calculated the proportion of inpatient deaths reporting sepsis, a condition underreported in New York City, to assess whether documentation practices changed in response to clarifications made during the intervention. Results Citywide discordance between death certificates and discharge data decreased from 14.9% in 2008 to 9.6% in 2010 (P < .001), driven by a decrease in discordance at intervention hospitals (20.2% in 2008 to 8.9% in 2010; P < .001). At intervention hospitals, reporting of sepsis increased from 3.7% of inpatient deaths in 2008 to 20.6% in 2010 (P < .001). Conclusion Overreporting of heart disease as cause of death declined at intervention hospitals, driving a citywide decline, and sepsis reporting practices changed in accordance with health department training. Researchers should consider the effect of overreporting and data-quality changes when analyzing New York City heart disease mortality trends. Other vital records jurisdictions should employ similar interventions to improve cause-of-death reporting and use linked discharge data to monitor data quality.


Introduction
Mortality statistics, derived from death certificates, summarize population disease burden, inform research and program evaluations, and help determine public health priorities (1,2). Inaccurate cause-of-death reporting may result in misinformed policies, programs, and research (3)(4)(5). Previous studies found low levels of agreement between the death certificate cause of death and the sequence of events reported on the medical record (2,6,7). One possible reason for this disagreement is inadequate training of health The opinions expressed by authors contributing to this journal do not necessarily reflect the opinions of the U.S. Department of Health and Human Services, the Public Health Service, the Centers for Disease Control and Prevention, or the authors' affiliated institutions.
care providers in death certification (8,9). Clinicians throughout the United States overreport heart disease as a cause of death and do so to a greater degree in New York City. A 2001 study of 4 US cities found that medical records did not substantiate 20% of death certificates that reported heart disease as an underlying cause of death (10), whereas a similar 2003 New York City study found an average discrepancy of 33% (11).
Training can improve the accuracy of cause-of-death reporting (12,13). Since 2008, the New York City Department of Health and Mental Hygiene (DOHMH) has produced numerous cause-ofdeath training resources, including written materials (14), Webbased modules (15), and a hospital intervention with staff in-service trainings (16,17). The training intervention was launched in 2009 at 8 noncardiac-specialty hospitals, each of which had reported more than 60% of deaths as due to heart disease. The main training objective was to teach and review methodology for accurate cause-of-death reporting on the death certificate. Course materials also emphasized legal requirements for reporting and the importance of valid documentation. Heart disease deaths at these hospitals fell from 68.8% in 2008 to 32.4% in 2010; an increase in other causes was generally proportional to their burden in the population (16,18). Clinicians reported at DOHMH trainings that the New York City medical examiner's policy of not allowing sepsis to stand alone as an underlying cause of death (14) had been misinterpreted by hospital staff to mean that the terms "sepsis" or "septicemia" were not accepted for use anywhere on the death certificate.
The objective of our study was to assess the degree to which death certificates citing heart disease as cause of death agreed with hospital discharge data, which were used in validation studies as a proxy for the medical record (3,19); the discharge diagnosis associated with lack of agreement; and whether New York City's cause-of-death training efforts improved agreement. More generally, we evaluated the feasibility of using linked hospital discharge data and death certificate records for future initiatives to improve data quality.

Intervention hospitals
Our analysis included the 8 intervention hospitals targeted in the 2009 New York City DOHMH training intervention. These hospitals had among the highest ratios of reported heart disease deaths to total deaths (16). Nonintervention hospitals included the 50 remaining hospitals reporting inpatient deaths in 2009. Additional characteristics of intervention and nonintervention hospitals have been described previously (11,16,18). For our analysis, we defined "preintervention" as the period from January 1 through December 31, 2008, and "postintervention" as the period from January 1 through December 31, 2010.

Data sources and key variables
Death certificates and cause of death. The New York City Health Code mandates complete and accurate reporting of all deaths that occur in the city (20  (22) to each condition, event, or injury reported as contributing to death. The software then applies a standardized algorithm to assign an underlying cause to each death. A trained New York City DOHMH nosologist assigns causes manually when automatic coding fails (21).
The World Health Organization defines the underlying cause of death as the "disease or injury which initiated the train of morbid events leading directly to death, or the circumstances of the accident which produced the fatal injury" (23). We flagged deaths as due to heart disease if the ICD-10 code for the underlying cause of death was I00-I09, I11, I13, or I20-I51, in accordance with NCHS definitions (24).
Hospital discharge records and diagnosis codes. The New York Statewide Planning and Research Cooperative System (SPARCS) maintains a database of patient-level diagnoses and treatments of every hospital discharge, as mandated by public health law (25). Medical coders review the medical record to determine a principal diagnosis, defined as the "condition established after study to have been chiefly responsible for occasioning the admission of the patient to the hospital for care," as well as other diagnoses, which include "all conditions that coexisted at admission, or developed subsequently, which affected the treatment received or length of stay" (26).
SPARCS diagnoses are coded according to the International Classification of Disease, 9th Revision, Clinical Modification (ICD-9-CM) (27). To compare SPARCS diagnoses and underlying causes of death, we created a cross reference between ICD-9-CM and ICD-10 codes for heart disease and other conditions (Box). We analyzed linked data for New York City inpatient, naturalcause deaths that occurred from 2008 through 2010 by comparing diagnoses from the hospital discharge record with the underlying cause on the death certificate to identify unsubstantiated reports of heart disease. For this study, we focused on inpatient deaths, because our intervention took place only at hospitals. We also lim-ited our analysis to hospital admissions during which the patient died. Our final sample included 74,373 inpatient deaths, approximately 25,000 for each year of study, and reflected 46.6% of all deaths that occurred in New York City from 2008 through 2010. Our sample represented 91.3% of inpatient deaths. Unmatched deaths probably resulted from misaligned identifiers or misreported place-of-death information. Nonetheless, our matched sample closely represented reporting of heart disease for New York City inpatient deaths, because 29.3% of all inpatient death certificates and 28.8% of matched inpatient death certificates reported an underlying cause of heart disease ( Table).

Analysis
The focus of our study was agreement between the underlying cause of death reported on the death certificate and the principal and other diagnoses listed on the hospital discharge record. We defined a discordant record as one in which the death certificate reported an ICD-10 underlying cause of death of heart disease and the SPARCS record had no corresponding ICD-9-CM diagnosis for heart disease. We calculated the proportion of discordant records by year and intervention hospital status and tested statistical significance for changes from 2008 through 2010 using a z test for proportions. Cited probabilities are 2-tailed, and a value of P < .01 was considered significant because of the large size of the population.
Among discordant reports, we conducted a secondary analysis of the principal diagnosis to identify conditions that were important to death but misrepresented by a reported underlying cause of heart disease. We summarized top principal diagnoses among discordant reports by year, grouped into ICD-9-CM disease classes (Box).
We calculated post hoc the proportion of all inpatient death certificates that reported sepsis as an immediate, intermediate, or underlying cause of death to assess whether reporting practices changed in response to clarifications made during the intervention. We defined sepsis by the NCHS classification of septicemia (ICD-10 codes A40, A41) (24). All analyses were conducted using SAS version 9.2 (SAS Institute, Inc).

Results
Heart disease mortality and discordant reports of heart disease deaths  Abbreviation: NA, not applicable. We defined discordant reports of heart disease as deaths for which the death certificate reported an underlying cause of death of heart disease and the Statewide Planning and Research Cooperative System hospital discharge record had no corresponding diagnosis of heart disease.

Principal diagnoses among discordant reports
The top 5 disease classes reported on death certificates were infectious and parasitic diseases (42.3%; n = 464); respiratory diseases (16.0%; n = 175); circulatory diseases (9.6%; n = 105); digestive diseases (7.3%; n = 80); and neoplasms (6.0%; n = 66). The overrepresentation of infectious and parasitic diseases among discordant cases prompted post hoc analyses of this class. The top principal diagnosis among discordant reports was sepsis, a condition classified within infectious and parasitic diseases, which was reported in 38.5% (n = 422) of discordant cases (90.9% of all the parasitic disease discordant cases) in 2008.
Because sepsis necessarily proceeds from another disease or condition and is thus not considered an appropriate underlying cause

Discussion
To our knowledge, ours is the first study to use hospital discharge data linked to death certificates to demonstrate the effect of training on cause-of-death reporting practices. Agreement between reporting of heart disease as an underlying cause of death and a diagnosis on the corresponding SPARCS record increased after a 2009 cause-of-death training intervention at New York City hospitals. The improvement was substantial at intervention hospitals; agreement increased 11.3%, from 79.8% in 2008 to 91.1% in 2010. Citywide, agreement increased 5.3 percentage points from 85.1% in 2008 to 90.4% in 2010. If we assume hospital discharge records are a reasonable proxy for the medical record, these results suggest the training improved the accuracy of cause-of-death reporting, which we could not conclude from the observed decline in heart disease deaths alone. The 2.0 percentage point change in agreement at nonintervention hospitals from 88.2% in 2008 to 90.2% in 2010 was also significant, potentially explained by simultaneous implementation of other lower-intensity citywide DOHMH cause-of-death training efforts (15,16).
Further evidence of the impact of the intervention on accurate cause-of-death reporting was the increase in reporting of sepsis on death certificates that paralleled the increase in agreement for heart disease deaths ( Figure 2). In 2008, sepsis was the most common principal diagnosis listed for discordant cases, likely reflecting the confusion surrounding protocol for reporting sepsis. Despite clarifications made in the DOHMH City Health Information bulletin on cause-of-death reporting (14) and e-Learning course (15), rejection of stand-alone reports of sepsis appears to have resulted in misinterpretation that sepsis was not allowed to be listed as part of the cause-of-death mechanism at all. This misconception probably arose from the New York City medical examiners' practice of not approving death certificates reporting sepsis as a stand-alone cause of death during their cremation clearance process (ie, review to approve human remains for cremation) out of concern that sepsis can at times result from an injury (14).
In cases of sepsis, hospitals apparently tended to report heart disease instead. In a survey conducted among New York City resident physicians in 2010, 70.0% of respondents believed they were forced to select an alternate cause of death when a patient died of septic shock, and among those who knowingly reported an alternate cause of death, 64.6% reported cardiovascular disease as the most frequent diagnosis assigned (9). Hospitals face pressure to expedite death registration. Inadequate training in avoiding delays through proper documentation may have resulted in staff reporting a common and uncomplicated cause of death to streamline the process. The New York City DOHMH training included detailed information on sepsis reporting and apparently enhanced under-standing of reporting protocol; at intervention hospitals, sepsis was listed as a cause of death for 20.6% of deaths in 2010 compared with 3.7% in 2008. Still, sepsis remains underreported citywide. The public health significance of this data-quality issue was recently illustrated when the NYS Department of Health was unable to use death certificate data for their initiative to prevent in-hospital, sepsis-related mortality (29). Any mention of sepsis on New York City death certificates is much lower than the rest of the state, and although the gap in sepsis mortality rates between the 2 regions narrowed in 2009 and 2010, the data are not comparable for measuring the burden of sepsis mortality in New York City versus New York State hospitals (New York State Department of Health, oral communication, January 29, 2014).
These results highlight the need for cause-of-death training to improve the accuracy and validity of mortality statistics (16). The observed increased agreement between heart disease deaths and hospital discharge records corresponds to the decrease in heart disease deaths reported following the 2009 training and suggests the accuracy of heart disease reported as a cause of death increased as a result. Researchers using New York City mortality data must consider that some portion of observed declines in citywide heart disease mortality across this period reflects inflated estimates before 2008 and that observed declines are not solely due to changes in behaviors or risk factors or improved treatment. Other vital records jurisdictions can evaluate the quality of cause-of-death reporting at hospitals by using metrics such as discordance between the death certificate and hospital discharge record. These methods can serve as a reasonable proxy for resource-intensive medical record audits (19). Specifically, jurisdictions could target interventions at hospitals with poor agreement for causes of death of public health interest and monitor trends in agreement to evaluate the influence of training efforts.
A limitation of this study is that we evaluated agreement between the underlying cause of death and hospital discharge diagnoses as opposed to directly assessing accuracy of the recorded underlying cause of death through medical chart review. For our analysis, we assumed hospital discharge diagnoses were both accurate and related to death. SPARCS data are used for billing purposes so not all listed diagnoses are expected to be related to death. A single hospitalization may last weeks, and some diagnoses may relate to conditions that were stable or had resolved before death. Nonetheless, the condition reported as the cause of death would likely have been diagnosed or treated during the final admission and therefore coded in the hospital discharge record. Furthermore, it is unlikely that the extent to which SPARCS data were related to death changed during the study period, so our overall conclusions, which were based on the decreasing discordance rates over time, do not depend on this assumption. Our definition of agreement required that the discharge record indicate a diagnosis of heart disease for deaths reporting an underlying cause of heart disease, not that any diagnosis of heart disease had to have a corresponding underlying cause of death. Thus, our analysis is conservative. These assumptions may have led to erroneous classification of cases as in agreement, resulting in an overestimation of agreement between the 2 sources. These limitations may reflect the difficulty in determining a single cause of death or selection of diagnosis codes in patients with complex medical histories. Regardless, we expect that a condition identified as the underlying cause of death should in most cases be mentioned as a diagnosis affecting a patient's care and hospital stay.

PREVENTING CHRONIC DISEASE
We limited our analyses to hospital inpatient deaths because our intervention was carried out in hospitals. Therefore, our findings are not generalizable to outpatient deaths. Because not all heart disease death certificates were successfully matched, our ability to draw conclusions about all New York City inpatient heart disease deaths may be limited. However, we found that the matched data had a near-identical cause-of-death distribution to all New York City inpatient deaths, so it is unlikely that our measures of agreement were skewed by these missing observations.
We plan continued application of the data set to monitor trends in agreement between the 2 sources, extending the time period to evaluate later cause-of-death training activities and to ensure that improvement in agreement for heart disease did not occur as a consequence of practices of inaccurate reporting shifting to rely on other causes of death. We also plan to evaluate agreement for additional causes of death, including stroke, a condition suspected to be underreported in New York City; the 2010 New York City ageadjusted rate of stroke as cause of death was 18.8 per 100,000 compared with a national rate of 39.1 per 100,000 (24,30). Given the potential for these linked data to inform clinical and data-quality improvement, more work should be done to validate hospital discharge records against the medical record, an area of current research at DOHMH.
The linked death certificate-hospital discharge data set permitted evaluation of the effect of the 2009 New York City DOHMH cause-of-death training intervention without a resource-intensive medical record audit. Other vital records jurisdictions with access to linked data should consider using agreement measures to evaluate the quality of cause-of-death data and take action as needed (3). Jurisdictions without linked data should continue to address barriers preventing them from obtaining these linkages. Researchers depending on mortality data should work with vital records jurisdictions to understand data-quality concerns and support quality-improvement interventions for cause-of-death reporting to enhance their own work.