Detailed Evaluation of Changes in Data Collection Methods

Background and Process

• Maternal mortality is an important health indicator, and correctly identifying and reporting maternal deaths has been a longstanding challenge.
• A maternal death is defined as, “the death of a woman while pregnant or within 42 days of termination of pregnancy,” but excludes those from accidental or incidental causes.
• Official statistics are obtained through death certificates completed by physicians and reported to states. To promote standardization, the National Center for Health Statistics (NCHS) periodically organizes a national consensus process that makes recommendations to states on the content of birth and death certificates.
• Responding to research showing that maternal deaths were undercounted on death certificates and that states used noncomparable methods, in 2003, this consensus process recommended that all states add a standardized checkbox to improve the identification of maternal deaths. Physicians were instructed to check a box for a known pregnancy in defined timeframes before death.
• Vital registration systems are controlled individually by each state, and the standardized checkbox was implemented as funding, technology, and state laws allowed. The last state added the checkbox to the death record form in 2017.
• As state systems diverged, the comparability of data diminished to the point where in 2007, NCHS suspended the publication of national maternal mortality data.

Evaluation of the Impact on Maternal Mortality Rates (MMRs)

• With states now using more comparable approaches, NCHS undertook a rigorous evaluation of maternal death reporting.
• The new checkbox, as intended, substantially increased the identification of maternal deaths. The addition of the checkbox led to capturing twice the rate of maternal deaths through death certificates.
• If all states added the checkbox at once, there would have been a large, one-time increase in the reported rate. However, since the checkbox was added one state at a time, over an extended period, assessing the real trends in maternal mortality over time has been a challenge.
• Two NCHS reports capture the result of a rigorous evaluation by NCHS: “Evaluation of the Pregnancy Status Checkbox on the Identification of Maternal Deaths pdf icon[PDF – 1 MB]” describes the changes in maternal death rates due to the inclusion of the checkbox. “The Impact of the Pregnancy Checkbox and Misclassification on Maternal Mortality Trends in the United States, 1999–2017 pdf icon[PDF – 6 MB],” uses a set of sophisticated statistical models to look at trends over this period to better understand the impact of the gradual adoption of the checkbox on national statistics.
• A separate report, “Maternal Mortality in the United States: Changes in Coding, Publication, and Data Release, 2018 pdf icon[PDF – 1 MB]” includes the 2018 MMRs and discusses new coding procedures.

What NCHS Found

• The understanding of the trend in maternal mortality changed significantly with these new studies. In short, using more comparable data across states, NCHS found that the increase in maternal mortality in the United States is not likely due to a true increase in the underlying extent of maternal mortality. Rather, the majority of the observed increase in the MMR is attributed to changes in data collection methods (i.e., the gradual adoption of the checkbox). Based on the pre-2003 coding method, the MMR was 8.9 in 2002 and 8.7 in 2018. (NVSR Vol. 69, No. 2, Table Epdf icon).
• The understanding of this trend comes from looking at this in two different ways, and the congruence of these two different approaches strengthens our confidence in the findings. First, NCHS looked at direct counts from certificates, taking into account the addition of the checkbox over time. Second, NCHS developed a sophisticated model of the data to test two different scenarios: What would the data have looked like if no state had used a checkbox, and what if all states had implemented a checkbox at the same time? In both cases, NCHS found that the MMR did not change significantly over this period.
• NCHS also found that while the addition of the checkbox allowed more pregnancies to be identified, it also likely introduced errors and overcounting with increasing age. For example, in deaths to women over age 44, NCHS found the rates were as much as 124.1 times those of younger women, and that in many of these cases, there was no accompanying cause-of-death information that supported the conclusion that the death was likely related to pregnancy.

For more information, see Implementation of New Coding Methods and the 2018 Maternal Mortality Reports.