Monograph Menu Report Home Director's Foreword Table of Contents Tables and Figures Acknowledgements Abstract Prologue Introduction Navigating Health Futures Valuing Conditions Crafting Conditions Perceiving Dynamic Conditions Reorienting Public Health Work Transforming Conditions Reflecting on Public. Health. Work. Glossary References About the Author Download Full Report (PDF-8.9Mb)

Transforming Conditions

Simulation Modeling: Finding Plausible Futures for Diabetes Prevalence⁴²

Previous | Next

In each of the past three decades in the United States, national health objectives have been set 10 years into the future and published as the Healthy People Objectives for the Nation (United States Public Health Service, 1980, 1990, 2000). These objectives define specific, numerical targets for reductions in most major health problems as well as for increases in the prevalence of health-promoting behaviors. Michael McGinnis, a chief architect of the Healthy People enterprise, asserts that, “Of the broad range of governmental responsibilities in public health, perhaps none is more fundamental than the obligation to provide perspective and direction to guide health programs along a productive course—the agenda-setting function” (McGinnis, 1985)

Considering the widespread use and significance of the Healthy People (HP) objectives for planning and evaluating public health work at all levels of practice, health care practitioners may expect national health objectives to be feasible, that is, to be achievable within the specified time frame. However, Healthy People objectives may not always meet this feasibility standard (Mendez and Warner, 2000). The objectives for 2010, in particular, were set on the basis of a policy goal of eliminating health disparities among racial and ethnic groups.

Material for this section comes from an ongoing collaborative project known as the CDC Diabetes System Modeling Project (involving Jack Homer, Drew Jones, and Don Seville as the lead system dynamics modelers, along with Joyce Essien, Dara Murphy, myself, and many others as the CDC participants). Additional details on the project can be found in Homer, Jones, Seville, et.al., 2004; Jones, Homer, Murphy, et.al., 2006. The text for this section is adapted from Milstein, Jones, Homer, et.al., 2007.

Consequently, planners used a "better than the best" approach wherein each objective was set at a level better than that of the "best" (i.e., most healthy) racial or ethnic group. That approach advanced health equity as an important philosophical ideal, which, in turn, generated an ambitious aspiration for health policy-making. But it may not have yielded, in all cases, objectives that are achievable and compatible with other public health objectives. In addition, the practice of conducting midcourse reviews and periodic evaluations of progress toward meeting HP objectives may convey the impression that the numerical targets are actually achievable by 2010 and are therefore meaningful referents for assessing progress (Mukhtar, Jack, Martin, et.al., 2006; United States. Office of Disease Prevention and Health Promotion, 2006).

Questioning whether long-range objectives can, in fact, be reached also raises questions about the thinking and analytic procedures that guide objective-setting itself, a complicated and still-poorly understood dimension of public health science. In this section, we examine how a team of investigators used system dynamics simulation methods to (1) interpret the

U.S. track record regarding the diagnosed prevalence of diabetes; and (2) anticipate plausible futures through 2010 under various scenarios.

Tracking Past Performance
Figure 17 displays the observed trend in diagnosed diabetes prevalence per thousand population between 1985 and 2003; it also shows the two paths projected in the HP 2000 and HP 2010 documents. In 1990, after three decades of mostly rising prevalence (Kenny, Aubert, Geiss, 1995), the HP 2000 baseline was set using 1987 data (point A), and the objective called for an 11% reduction by 2000 (point B). During the intervening years, diagnosed prevalence did not decrease but instead increased 33% (from point B to D). The official HP 2000 final review reported that diagnosed prevalence “moved away from target” by 367% (calculated by comparing the D-to-B gap with the A-to-B target decrease) (National Center for Health Statistics, 2001).

Figure 17: Diagnosed Diabetes Prevalence per Thousand Total Population—United States, 1985—2003a, with Healthy People Objectives 2000 and 2010^b

The Healthy People 2010 objective (based on 1997 data) called for an even more ambitious 38% reduction (from point C to E). But again, surveillance data reveal a worsening trajectory. From 1997 to 2003, diagnosed prevalence rose another 25% (point C to F), making the 2010 objective even more unattainable.

What accounts for these discrepancies between objectives and actual experience? Are they due to poor performance of the overall national health protection strategy, which includes an array of separately focused programs and policies (Murphy, Chapel, Clark, 2004)? Or, are they perhaps the result of a flaw in the attainability of the numerical targets themselves? If  the latter is the case, what are some of the more plausible trajectories that might unfold by 2010?

Members of the CDC Diabetes System Modeling Project (Homer, Jones, Seville, et.al., 2004; Jones, Homer, Murphy, et.al., 2006) sought to answer these questions by conducting a series of simulation experiments using an existing system dynamics model designed specifically to explore the population dynamics of diabetes in the United States (Homer, Jones, Seville, et.al., 2004; Jones, Homer, Murphy, et.al., 2006). The model was designed to explore the incremental effects of a variety of possible policy interventions on the burden of diabetes. To achieve this result, the SD model, unlike other diabetes models (for example, a Markov model by Honeycutt et al (Honeycutt, Boyle, Broglio, et.al., 2003)), comprehensively accounts for a chain of population flows that begins when a person becomes at risk for diabetes and continues through initial onset, diagnosis, progression, and death. Such breadth of scope allows the SD model to anticipate nonlinear changes in variables, such as the incidence rate, that narrower models would miss (Homer, 2006).

The SD diabetes model was developed using well established techniques for model formulation and testing (Forrester, 1961, 1969; Homer and Hirsch, 2006; Homer and Oliva, 2001; Sterman, 2000, 2001). Data obtained from national health surveys (i.e., the National Health Interview Survey, the National Health and Nutrition Examination Survey, and the Behavioral Risk Factor Surveillance System), the U.S. Census, and publications in the scientific literature are the foundation for parameter selection and estimation. Some parameter estimates could be drawn directly from available information, while others were obtained through a process of historical curve-fitting analogous to statistical regression (for more detail see Homer, 2006; Jones, Homer, Murphy, et.al., 2006) .

The Structure of the Diabetes System
The group’s first step toward assessing the dynamics of the diabetes system was to develop a structural stock-and-flow diagram. Such a diagram specifies how population groups accumulate in several states of health (such as people with prediabetes, uncomplicated diabetes, and complicated diabetes) along with the rates at which people flow from one health state to the next (Jones, Homer, Murphy, et.al., 2006). The full model contains many such states and rates; however, Figure 18 shows only a simplified and generic view for explanatory purposes.

Figure 18 shows how changes in the diagnosed prevalence of any disease, not only diabetes, may be depicted. One may think of the box labeled diagnosed prevalence as a bathtub, with the level of water representing the number of people who have been diagnosed with a disease (Booth-Sweeney and Sterman, 2000). The rate at which people are being diagnosed (or diagnosed onset) is analogous to the rate at which water flows from a faucet into the tub, and the rates of recovery or death for people with diagnosed disease are analogous to the rates at which water flows out of the tub through two separate drains. As the figure indicates, all changes in diagnosed prevalence must be accounted for by changes in the related flows. The flows of births, migration, deaths among those without the disease, as well as deaths among the undiagnosed group are relevant, but for clarity, are not depicted in the figure.

Figure 18: Generic Stock and Flow Structure for Diagnosed Prevalence of a Disease

What do we know about the elements of Figure 18 with respect to diabetes? The following is a summary of the evidence that the team was able to compile.

• Diagnosed Prevalence: Historical data for 1980-2003 are in Figure 17. In 2000, about
• 12.0 million people of all ages in the United States had diagnosed diabetes, of whom 98% were adults aged 20 and over. This translates to about 4.4% of the total population and 6.0% of the adult population (Centers for Disease Control and Prevention, 2005b; Honeycutt, Boyle, Broglio, et.al., 2003).
• Diagnosed Onset: About 880,000 cases of diabetes were newly diagnosed in 1997, and that figure rose to 1.1 million by 2000, of whom more than 96% were adults. This translates to a diagnosis rate among the adult population of about 5.2 per thousand in 2000 (Centers for Disease Control and Prevention, 2005b; Honeycutt, Boyle, Broglio, et.al., 2003).
• Recovery: Recovery is a significant factor for many acute illnesses. But for diabetes, as for all chronic diseases that lack a full and permanent cure, it is virtually non-existent.
• Death Among those Diagnosed: Diabetes, like other chronic diseases, has a relatively small annual death rate. In 2000, of the 12.0 million people with diagnosed diabetes, about 500,000 (4.2%) died (Honeycutt, Boyle, Broglio, et.al., 2003), including approximately 213,000 deaths (a rate of about 1.8% per year) attributed to complications of the disease (Centers for Disease Control and Prevention, 2005b).
• Undiagnosed Prevalence: Since 1976, a random sample of participants in the periodic National Health and Nutrition Examination Survey (NHANES) without a diagnosis of diabetes were selected for a blood glucose test (Kenny, Aubert, Geiss, 1995). By dividing the number of people found to have diabetes by the total number of people tested, researchers estimated the fraction of Americans with diabetes whose disease was undiagnosed for each of the following NHANES periods: 1976-1980: 38%; 1988-1994: 36%; and 1999-2000: 29%(Gregg, Cadwell, Cheng, et.al., 2004).
• Initial Onset: There is no actual measure of the rate of initial onset of diabetes (i.e., the number of people per year who develop diabetes) as opposed to the rate of diabetes  diagnosis. However, the team estimated the initial onset rate by combining the data  described above on diagnosed prevalence, undiagnosed prevalence, and death using  the causal logic of Figure 18. According to their estimates, in 2000, 1.25 million U.S.  adults experienced the initial onset of diabetes, a rate of 6.0 people per thousand.
• Population without Diabetes: This category includes people with normal glycemic levels,  as well as people whose moderately elevated blood sugar qualifies them as prediabetic.  Based on blood testing data from NHANES 1988-1994, about 40% of Americans  aged 40-74 have prediabetes (Centers for Disease Control and Prevention, 2005b;  Expert Committee on the Diagnosis and Classification of Diabetes Mellitus, 2003).  Extrapolating to the rest of the adult population (taking into account estimated differences in prediabetes prevalence for ages 18-39 and 75+, based on historical data on age-specific diabetes incidence (14), and projecting forward in time, the team estimated that at least 52 million (or 25%) of American adults 18 and over were prediabetic in the year 2000.

Exploring Scenarios for the Future
The SD model tracks the flows and accumulations of people with normal glycemic levels, undiagnosed or diagnosed prediabetes, undiagnosed or diagnosed diabetes without complications, and undiagnosed or diagnosed diabetes with complications. The model specifies key factors—some of them potentially amenable to policy intervention—that may change over time and that affect the model’s population flows. These variable policy factors include the prevalence of obesity (i.e., the leading modifiable risk factor for diabetes); the prevalences of glycemic screening, prediabetes management, and diabetes management; as well as the percentage of the population with access to preventive care (Jones, Homer, Murphy, et.al., 2006). A “scenario” involves specifying future values of each variable factor. The model may then be simulated to explore the consequences of any given scenario for future trajectories of diagnosed prevalence and other measures of disease burden.

A Status Quo Future
Results from the team’s first simulation experiment focused on a status quo future, in which it is assumed there are no further changes, starting in 2004, in the scope or effectiveness of prevention, detection, or management efforts, nor in population obesity.

In Figure 19, the line marked “status quo” (from point F to G) shows that diagnosed prevalence increased steadily from 1990 to 2003.

A straightforward comparison of the estimates of inflow (diagnosis) and outflow (death) explains why the upward trend in diabetes prevalence, which began around 1990, will not soon abate. If the diagnosed onset rate in 2000 of approximately 1.1 million cases per year and the death rate of about 500,000 per year were to stay the same, the diagnosed prevalence would continue to increase. Although the model suggests that this gap between inflow and outflow is gradually closing, the inflow of diagnosed diabetes onset would have had to drop substantially (e.g., by about 50% in 2006) just for diagnosed prevalence to stop increasing, let alone to begin decreasing.

Accounting for Program/Policy Interventions
The SD model revealed certain insights about the long-term effects of interventions to reduce onset, boost detection, or better manage the disease. For example, aside from prevalence, another HP 2010 objective calls for the diagnosed percentage of cases of diabetes to increase from 68% to 80% (Objective 5-4). Such an increase in detection would, in terms of Figure 18, increase the flow of people being diagnosed, and thereby increase diagnosed prevalence above what it otherwise would have been. Figure 19 quantifies the effect of this scenario as the difference between points H and G (i.e., 63.2 vs. 59.1 in the year 2010).

Yet another HP 2010 objective calls for an 11% reduction in the diabetes-attributable death rate (Objective 5-6), a result presumably to be achieved through improved disease management. The structure in Figure 18 indicates that a reduction in the outflow of people dying also increases diagnosed prevalence: as the outflow drain becomes smaller more people with the disease remain alive (i.e., stay in the bathtub). Figure 19 does not display a curve for this scenario because it overlaps the status quo line (i.e., 59.3 vs 59.1 in the year 2010). The inconsistency in these HP 2010 objectives for diabetes is clear: meeting the objectives for increasing the diagnosis rate or decreasing the mortality rate would, in both cases, tend to increase the prevalence of diagnosed diabetes further upward and away from its Healthy People 2010 objective.

Figure 19: Diagnosed Diabetes Prevalence per Thousand Total Population—United States, 1985—2003^a, with Healthy People Objectives 2000 and 2010^b, and Simulation Model Output 2003—2010^c

One type of public health intervention that might reduce diagnosed prevalence is an effort to reduce initial onset of diabetes. Healthy People 2010 calls for a 29% reduction in the number of new cases per thousand (Objective 5-2), presumably to be achieved through a combination of efforts to detect and manage prediabetes, perhaps combined with efforts to reduce the leading modifiable risk factor for prediabetes and diabetes, namely obesity. A reduction in the flow of initial disease onset is clearly a move in the right direction because it leads to a lower diagnosed prevalence than would be the case under the status quo (i.e., without an intervention to reduce onset of disease). But a reduction in diagnosed prevalence relative to the status quo is not the same as an absolute reduction over time—an actual reversal of growth. The previous comparison of the inflow and outflow rates in Figure 18 indicated that a reduction in onset on the order of 50% would be required to halt the growth in diagnosed prevalence. Still, one may ask, to what extent could a 29% reduction in onset at least slow the growth of diagnosed prevalence?

To address this question, the group simulated an intervention starting in 2003 that by 2010 reduces diabetes onset 29% below its 1997 level. The effect on diagnosed prevalence is shown in Figure 19 as the line labeled “meet onset objective.” From 2003 to 2010, diagnosed prevalence per 1,000 population increases by 7% (from F to I), as opposed to increasing by 21% (from F to G) in the status quo scenario (i.e., 52.3 vs. 59.1 in the year 2010). Slower growth certainly signifies improvement but may disappoint those expecting an absolute decline in prevalence following such an ambitious and successful effort to reduce initial onset.

Thus, the simulation model helps quantify what the stock-and-flow logic of Figure 18 and numerical analysis suggested previously: namely, that the 29% target is too modest a reduction in onset to achieve the desired reduction in prevalence and can only slow the growth of prevalence. The simulation model can also be used to explore more extreme possibilities. For example, what would happen if initial onset were to drop suddenly to zero during 2004? The simulation model suggests that even under this impossible-to-achieve scenario, diagnosed prevalence would fall only by 14% from 2003 to 2010 (data not shown). This relatively modest reduction occurs in part because of some continued new diagnosis during this period (diagnosis may continue even though initial onset has ceased) and in part because of the relatively small death rate among people with diabetes (only about 4% per year). The 14% reduction in diagnosed prevalence during 2003-2010 in this most optimistic scenario still falls far short of the 38% objective in Healthy People 2010.

Learning to Chart Plausible Paths
Based on their innovative analyses, the team concluded that the Healthy People 2010 objective for reducing diagnosed diabetes prevalence by 38% will not be achieved—not because of ineffective or under-funded health protection efforts, but rather because the objective itself is unattainable given the historical processes under way. Moreover, if current investments in diabetes screening and disease management do succeed in diagnosing a greater fraction of the undiagnosed and in enabling people to live longer lives with the disease, then diagnosed prevalence will move still farther away from the Healthy People 2010 target.

When called upon to set long-range numerical targets for health indicators, particularly those that may be viewed as intervention outcomes, it is important to recognize that the diagnosed prevalence metric is prone to misinterpretation and unrealistic expectations. There are two basic reasons for this difficulty.

1. The task of setting plausible prevalence objectives requires an understanding that the growth in prevalence of many chronic diseases can, at best, be slowed and reversed only gradually. This is because the outflow of death is small relative to the inflow of disease onset (perhaps, as in the case of diabetes, because of a decades-long increase in the at-risk population), and there is no significant outflow of recovery. Therefore, the task of reducing prevalence is like attempting to return a fast-moving train to a station that it passed miles back: the first requirement is to slow down, not reverse direction.

2. Furthermore, successful interventions to increase disease detection and management result in people living longer with their disease rather than dying prematurely of it. But by increasing detection and extending life, such interventions also have the effect of increasing diagnosed prevalence. The only practical way of slowing (let alone reversing) the growth in diagnosed prevalence is through health protection programs that reduce initial disease onset. However, initial onset must not only decline, but must fall far enough to more than offset the increase due to improved detection and management. Prevalence objectives will not be achievable unless this fact is taken into account.

If prevalence objectives are to be attainable within their specified time frame, it is important first to recognize what the future trajectory would be under status quo assumptions and then to factor in the effects of any planned interventions, recognizing that some may undercut the effects of others. In the case of diabetes, the team found that current conditions—without any new interventions—would drive diagnosed prevalence to increase another 21% from 2003 to 2010. Current detection and care initiatives, if successful, will increase that number even further. If the emphasis on reducing prevalence is intended to help assess the performance of those interventions working to reduce diabetes onset, then planners wanting to set future targets should take as a starting point both the status quo future and the compounding effects of successful detection and control interventions.

Recognizing the Benefits of Formal Modeling
Simulation modeling helps improve our collective understanding of health and disease dynamics, and in turn, supports the development of long-range objectives that are both achievable and mutually consistent. Such models enable planners and policymakers to explore for themselves the plausible short- and long-term consequences of historical trends and compare the effects of alternative interventions before committing limited resources. For that reason, diabetes program planners in Vermont, Minnesota, California and other states are now working with members of the CDC Diabetes System Modeling team to use the model described here as a support for their efforts to set plausible and internally consistent objectives for diabetes-related outcomes at the state level (Edelman, 2006; Murphy, Homer, Nanavati, et.al., 2006). Planners in Minnesota, California, Alabama, Tennessee, and Florida are currently exploring similar uses.

Without the reality checks available through formal system science methods, long-range target setting may fall prey to the weaknesses of flawed and sometimes biased intuition or mental models (Booth-Sweeney and Sterman, 2000; Sterman, 2000). Intuition may often neglect real-world sources of inertia and delay and suggest that things can change more rapidly than is actually possible. The prevalence of a chronic disease like diabetes changes only gradually, because, as noted above, its outflow of death is relatively small and recovery is not possible. In this respect, chronic diseases are unlike many acute infectious diseases such as influenza or measles, where patients do not linger in the disease condition for years, but instead either recover or die relatively quickly. For such acute diseases, the large outflow creates a close correlation between drops in onset and drops in diagnosed prevalence. For chronic illnesses, however, drops in onset do not correlate with immediate drops in prevalence; instead, they correlate with prevalence increasing more slowly.

Those working to prevent and manage chronic diseases may use stock-and-flow diagrams to develop a clearer understanding of the characteristic dynamics of these diseases. In addition, simulated policy experiments may bring new insights to the task of charting a viable course for the nation's health. That approach could help ensure that numerical objectives are mutually consistent and achievable within their stated time frames. The objectives may still be difficult to achieve in practice, and in that sense may be aspirational; but even aspirational objectives can and should be crafted in a way that is consistent, logical, and feasible given the causal structure of the system and the historical processes under way.

Although simulation models can help improve our understanding of chronic disease dynamics, they have several inherent limitations. All models are incomplete simplifications of reality and their conclusions are affected both by structural boundaries and the uncertainties of the data with which they are calibrated (Sterman, 2002). Techniques such as boundary critique (Ulrich, 2002) and sensitivity testing (Sterman, 2002) can be used to assess the extent to which model findings may be affected by those simplifications and uncertainties. In the case of the diabetes SD model, sensitivity testing suggests that the magnitudes of its simulated futures, such as those seen in Figure 19, are subject to some imprecision due to uncertainties about input parameters, but that the directions of change and, thus, the general findings, are unaffected by those uncertainties.

Even with their inevitable imprecision and incompleteness, however, the formal tools of system dynamics can enhance learning and decision-making, and that is their primary purpose (Sterman, 2000, 2006). In particular, these tools can improve our collective intuition about how interventions will affect health indicators over many years within the complex systems of cause and effect that shape the public’s health.
 

42. Material for this section comes from an ongoing collaborative project known as the CDC Diabetes System Modeling Project (involving Jack Homer, Drew Jones, and Don Seville as the lead system dynamics modelers, along with Joyce Essien, Dara Murphy, myself, and many others as the CDC participants). Additional details on the project can be found in (Homer, Jones, Seville, et.al., 2004; Jones, Homer, Murphy, et.al., 2006). The text for this section is adapted from (Milstein, 2007 #5713).

Previous | Next

Back to top
 

Page last reviewed: January 30, 2008
Page last modified: January 30, 2008

Content source: Division of Adult and Community Health, National Center for Chronic Disease Prevention and Health Promotion