Volume 7: No. 3, May 2010

ORIGINAL RESEARCH
Evaluating an Insurance-Sponsored Weight Management Program With the RE-AIM Model, West Virginia, 2004-2008

	TABLE OF CONTENTS
	Este resumen en español Print this article E-mail this article: Send feedback to editors Download this article as a PDF (555K) You will need Adobe Acrobat Reader to view PDF files.
	Navigate This Article • Abstract • Introduction • Methods • Results • Discussion • Acknowledgments • Author Information • References • Tables • Appendix

Christiaan G. Abildso, PhD, MPH; Sam J. Zizzi, EdD; Bill Reger-Nash, EdD

Suggested citation for this article: Abildso CG, Zizzi SJ, Reger-Nash B. Evaluating an insurance-sponsored weight management program with the RE-AIM model, West Virginia, 2004-2008. Prev Chronic Dis 2010;7(3):A46. http://www.cdc.gov/pcd/issues/2010/
may/09_0114.htm. Accessed [date].

PEER REVIEWED

Abstract

Introduction
Evaluations of weight management programs in real-world settings are lacking. The RE-AIM model (reach, effectiveness, adoption, implementation, maintenance) was developed to address this deficiency. Our primary objective was to evaluate a 12-week insurance-sponsored weight management intervention by using the RE-AIM model, including short-term and long-term individual outcomes and setting-level implementation factors. Our secondary objective was to critique the RE-AIM model and its revised calculation methods.

Methods
We created operational definitions for components of the 5 RE-AIM indices and used standardized effect size values from various statistical procedures to measure multiple components or outcomes within each index. We used χ² analysis to compare categorical variables and repeated-measures analysis of variance to assess the magnitude of outcome changes over time.

Results
On the basis of data for 1,952 participants and surveys completed by administrators at 23 sites, RE-AIM indices ranging from 0 to 100 revealed low program reach and adoption (5.4 and 8.8, respectively), moderate effectiveness (43.8), high implementation (91.4), low to moderate individual maintenance (21.2), and moderate to high site maintenance (77.8). Median (interquartile range) weight loss was 13 lb (6.5-21.4 lb) among participants who completed phase I (12 weeks; 76.5%) and 15 lb (6.1-30.3 lb) among those who completed phase II (1 year; 45.7%).

Conclusions
This program had a significant, positive effect on participants and has been sustainable but needs to be expanded for more public health benefit. The RE-AIM model provided a useful framework to determine program strengths and weaknesses and to present them to the insurance agency and public health decision makers.

Back to top

Introduction

Regardless of the success of diet and physical activity programs in controlled research settings (1-6), if large-scale programs are not effective or easily implemented in real-world settings they fall short of their intended purpose. In the Strategic Plan for NIH Obesity Research, the National Institutes of Health Obesity Research Task Force (7) outlined a national strategy for obesity research emphasizing the use of translational research. However, evaluation of the translation of effective programs to real-world settings is lacking (8).

The RE-AIM model (9) was developed as a tool to evaluate health promotion programs along 5 dimensions: reach, effectiveness, adoption, implementation, and maintenance. Reach refers to the percentage of potential participants who enroll and their representativeness of the target population; effectiveness, to the changes in participant outcomes during the program; adoption, to the number and representativeness of sites willing to conduct the program; implementation, to site adherence to program protocols; and maintenance, to individual outcomes after final intervention contact and program continuation at sites over the long term (8-10).

The RE-AIM model has been used to evaluate a limited number of programs, including 14 WISEWOMAN cardiovascular screening interventions in North Carolina (11), a church-based physical activity intervention in South Carolina (12), and 2 diabetes self-management interventions (13). To our knowledge, no evaluation of a large-scale weight management program using RE-AIM methods has been published, but studies have shown the model is applicable to a variety of health behaviors and programs. Additionally, researchers have recommended revising RE-AIM index calculation methods to reduce the likelihood of difficult-to-interpret negative values (13).

Our primary objective was to use RE-AIM to evaluate a multisite, insurance-sponsored weight management program that produced positive short-term physical and psychosocial changes in participants in a university laboratory setting. Our secondary objective was to critique the RE-AIM model’s usefulness as an evaluation tool for health promotion programs. We analyzed 3- and 12-month participant outcomes and site-level program implementation information.

Back to top

Methods

This study evaluated a weight management program that has been offered since 2004 by a large public employees’ insurance agency in West Virginia as a benefit to its members. Obese insured members (body mass index [BMI] ≥30 kg/m²) are eligible for the program, as are overweight members (BMI 25.0-29.9 kg/m²) who also report being treated for hypertension, diabetes, heart disease, metabolic syndrome, or sleep apnea. Participants enroll in the program by calling the insurance agency to be screened for height, weight, and health information and must have written approval from their primary care physician.

The insurance agency also screens site providers to ensure the site has adequate safety devices, exercise equipment, and staffing to accommodate high-risk participants. At the time of this study, the insurance agency had approved 31 fitness facilities (“sites”) to accept participants: 2 housed in university cardiac rehabilitation/exercise physiology laboratories, 2 in physical therapy facilities, 5 in hospitals, and 22 private wellness/fitness centers. Searches of available print (14) and online (www.wvha.com, www.yellowpages.com) resources yielded 352 of these types of facilities in West Virginia. Each approved site is required to designate an administrator to act as liaison with the insurance agency, handle participant contact, coordinate site services, and enter participant data into a database.

The weight management program evaluated in this study is a 1-year benefit, during which weight loss is promoted by using a combination of behavior therapy (eg, food logs), individualized site-based exercise programming, and individualized diet therapy. Case management nurses track participant progress on the basis of monthly entry of the following body measurements by the sites into a Web-based database: weight and BMI, systolic and diastolic blood pressure, body fat percentage, waist circumference, and number of site visits. Each site determines its measurement protocols on the basis of available instruments and staff training. Sites are compensated via monthly member copayments and insurance agency payments for the provision of on-site services in 2 phases.

Phase I (months 1-3)

Services provided during phase I include 1) access to the exercise site, 2) a 60-minute fitness evaluation and exercise program designed by an exercise physiologist, 3) a 60-minute evaluation and nutrition program designed by a registered dietitian, 4) monthly 30-minute exercise sessions with a personal trainer, and 5) 60-minute follow-up reassessments by the exercise physiologist and registered dietitian at the end of month 3. Participants may be removed from the program if they fail to exercise at the site at least twice per week; turn in food logs periodically; attend appointments with the exercise physiologist, registered dietitian, or personal trainer; or have body measurements taken by site staff monthly. Phase I is considered the “intervention” period for this evaluation. If participants meet the behavioral requirements and lose 12 lb (or show substantial improvements in other physical measures) they may continue to phase II.

Phase II (months 4-12)

Phase II spans the remainder of the first year (9 months). This phase is considered the postintervention period for the current evaluation because it is more like a private gym membership than a weight-loss intervention; it provides access to the facility and 1 personal training session per month. Participants are expected to continue exercising at the facility twice per week and have physical measurements and visit data submitted monthly by the sites to maintain the benefit.

The insurance agency dictates what services are offered, but not how the services are implemented. The use of relevant weight management guidelines of the American College of Sports Medicine (ACSM) (15) and the American Dietetic Association (16) is suggested. For example, the exercise physiologist may choose from fitness testing methods recommended by ACSM (eg, 6-minute walk test, 1-mile walk test) and the registered dietitian may use his or her discretion in diet composition or outcome goals.

Data sources

We used archival participant data and site administrator surveys to evaluate the weight management program. The insurance agency granted us access to an online password-protected database containing demographic and physical measurement data of all program participants. The university’s institutional review board approved the study for the protection of human participants. We downloaded, verified, de-identified, and imported archival data of all participants into SPSS version 14.0 (SPSS, Inc, Chicago, Illinois).

To gather site-level program evaluation data, we recruited site administrators by mail, e-mail, and telephone (17) to complete an online survey. The survey assessed 1) site staffing, equipment, and facilities; 2) program implementation methods; and 3) barriers to program implementation. Section 2 of the survey was relevant to this study; it included items that ascertained the procedures (open-ended) or frequency (categorical) of conducting 14 program components during phase I as recommended or required by the insurance agency: measuring weight, waist circumference, body fat percentage, and blood pressure; providing individualized nutrition and fitness assessments, nutrition classes, fitness plan, one-on-one personal training, and follow-up nutrition and fitness assessments; tracking participant attendance; and reviewing home exercise and food logs or journals. Responses were recoded to dichotomous variables (yes/no) and the sum of yes responses tallied to represent the number of program components implemented.

Statistical analyses of RE-AIM indices

Presenting results using the RE-AIM model first requires creating operational definitions for multiple components in each of the 5 RE-AIM dimensions (10) (Box). These components were used to calculate RE-AIM indices (Appendix). Index calculations used standardized effect size (18) values from various statistical procedures to measure multiple components or outcomes in each index. We used χ² analysis to compare categorical variables and repeated measures analysis of variance (ANOVA) to assess the magnitude of outcome changes over time. In the indices, “positive” effects were reduced by differential or “negative” effects (eg, program attrition). This index calculation method has resulted in negative values in prior studies (13) for 2 reasons. First, in calculating index values, differential effects have been subtracted directly from outcome effects, overstating the “negative” program effect. To limit this potential distortion, we calculated a “proportion of positive effect not explained by differential effects” by first subtracting the differential (“negative”) effect size from 1 and then multiplying the difference by the “positive” effect (Appendix). Second, a negative index value may still result if the effect size is more than 1. We resolved this challenge by using effect size measures whose upper limit was 1.

Effect size measures (and interpretation) included the squared Cramer phi (φ_c²) for χ² tests (small [φ_c² ≈ .01], moderate [φ_c² ≈ .09], or large [φ_c² > .25]) and the squared curvilinear correlation coefficient (partial eta squared; η²) for ANOVA (small [η² ≈ .01], moderate [η² ≈ .06], or large [η² > .14]) (18,19). All raw index scores ranging from 0 to 1 were multiplied by 100 for summary index values of 0 to 100. Missing data were excluded from analyses, which were conducted by using SPSS version 14.0.

Back to top

Results

The data for 1,952 participants from 31 sites were exported, and administrators from 27 of 31 active sites (87.1% response rate) completed online surveys. Responses from 4 new sites whose participants had not yet completed phase I of the program were excluded from analysis, resulting in 23 completed site administrator surveys. The distribution of most of the participants’ measurement data was skewed, and data are presented as median (interquartile range) unless otherwise noted.

Individual-level impact ([reach * effectiveness] / 100)

A total of 60,041 adult members were covered by the insurance agency (N. Henderson, health promotions director, written communication, January 2008). By using the state obesity (31%) and overweight (36%) prevalence rates (20), and national prevalence of overweight adults with at least 1 comorbid condition (66%) (21), we estimated that 32,878 insurance members may have been eligible for the program. At the time of data collection, 1,952 members had participated (5.9% individual participation rate) (Table 1).

Baseline physical measurements for men and women are presented in Table 2. A significantly larger percentage of participants than the eligible population were women (80.0% vs 54.1%; P < .001, φ_c² = .073), and more of them were aged 45-54.9 years and fewer of them were aged 65 years or older (P < .001, φ_c² = .113), yielding a reach value (R) of 5.4 (Table 1).

Phase I outcome changes are presented in Table 3. Of 1,647 participants who could have completed phase I (those who had started the program at least 12 weeks before data collection), 76.5% had done so. Fewer women (74.8%) than men (83.7%) completed phase I (P = .001, φ_c² = .007), and completion rates tended to rise with age. Participants who completed phase I had significant weight loss (13 lb [6.5-21.4 lb]; P < .001, η² = .592), and men lost significantly more weight than women (P < .001, η² = .050). Differences in weight loss were also revealed among age groups (P = .003, η²= .014). These components produced an effectiveness value (E) of 43.8 and an individual-level impact ([R * E] / 100) of 2.4 (Table 1).

Setting-level impact ([adoption * implementation] / 100)

There were 31 active weight management program sites out of a total of 352 potential sites in West Virginia at the time of data collection, resulting in an adoption value (A) of 8.8 (Table 1). Site survey responses (N = 23) showed that sites implemented a mean of 12.8 (standard deviation [SD] 1.0) of 14 program components during phase I. All sites measured weight, waist circumference, body fat percentage, and blood pressure; tracked attendance; and provided initial and follow-up nutrition assessments with a registered dietitian, nutrition classes designed by the registered dietitian, and an individualized exercise prescription. The least frequently implemented component, by 15 sites (65%), was having participants maintain home exercise logs. Implementation survey data yielded an implementation value (I) of 91.4 and a setting-level impact ([A * I] / 100) of 8.0 (Table 1).

Long-term maintenance ([individual maintenance * setting maintenance] / 100)

Phase II outcome changes are presented in Table 4. Of 762 participants who could have completed phase II (those who had started the program at least 1 year before data collection), 348 (45.7%) had done so (ICR_PhII = .457) (Table 1). Excluding the small samples of participants aged 18-24.9 years (n = 2) and 65 years or older (n = 9), completion rates tended to increase with age and be higher in men in each age group. Participants who completed phase II achieved significant weight loss from baseline (15 lb [6.1-30.3 lb]; P < .001, η² = .467), shedding 6.7% (2.7%-12.7%) of baseline body weight. Weight loss was similar among phase II completers of different age groups (P = .61, η² = .011) and between sexes (P = .21, η² = .005). These values combined to yield an individual maintenance value (M_I) of 21.2 (Table 1).

Four of 18 sites (22%) that had been approved to accept participants at least 1 year before this study had stopped or had been disallowed to continue accepting participants, resulting in a setting maintenance value (M_S) of 77.8 and a long-term maintenance value ([M_I * M_S] / 100) of 16.5 (Table 1).

Back to top

Discussion

We achieved our primary objective of using RE-AIM to evaluate a weight management program. We found moderate program effectiveness and high implementation, suggesting the program has been beneficial for participants and can be implemented in a variety of settings. We found low program reach and adoption, suggesting the program could be improved by recruiting new participants and sites. Recruitment may prove difficult, however, because participants must be highly motivated to enroll in the program, and sites are required to offer services by highly trained personnel often unavailable in rural areas of West Virginia. In the long term, site maintenance was high, but individual maintenance was fairly low, indicating the program is sustainable but the services of phase II may need to be revised to improve participant outcomes. The summary results suggest this weight management program has potential to be expanded for more translation and public health benefit and should be considered a viable model for other public and private insurers.

Individual short-term and long-term outcome changes are comparable with those of other behavioral programs and clinical trials. Short-term attrition from this weight management program (23.5%) was slightly higher than is commonly seen in behavioral programs of similar length (10%-15%), though the median weekly weight loss in this program is comparable (1.23 lb vs 1.1 lb) (22). This program also compares favorably with clinical trials of similar length, which average 85%-95% completion rates and approximately 1 lb of weekly weight loss (1,5).

Long-term individual results also compare favorably with other behavioral programs and clinical trials. Participants who completed phase II (n = 348) lost a mean of 20.9 (SD, 22.3) lb from baseline, with some recidivism. Slightly more than half of phase II completers (51.5%) maintained phase I weight loss or continued losing weight in phase II. In comparison, approximately 60% to 70% of weight loss is maintained for a year after treatment in other short-term behavioral interventions (22). One-year results indicate more average weight losses but lower completion rates than randomized control trials of similar length (1,5).

We also achieved the secondary objective of critiquing the RE-AIM model and revised index calculations. The strength of the RE-AIM model is its ability to quantify for decision makers a program’s strengths and weaknesses. Comparison with other health promotion program evaluations that used RE-AIM is limited at this point in the model’s refinement because no 2 studies have used the same index calculation methods. We believe this study advances the RE-AIM model by 1) providing methods for assessing long-term maintenance at the individual and site levels and 2) addressing 2 methodologic concerns with existing index calculation methods (ie, negative index values and effect sizes with varying maximum values). The revised methods in this study produced positive R and M_I values, whereas previously used methods (10) would have yielded negative values.

Limitations

The study is limited in a number of ways. Multiple sources of measurement error may have affected the data, including 1) lack of standardized procedures and instruments for measuring health outcomes, 2) missing outcome data, and 3) social desirability of sites when entering participant data and completing survey items. Potential error was addressed in multiple ways. Trained exercise professionals took measurements using accepted professional standards of practice, the insurance agency periodically audited site data, we contacted sites to collect missing data, and all survey recruiting material stressed the informative (not punitive) nature of the study and independence of the investigators from the insurance agency. Measurement error would be more important in a small clinical trial assessing an intervention’s efficacy than in this study with its large sample and focus on standardized effect sizes.

Conclusions

Numerous questions remain to be answered before the public health effect of this and other weight management programs can be better understood. This program had a significant positive effect on participants and was sustainable, but needs to be expanded. The RE-AIM model provided a framework through which the translation of this program could be evaluated and presented to public health decision makers. In fact, the summary data of this project were presented to the insurance agency and used as evidence for changing the program benefit to address the low reach, adoption, and long-term individual maintenance. We encourage continued study of RE-AIM index calculations and application of the model to evaluate obesity treatment programs. It may be beneficial to add qualitative process data to these outcomes to work toward developing a set of best practices in behavioral weight management (11,12). Additionally, because insurance agencies often provide benefits for multiple weight management modalities (eg, behavioral, surgical, pharmacologic), RE-AIM methods should be used to evaluate multiple modalities concurrently to allow for side-by-side comparison and facilitate decision making about resource allocation. These evaluations will need to link program costs, insurance claims, individual outcomes, and future cost savings. Such analyses may affect the benefit and incentive structure of this insurer and others to improve the translation of knowledge regarding clinical obesity treatment into innovative programming that can be widely implemented.

Back to top

Acknowledgments

This research was funded by the State of West Virginia Public Employees Insurance Agency.

Back to top

Author Information

Corresponding Author: Christiaan G. Abildso, PhD, MPH, West Virginia University College of Physical Activity and Sport Sciences, PO Box 6116, Morgantown, WV 26506-6116. Telephone: 304-293-0860. E-mail: cgabildso@mail.wvu.edu.

Author Affiliations: Sam J. Zizzi, Bill Reger-Nash, West Virginia University, Morgantown, West Virginia.

Back to top

References

1. Andersen RE, Wadden TA, Bartlett SJ, Zemel B, Verde TJ, Franckowiak SC. Effects of lifestyle activity versus structured aerobic exercise in obese women: a randomized trial. JAMA 1999;281(4):335-40.
2. Dunn AL, Garcia ME, Marcus BH, Kampert JB, Kohl HW, Blair SN. Six-month physical activity and fitness changes in Project Active, a randomized trial. Med Sci Sports Exerc 1998;30(7):1076-83.
3. Dunn AL, Marcus BH, Kampert JB, Garcia ME, Kohl HW III, Blair SN. Comparison of lifestyle and structured interventions to increase physical activity and cardiorespiratory fitness: a randomized trial. JAMA 1999;281(4):327-34.
4. Neter JE, Stam BE, Kok FJ, Grobbee DE, Geleijnse JM. Influence of weight reduction on blood pressure: a meta-analysis of randomized controlled trials. Hypertension 2003;42(5):878-84.
5. Wadden TA, Berkowitz RI, Womble LG, Sarwer DB, Phelan S, Cato RK, et al. Randomized trial of lifestyle modification and pharmacotherapy for obesity. N Engl J Med 2005;353(20):2111-20.
6. Wadden TA, Vogt RA, Andersen RE, Bartlett SJ, Foster GD, Kuehnel RH, et al. Exercise in the treatment of obesity: effects of four interventions on body composition, resting energy expenditure, appetite, and mood. J Consult Clin Psychol 1997;65(2):269-77.
7. Strategic plan for NIH obesity research. Washington (DC): US Department of Health and Human Services, National Institutes of Health Obesity Research Task Force; 2004.
8. Glasgow RE, Klesges LM, Dzewaltowski DA, Bull SS, Estabrooks P. The future of health behavior change research: what is needed to improve translation of research into health promotion practice? Ann Behav Med 2004;27(1):3-12.
9. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999;89(9):1322-7.
10. Glasgow RE, Klesges LM, Dzewaltowski DA, Estabrooks PA, Vogt TM. Evaluating the impact of health promotion programs: using the RE-AIM framework to form summary measures for decision making involving complex issues. Health Educ Res 2006;21(5):688-94.
11. Farris RP, Will JC, Khavjou O, Finkelstein EA. Beyond effectiveness: evaluating the public health impact of the WISEWOMAN program. Am J Public Health 2007;97(4):641-7.
12. Bopp M, Wilcox S, Laken M, Hooker SP, Saunders R, Parra-Medina D, et al. Using the RE-AIM framework to evaluate a physical activity intervention in churches. Prev Chronic Dis 2007;4(4). http://www.cdc.gov/pcd/issues/2007/oct/06_0155.htm. Accessed November 17, 2007.
13. Glasgow RE, Nelson CC, Strycker LA, King DK. Using RE-AIM metrics to evaluate diabetes self-management support interventions. Am J Prev Med 2006;30(1):67-73.
14. West Virginia Department of Health and Human Resources Health Statistics Center. The burden of cardiovascular disease in West Virginia; 2002. http://www.wvdhhr.org/bph/oehp/hsc/burdencvd/default.htm. Accessed November 17, 2007.
15. American College of Sports Medicine. ACSM’s guidelines for exercise testing and prescription. 7th edition. Hagerstown (MD): Lippincott, Williams, and Wilkins; 2005.
16. Seagle H, Strain GW, Makris AP, Reeves RS. Position of the American Dietetic Association: weight management. J Am Diet Assoc 2009;109(2):330-46.
17. Dillman DA. Mail and internet surveys. 2nd edition. New York (NY): Wiley; 2000.
18. Cohen J. Statistical power analysis for the behavioral sciences. 2nd edition. Hillsdale (NJ): Lawrence Erlbaum Associates; 1988.
19. Witte RS, Witte JS. Statistics. 6th edition. Orlando (FL): Harcourt, Inc; 2001.
20. Behavioral Risk Factor Surveillance System: 2006 prevalence data. Centers for Disease Control and Prevention. http://apps.nccd.cdc.gov/brfss/. Accessed November 2, 2007.
21. Must A, Spadano J, Coakley EH, Field AE, Colditz G, Dietz WH. The disease burden associated with overweight and obesity. JAMA 1999;282(16):1523-9.
22. Brownell KD, Wadden TA. Etiology and treatment of obesity: understanding a serious, prevalent, and refractory disorder. J Consult Clin Psychol 1992;60(4):505-17.

Back to top

Tables

Table 1. RE-AIM Model Component and Index Values Used to Evaluate a Weight Management Program, West Virginia, 2004

Abbreviation: RE-AIM, reach, effectiveness, adoption, implementation, maintenance.
^a Methods of calculating the values are described in the Appendix.

Table 2. Baseline Measurements of Participants, Weight Management Program, West Virginia, 2004

Abbreviations: IQR, interquartile range; BMI, body mass index.
^a Sample sizes vary because of missing data for some measurements.
^b Median and IQR are presented for data with skewed distributions unless otherwise noted.
^c Measurement has a normal distribution and is presented as mean (standard deviation). 

Table 3. 12-Week Measurements and Changes From Baseline of Phase I Completers, Weight Management Program, West Virginia, 2004^a

Abbreviations: IQR, interquartile range; NA, not applicable; BMI, body mass index.
^a 1,647 participants could have completed phase I.
^b Sample sizes vary because of missing data for some measurements; negative change values indicate improvements from baseline to the end of phase I (12 weeks).
^c Median and IQR are presented for data with skewed distributions unless otherwise noted.
^d Measurement has a normal distribution and is presented as mean (standard deviation).

Table 4. One-Year Measurements and Changes From Baseline of Phase II Completers, Weight Management Program, West Virginia, 2004^a

Abbreviations: IQR, interquartile range; NA, not applicable; BMI, body mass index.
^a 762 participants could have completed phase II.
^b Sample sizes vary because of missing data for some measures; negative change values indicate improvements from baseline to the end of phase II (1 year).
^c Median and IQR are presented for data with skewed distributions unless otherwise noted.
^d Measurement has a normal distribution and is presented as mean (standard deviation).

Back to top

Appendix. RE-AIM Components, Indices, and Calculation Equations

Abbreviations: ES, effect size; φ_c², squared Cramer phi; η², partial eta squared; RM ANOVA, repeated measures analysis of variance; IV, independent variable; DV, dependent variable.

Back to top

The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.

Privacy Policy | Accessibility

CDC Home | Search | Health Topics A-Z

This page last reviewed March 30, 2012

Centers for Disease Control and Prevention National Center for Chronic Disease Prevention and Health Promotion

United States Department of Health and Human Services