|
|
Volume 5:
No. 4, October 2008
ORIGINAL RESEARCH
A New Evaluation Tool to Obtain Practice-Based Evidence of Worksite Health Promotion Programs
Diane O. Dunet, PhD, Phillip B. Sparling, EdD, James Hersey, PhD, Pamela Williams-Piehota, PhD, Mary D. Hill, BBA, Carl Hanssen, PhD, Frances Lawrenz, PhD, Michele Reyes, PhD
Suggested citation for this article: Dunet DO, Sparling PB, Hersey J, Williams-Piehota P, Hill MD, Hanssen C, et al. A new evaluation tool to obtain practice-based evidence of worksite health promotion programs. Prev Chronic Dis 2008;5(4).
http://www.cdc.gov/pcd/issues/2008/
oct/07_0173.htm. Accessed [date].
PEER REVIEWED
Abstract
Introduction
The Centers for Disease Control and Prevention developed the Swift Worksite Assessment and Translation (SWAT) evaluation method to identify promising practices in worksite health promotion programs. The new method complements research studies and evaluation studies of evidence-based practices that promote healthy weight in working adults.
Methods
We used nationally recognized program evaluation standards of utility, feasibility, accuracy, and propriety as the foundation for our 5-step method: 1) site identification and selection, 2) site visit, 3)
post-visit evaluation of promising practices, 4) evaluation capacity building, and 5) translation and dissemination.
An independent, outside evaluation team conducted process and summative evaluations of SWAT to determine its efficacy in providing accurate,
useful information and its compliance with evaluation standards.
Results
The SWAT evaluation approach is feasible in small and medium-sized workplace settings. The independent evaluation team judged SWAT favorably as an evaluation method, noting among its strengths its systematic and detailed procedures and service orientation. Experts in worksite health promotion evaluation concluded that the data obtained
by using this evaluation method were sufficient to allow them to make judgments
about promising practices.
Conclusion
SWAT is a useful, business-friendly approach to systematic, yet rapid, evaluation that comports with
program evaluation standards. The method provides a new tool to obtain practice-based evidence of worksite health promotion programs that help prevent obesity and, more broadly, may advance public health goals for chronic disease prevention and health promotion.
Back to top
Introduction
The workplace setting is an important venue for health promotion practices among adults because most working-age men and women are in the workplace most days (1). Worker safety and health insurance benefits are focal points both for the Centers for Disease Control and Prevention (CDC) and for employers. CDC’s National Institute
for Occupational Safety and Health is the key federal entity working to prevent employee injury and illness, and CDC partners with the business community
as part of an overall strategy to achieve public health goals for adult health and quality of life through prevention (2). For example, as employers increasingly sought to promote employee health and reduce their own burden of health care costs through worksite health promotion programs, CDC partnered with the National Business Group on Health to develop A Purchaser’s Guide to Clinical Preventive Services (3).
Obesity affects employee health care costs (4), productivity (5), absenteeism
(6,7), injury (8), and chronic diseases (9). Well-executed studies reported in
the peer-reviewed scientific literature provide an evidence base for designing
worksite health promotion programs that may decrease the prevalence of obesity
and related chronic diseases. CDC undertakes
rigorous, systematic scientific reviews of published studies on obesity
prevention and control in conjunction with the Task Force on Community
Preventive Services, which publishes the Guide to Community Preventive Services
(10). CDC has also systematically reviewed secondary literature sources, such as
dissertations and evaluation reports, to identify potentially promising
practices that merit further attention and formal study.
Worksites that provide health promotion programs to their employees are
another source of information about interventions that may be
effective in addressing obesity. To better understand such programs, CDC
developed the Swift Worksite Assessment and Translation (SWAT) method to assess
worksites rapidly and to add to the body of what has been called “practice-based
evidence” (11). Research studies require strict adherence to protocols that
often intentionally hold an intervention static over time to control threats to
study validity.
However, organizations may create new health promotion strategies or adapt and customize research-based strategies to the demographics and health status of their workforce, available resources, or other factors. Current events and news media that focus on such issues as obesity also can stimulate new ideas.
Through professional networking and sources such as worksite health promotion award programs, CDC staff regularly become aware of “home-grown” interventions that are reputed to achieve desired health outcomes. An evaluation method is needed, however, to investigate these types of emerging practices, particularly to identify practices that merit more comprehensive, rigorous (and expensive) evaluation research. Worksite health promotion strategies already being implemented by employers are
obviously feasible, in contrast to research-based interventions that may not necessarily be adaptable to myriad workplace settings.
Most research studies of corporate worksite health promotion programs involve large companies.
However, more than 70% of adults in the US workforce are employed in organizations with fewer than 5,000 employees (12). Smaller organizations may have fewer resources available for evaluation; however, they may have the ability to change their policies or programs rapidly, unencumbered by the administrative systems of large organizations. The SWAT method was designed
specifically for evaluating organizations we defined as small (<300 employees) and medium-sized (<5,000 employees) because of the paucity of data in these settings. The method has since been adapted for large worksites.
Within this worksite health promotion context, we developed the SWAT approach as a middle-ground evaluation method that aims to be business-friendly while being solidly based in good evaluation practice. SWAT is not meant to quantify the success of worksite health promotion practices but rather to assess quickly whether they are worthy of more rigorous evaluation. This article discusses the process used to develop and test SWAT, describes the SWAT method in detail, discusses the
program
evaluation standards used to refine it, and briefly summarizes the conclusions of an outside evaluation of the SWAT method.
Back to top
Methods
Beginning in June 2005, the project team of researchers from CDC and RTI International identified nationally recognized experts in worksite health promotion, site-visit methods, and evaluation. Fourteen experts from academic, corporate, worksite health promotion, and public health settings were invited to consult on the project (see Acknowledgments).
During the next 3 months, the project team developed an initial SWAT framework
(Table 1). The work was guided by CDC’s Framework for Program Evaluation
in Public Health (13), the centerpiece of which is nationally recognized program evaluation
standards (14). The overall SWAT framework explicates inclusion and exclusion criteria for worksite programs, operational definitions, site selection criteria, and overall procedures for conducting site visits. The 14-member expert panel was invited to provide
additional feedback and recommendations on the SWAT framework in writing or by telephone conference at 2 critical points during the framework development process.
The SWAT development project plan was reviewed by the associate director for science of CDC’s National Center for Chronic Disease Prevention and Health Promotion, who determined it to be public health practice and not research subject to institutional review board approval for human subjects.
A notable feature of the SWAT development process is that the SWAT evaluation method was concurrently evaluated by Western Michigan University’s Evaluation Center (WMUEC) (15). The SWAT meta-evaluation is summarized in
Table 2. The SWAT framework was further refined based on the process recommendations and summative findings of the WMUEC
evaluation team.
The SWAT evaluation method was designed as a tool for single-worksite assessments and uses a set of criteria
(Table 3), rather than a comparison group, to judge the promise of worksite health promotion practices. CDC funding was available to conduct 9 initial SWAT site visits. Because these assessments were the first application of the SWAT method,
they informed further
refinement of the SWAT process. A detailed description of the initial 9 worksites assessed for their innovative practices is reported elsewhere in this issue of the journal (16).
As shown in Table 1, the SWAT evaluation process is organized into 5 phases: 1) site identification and selection, 2) site visit, 3) evaluation of health promotion practices, 4) evaluation capacity building, and 5) translation and dissemination. The overall approach of a SWAT assessment is a document review process and an on-site examination of self-reported summary data from worksites, supplemented by key informant interviews and observation. The combination of evaluation methods provides a
general understanding of the worksite health program and its major components, measured outcomes, and data quality.
After potential SWAT sites were identified by using methods described in Table 1, the site selection process began with a brief, structured telephone interview to compile a preliminary summary of a site. This information was used to determine site eligibility and the potential for a SWAT assessment to identify a promising practice.
Table 4 presents the eligibility criteria we used to limit the types of worksites considered for our initial group of assessments. The initial 9 sites
ranged in size from 115 to 3,200 employees and included manufacturing, construction, health care, higher education, and government organizations.
The criteria presented in Table 3 were applied to determine whether a site appeared to be implementing a promising practice related to obesity or healthy body weight. As described below, these same criteria were the basis of the
post-visit assessment to evaluate whether a practice or program was judged to be promising.
An alternative to the SWAT evaluation approach would be for a site visit to be conducted by a “connoisseur” — that is, an expert with deep subject matter knowledge, extensive site-visit experience, perceptual sensitivities, and refined insights. Limitations to this approach are 1) its inherent subjectivity and 2) the dearth of connoisseurs who can evaluate emerging, innovative strategies (17). Although a site visit is at the heart of the process, SWAT differs from a connoisseur
evaluation because it uses 3 graduated levels of worksite health promotion and evaluation expertise
(Table 1). In a SWAT evaluation, connoisseurs (Level 3) make a summative evaluation judgment about whether a practice is promising, an assessment for which such high-level expertise is crucial.
To promote consistency in site-visit observations and to reduce subjectivity, SWAT protocols include interview guides and written templates. For example, site visitors used a data review template to guide their written inventory of the types of data
that sites collected and the kinds of routine analyses performed (Table 1, Phase 2, Step 8). Another example is a structured checklist used by the site visitor to conduct an observational environmental assessment of the worksite facilities and the
surrounding community, including such factors as distances to local restaurants and availability of places to engage in physical activity.
In exchange for hosting a CDC site visit, worksite staff members were
provided opportunities to receive CDC technical assistance throughout the
SWAT process. The first opportunity arose naturally during the site visit,
for example, as employers interacted with site visitors to complete an
inventory of their data collection and routine program evaluation
activities. At one site, the site visitor noted that the employer’s existing data could be used to assess employee participation by job category or health risk
factor. Analysis of these data could be used to plan recruitment strategies to engage hard-to-reach employees, especially those who might stand to derive the greatest health improvements.
A comprehensive site-visit report (approximately 15 pages), which was developed by the site-visit team, followed a structured template that explicitly addressed each of the SWAT criteria described in Table 3. A draft site-visit report was shared with each site to verify the report’s accuracy and to give sites the opportunity to add relevant information. Each site also received written technical assistance in the form of a 3- to 4-page “interpretive assessment” developed by members of
the project team who are CDC staff and Level 3 connoisseurs. The interpretive assessment highlighted the site’s practices that were particularly notable and
suggested improvements in the site’s health promotion program and evaluation methods. In formulating these site-specific
suggestions, the CDC experts were guided by a detailed outline of recommended practices for worksite health promotion programs
(Table 5). To avoid overwhelming a site with a long list of
recommendations, the authors of the interpretive assessments considered the site’s available resources and readiness to change, and limited suggestions to those that seemed readily achievable. Each site also was offered a 1-hour telephone consultation with CDC project team experts to discuss the report, the interpretive assessment, and any other topics related to worksite health promotion (Table 1, Phase 4, Step 11). The WMUEC team also conducted brief post-visit telephone interviews
with worksite staff to ascertain what benefits the sites perceived from participation in the SWAT project and to take suggestions to further improve the SWAT process.
A tenet of the SWAT approach is that the business and public health communities can learn from each other. During the 12 months
after the initial 9 participating sites were evaluated, CDC hosted a series of 1-hour telephone conferences approximately every 2 months as a way to foster peer-to-peer networking and to further build these sites’ capacities, especially with regard to using evaluation methods to strengthen programs. The technical assistance topics presented included how to
increase program participation, strengthen the collection and use of data, and develop a business case for worksite health promotion.
The SWAT method is built on CDC’s Evaluation Framework (13), which has as its core the
program evaluation standards of utility, propriety, accuracy, and feasibility (14), developed by the Joint Committee on Standards for Educational Evaluation. With the help of the meta-evaluation feedback, the SWAT project team considered the standards throughout the development process and documented decisions in meeting minutes and file notes. We report the outcomes of the SWAT development process and its success in meeting our goal for SWAT to be business-friendly, effective for our public health purpose in identifying promising practices, and appropriate in its balance of the
program evaluation standards of utility, propriety, accuracy, and
feasibility.
Back to top
Results
To ensure that the SWAT method was business-friendly as well as practical (the utility standard), it was designed to be shorter and less costly than a rigorous evaluation study while still generating sufficient data
to assess promising practices. A cornerstone of the SWAT approach is to establish a collegial relationship with each organization and to minimize the burden on employers so that site visits will be welcome. This approach was not only beneficial in the site-visit stage but also
has led to continuing collegial relationships. For example, 8 sites volunteered to participate in CDC worksite health promotion activities, including interviews for a CDC educational video.
As a scientific agency, CDC wanted accurate data (the accuracy standard). Accuracy was balanced against the feasibility of accessing data under nonresearch conditions (the feasibility standard). Few, if any, business organizations engage in health-related data collection that meets the rigorous standards of a scientific study. Instead, organizations routinely collect and analyze data for business processes to understand and track employee benefits, health insurance costs, absenteeism, and safety.
Recognizing that tradeoffs would be necessary, we used 2 approaches to balance the feasibility of data collection with the adequacy and accuracy of data.
First, site visitors used a template to guide their review of each site’s in-house data, including data quality and data analysis activities. Second, site visitors elicited examples of strategies that worksite staff thought were particularly successful based on their in-house data. For example, one site shared summary data that tracked participation before and after an incentive policy change from a cash bonus to paid time off, showing a dramatic increase in employee enrollment in the
health promotion program.
Ensuring data accuracy is particularly challenging in a practice-based approach because key definitions may vary widely among worksite staff, health promotion experts, and public health evaluators. In contrast to a research study in which terms are carefully defined, the initial 9 sites we visited
described program success in different ways, such as program participant satisfaction, improvements in employees’ health risk factors,
and
savings realized through reduced health insurance expenditures. To address the evaluation standard of accuracy and to strengthen the rigor of our approach, we developed written definitions of key terms, especially those related to program outcomes.
Federal privacy rules on individual-level health data dictated that we limit our interviews to key informants acting in their official capacity, such as a company president or worksite health promotion coordinator (the propriety standard). Individual employees/program participants were not interviewed. However, to better understand the employee perspective, site visitors conducted a group interview with an employee wellness committee, if one existed, since these employees were acting in their
official role as committee members. Site visitors asked employers to share only summary data and were vigilant in ensuring that they did not view individual data even when sites offered to share it.
Another opportunity to hear from site staff
after the site visit was the 1-hour telephone conference calls (Table 1, Phase 4, Step 11).
Several sites reported that they implemented new strategies or changed their practices
on the basis of the suggestions in CDC’s interpretive assessment. For example, 3 sites had taken steps to provide improved access to healthier foods in their cafeterias and vending machines, and 4 sites instituted new data analyses to enhance their tracking of employee participation and to customize recruitment efforts to attract nonparticipating employees.
A SWAT assessment is not designed to draw a conclusion about the effectiveness of particular strategies. Such an assessment would not be possible because SWAT site visitors do not collect or analyze individual-level data, nor do they verify accuracy of analyses shared by employers. To identify practices that hold promise for further, more rigorous evaluation, the SWAT approach relies on the judgment of Level 3 connoisseurs. In this case, 3 CDC staff who were not
part of the project team independently reviewed the site-visit reports and identified practices that were promising.
The SWAT project intentionally sought innovative programs as a counterpoint to more established programs reported in the peer-reviewed literature. However, a reality of practice-based evaluation is that organizations frequently change strategies to keep them fresh and interesting. To address this variability, site visitors focused on obtaining information on 1) key program practices, especially those that were continued over time, and 2) program practices that were related to weight control
or weight loss, which was consistent with the project’s goals.
The separate efficacy evaluation of SWAT conducted by WMUEC (Table 2) concluded that SWAT’s strengths were its systematic and detailed procedures, its service orientation (ie, its feedback to worksites and its pursuit of public health goals), and the accuracy of the information it provides. Potential weaknesses in the SWAT method were identified by WMUEC throughout the process, allowing the project team to revise and improve the approach. Remaining weaknesses identified (Table 2)
further highlight the challenges in conducting practice-based evaluation. The 3 benefits of participation most commonly cited to the WMUEC team at the post-visit evaluation were the opportunities to 1) reflect on their program, 2) learn from other organizations, and 3) be associated with CDC.
Suggestions for improving SWAT emphasized the need to convey the purpose of the
project at multiple points with multiple site staff.
SWAT is intended to be more rapid and less costly than full-scale, rigorous evaluation.
On the basis of project records and application of the SWAT method subsequent to our initial 9 sites, we estimate the number of hours for one SWAT assessment as follows: 87 hours for Level 1 evaluators; 34 hours for Level 2 experts; and 4 hours for Level 3 experts. The time from initial contact with a potential site through the technical assistance conference call requires a minimum of 2 months
and an average of 4 months.
Back to top
Discussion
Well-designed, well-executed research studies may provide the best possible scientific information; however, absent such studies, the best available information can still be useful in making progress toward public health goals (18). Furthermore, evaluation of evidence-based practices can be supplemented by accumulation of practice-based evidence from the evaluation of programs and strategies being implemented in the field. CDC identified a need for an evaluation method that can be
deployed more rapidly than a research study but that provides a more rigorous assessment than programs’ self-reports.
The limitations of the SWAT method in large part correspond to the SWAT criteria for assessing promising practices. We sought innovative programs; however, innovators were likely to innovate continually, making “program”
difficult to define. Data quality was another SWAT criterion used to assess programs; however, CDC and RTI site visitors neither collected nor analyzed primary data related to the worksite health promotion program participants or their health status
and outcomes. Instead, we relied on summary data provided by worksites to generally assess the site’s data quality and use of data for its program evaluation.
Using practice-based evidence for evaluation presented challenges not only to CDC but also to worksites. For example,
employers faced obstacles in obtaining employee health data for longitudinal analysis of health outcomes without compromising the confidentiality of employees’
private information. Some sites hired vendors to handle employee data, and others limited their data collection to repeated cross-sectional measures of indicators such as the blood pressure status of their
entire workforce or body weight measurements of weight loss team members.
A contribution of a practice-based approach to evaluation is the potential to shape a research agenda by identifying the practices that appear to be most promising and thus worthy of future investments in rigorous research that can measure their
effect precisely.
As researchers design a new intervention to study, they can look to practice-based evaluations to better understand how employers implement their programs and strategies in a wide range of worksite settings
and for diverse workforces. Such evaluations can yield feasible strategies that
can be broadly disseminated. Thus, evidence-based and practice-based approaches can work together to broaden the knowledge base as we address public health problems.
Although the SWAT approach was designed for small and medium-sized worksites to assess health promotion programs that may
affect adult obesity, CDC has already adapted this method for large worksites and expanded it to include other chronic diseases and preventive health areas. An extension of the SWAT approach recently used by CDC was for a large employer to fund a SWAT assessment of 3 of its business units’ worksite health promotion programs, policies, benefits, and environmental
supports. As part of this partnership, CDC conducted an assessment of the worksite program, with the understanding that the employer will implement several evidence-based strategies that CDC recommends based on the high-priority health issues identified through the assessment. Thus, CDC has new resources and an avenue to expand its practice-based evidence on worksite health promotion conducted in a natural setting.
The SWAT method has also been used to inform CDC’s knowledge base related to community environmental and policy strategies for childhood obesity prevention. A new project, funded by a large foundation and in partnership with CDC, is sponsoring 60 SWAT-type assessments through 2008. Finally, CDC is using the SWAT method as the basis for developing a comprehensive self-assessment tool for employers. Similar to the School Health Index (http://apps.nccd.cdc.gov/SHI/Default.aspx), a new
Workplace Health Index will guide employers in reviewing their programs, practices, and policies that promote employee health.
The SWAT development process generated a middle-ground evaluation method that is business-friendly and effective in guiding rapid assessment of potentially promising practices in worksite health promotion. An independent evaluation of the SWAT method concluded that the method is acceptably compliant with the
program evaluation standards of utility, propriety, accuracy, and feasibility and is
effective in providing data sufficient for experts in health promotion to identify promising and
innovative worksite health promotion strategies. A strength of SWAT is that it makes explicit how promising practices are determined, thus reducing the subjectivity of such determinations.
The public health response to the need for worksite health promotion strategies to address obesity includes both practice-based evidence and evidence-based practice as complementary sources of information. As a practice-based evidence approach, SWAT assessments are especially useful in providing preliminary information to guide future investments in research studies that can more rigorously examine those practices that appear most promising. Furthermore, insight into worksite health
promotions can help researchers design interventions with the potential for broad dissemination, creating a loop from practice to research and back to practice.
Back to top
Acknowledgments
The authors gratefully acknowledge the contributions of the following expert consultants: Desiree Backman, DrPH, California 5-A-Day for Better Health Campaign; Paul Estabrooks, PhD, Kaiser Permanente Colorado; Eric Finkelstein, PhD, Research Triangle Institute International; Ron Goetzel, PhD, The Medstat Group; James O. Hill, PhD, University of Colorado Health Sciences Center; Robert W. Jeffery, PhD, and Frances Lawrenz, PhD, University of Minnesota; Laura Levitan, PhD, and Tracy Orleans,
PhD, Robert Wood Johnson Foundation; Laura Linnan, PhD, University of North Carolina; Tim McDonald, General Motors; Barbara Moore, PhD, Shape Up America; Risa Wilkerson, MA, Michigan Governor’s Council on Fitness, Health
& Sports; Mark Wilson, PhD, University of Georgia; Deborah Bauer, RN, CDC Healthier Worksite Initiative; and John Librett, PhD, and Sarah Martin, PhD, CDC Division of Nutrition, Physical Activity,
and Obesity.
Drs Finkelstein, Lawrenz, Linnan, and Wilson also participated in a meeting at CDC in Atlanta, Georgia, in the summer of 2005.
The authors also thank Alison Heintz for her editorial assistance.
Back to top
Author Information
Corresponding Author: Diane O. Dunet, PhD, Division of Nutrition, Physical Activity, and Obesity, National Center for Chronic Disease Prevention and Health Promotion, Centers for Disease Control and Prevention, 4770 Buford
Hwy NE, MS K-26, Atlanta, GA 30341. Telephone: 770-488-5566. E-mail: Ddunet@cdc.gov.
Author Affiliations: Phillip B. Sparling, Georgia Institute of Technology, Atlanta, Georgia; James Hersey, Pamela Williams-Piehota, RTI International, Research Triangle Park, North Carolina;
Mary D. Hill, Michele Reyes, Centers for Disease Control and Prevention,
Atlanta, Georgia; Carl Hanssen, Hanssen Consulting, Ada, Michigan; Frances Lawrenz,
University of Minnesota, Minneapolis, Minnesota.
Back to top
References
- Edwards LN, Field-Hendrey E. Home-based workers: data from the 1990 Census of Population. Mon Labor Rev 1996;118(11):26-34.
- Healthy people in healthy places: healthy workplaces. In: Health protection goals and objectives.
Atlanta (GA): Centers for Disease Control and Prevention; 2007. http://www.cdc.gov/osi/goals/Objectives0307.pdf. Accessed August 13, 2007.
- Campbell KP, Lanza A, Dixon R, Chattopadhyay S, Molinari N, Finch RA, editors. A purchaser’s guide to clinical preventive services: moving science into coverage. Washington
(DC): National Business Group on Health; 2006.
- Finkelstein EA, Fiebelkorn IC, Wang G.
State-level estimates of annual medical expenditures attributable to obesity. Obes Res 2004;12(1):18-24.
- Goetzel RZ, Hawkins K, Ozminkowski RJ, Wang S.
The health and productivity cost burden of the "top 10" physical and mental
health conditions affecting six large U.S. employers in 1999. J Occup Environ Med 2003;45(1):5-14.
- Bungum T, Satterwhite M, Jackson AW, Morrow JR Jr.
The relationship of body mass index, medical costs, and job absenteeism. Am J Health Behav 2003;27(4):456-62.
- Tucker LA, Friedman GM.
Obesity and absenteeism: an epidemiologic study of 10,825 employed adults. Am J Health Promot 1998;12(3):202-7.
- Xiang H, Smith GA, Wilkins JR 3rd, Chen G, Hostetler SG, Stallones L.
Obesity and risk of nonfatal unintentional injuries. Am J Prev Med 2005;29(1):41-5.
- Must A, Spadano J, Coakley EH, Field AE, Colditz G, Dietz WH.
The disease burden associated with overweight and obesity. JAMA 1999;282(16):1523-9.
- Guide to community preventive services. Atlanta (GA): Centers for Disease
Control and Prevention. http://www.thecommunityguide.org/. Accessed July 20, 2007.
- Green LW.
Public health asks of systems science: to advance our evidence-based
practice, can you help us get more practice-based evidence? Am J Public
Health 2006;96(3):406-9.
- Employment size of employer and nonemployer firms. Washington (DC): U.S.
Census Bureau; 2003. http://www.census.gov/epcd/www/smallbus.html#EmpSize. Accessed April 20, 2007.
- Centers for Disease Control and Prevention.
Framework for program evaluation in public health. MMWR Recomm Rep 1999;48(RR-11):1-40.
- The Joint Committee on Standards for Educational Evaluation. The program evaluation standards. 2nd ed. Thousand Oaks (CA): Sage Publications; 1994.
- Hanssen CE, Lawrenz F, Dunet DO. Concurrent meta-evaluation: a critique. Am J Eval.
In press.
- Hersey J, Williams-Piehota P, Sparling PB, Alexander J, Hill MD, Isenberg KB, et al.
Promising practices in promotion of healthy weight at small and medium-sized
US worksites. Prev Chronic Dis 2008;5(4).
http://www.cdc.gov/pcd/issues/2008/oct/07_0172.htm.
- Stuffelbeam DL. Evaluation models. New Directions for Evaluation 2001;89:8-106.
- Koplan JP, Liverman CT, Kroak VI, editors. Preventing childhood obesity: health in the balance. Washington
(DC): National Academies Press; 2005.
Back to top
|
|
