Skip Navigation Links
Centers for Disease Control and Prevention
 CDC Home Search Health Topics A-Z

Preventing Chronic Disease: Public Health Research, Practice and Policy

View Current Issue
Issue Archive
Archivo de números en español








Emerging Infectious Diseases Journal
MMWR


 Home 

Volume 3: No. 1, January 2006

STEP-BY-STEP: MAKING YOUR COMMUNITIES HEALTHIER
A Six-Step Model for Evaluation of Community-based Physical Activity Programs


TABLE OF CONTENTS


Print this article Print this article
E-mail this article E-mail this article:



Send feedback to editors Send feedback to editors
Download this article as a PDF Download this article as a PDF (245K)

You will need Adobe Acrobat Reader to view PDF files.


Navigate This Article
Introduction
Selecting an Intervention
Steps in an Evaluation of a Physical Activity Intervention
Conclusion
Author Information
References
Tables


Sarah Levin Martin, PhD, Gregory W. Heath, DHSc, MPH

Suggested citation for this article: Martin SL, Heath GW. A six-step model for evaluation of community-based physical activity programs. Prev Chronic Dis [serial online] 2006 Jan [date cited]. Available from: URL: http://www.cdc.gov/pcd/issues/2006/
jan/05_0111.htm
.

Introduction

Physical activity is a leading health indicator (1) and has numerous benefits, including reduced risk of coronary heart disease, hypertension, colon cancer, and diabetes (2). Regular physical activity can help prevent the onset of diabetes (3), even among those at high risk (i.e., those with impaired glucose tolerance [4]), and is part of diabetes self-management among people with the disease (5).

The Diabetes Prevention Program, a study funded by the National Institute of Diabetes & Digestive & Kidney Diseases, found that participants with impaired glucose tolerance who were assigned to an intensive lifestyle intervention reduced their risk of getting type 2 diabetes by 58%. On average, this group maintained their physical activity at 30 minutes per day, usually with walking or other moderate-intensity exercise, and lost 5% to 7% of their body weight (4).

It is likely that health educators at local health departments addressing diabetes and other chronic diseases will be asked to evaluate a physical activity program, because evaluation has a central role not only in improving programs but also in satisfying accountability requirements. A carefully planned evaluation can engage community members and build community capacity, and the results can be used to influence policy makers, share what works and what doesn’t work with other communities, and help ensure funding and sustainability.

The Centers for Disease Control and Prevention (CDC) has published the Framework for Program Evaluation in Public Health (6), which recommends six steps for effective program evaluation: 1) engaging stakeholders, 2) describing the program, 3) focusing the evaluation design, 4) gathering credible evidence, 5) justifying conclusions, and 6) ensuring use and sharing lessons learned. In this article, we describe these six steps using a hypothetical example of physical activity programming aimed at diabetes prevention. For this example, we assume that the reader is a community-based health educator at a local health department.

Back to top

Selecting an Intervention

Before planning the evaluation, you should be familiar with strategies or interventions proven to increase physical activity at the population level. The Guide to Community Preventive Services (Community Guide) (7), available from www.thecommunityguide.org/pa/default.htm*, includes eight recommended strategies that fall within three domains: informational approaches to increasing physical activity, behavioral and social approaches to increasing physical activity, and environmental and policy changes to increasing physical activity. Becoming familiar with these strategies is important background work. Because you may not have all the resources needed to carry out a population-based physical activity intervention, it is essential to work with partners. In fact, your role may be to influence others to carry out the program. One possible way to select the intervention strategy is by using the RE-AIM framework (information available from www.re-aim.org/*[8]), which considers the reach, efficacy, adoption, implementation, and maintenance of public health interventions.

For this article, we selected the strategy of creating or enhancing access to places for physical activity, combined with informational outreach activities, from the Community Guide. (A description of this strategy is available from www.thecommunityguide.org/pa/pa-int-create-access.pdf*.) This strategy involves the efforts of worksites, coalitions, agencies, and communities to change the local environment to create or improve access to opportunities for physical activity.

Back to top

Steps in an Evaluation of a Physical Activity Intervention

The CDC’s six-step Framework for Program Evaluation in Public Health will be used to guide this step-by-step example (6). These six steps have been adapted for use in physical activity programs and published in the Physical Activity Evaluation Handbook (9), available from www.cdc.gov/nccdphp/dnpa/physical/handbook/index.htm. Because an evaluation is not worth doing if the information gleaned will not be used, utility is perhaps the most important standard for program evaluation. The other standards are feasibility (you cannot evaluate with resources you do not have), accuracy (you cannot evaluate with poor or invalid measures), and propriety (you cannot evaluate if you are not fair and ethical to everyone involved).

Step 1: engage stakeholders

Important stakeholders for you as a health educator at a local health department are partners; these partners will carry out the intervention strategy. For an intervention to create or enhance places for physical activity, potential partners might include a city park, a shopping mall, the YMCA, the tourism bureau, and the community college. Also of great importance in terms of meeting the utility standard are decision makers — individuals who can use evaluation results to allocate future funds or cut programs. Examples might include the city mayor, the president of the community college, and the county-level director of parks and recreation.

You should invite all of the partner and decision-maker stakeholders to a meeting to describe the recommended strategy. The group should then discuss its role in making this strategy into a reality in its community and what evaluation resources it can offer.

Another group of stakeholders is the participants, individuals at high risk of developing diabetes. One way to engage participants is to invite them to a focus group or town meeting. At such a gathering, their ideas about the program can be assessed and used to refine the program to meet their needs. For the “creating access” strategy, for example, the participants can reveal what physical activity offerings would interest them and what venues might best reach them with information about these offerings.

Step 2: describe the program

The partners should be invited back for a second meeting to work on a logic model to depict graphically the proposed relationship between activities and expected outcomes. In this step, the work pertains to planning both the intervention and its evaluation. On the basis of step 1, the health educator is able to share the list of activities discussed at the first meeting and indicate which ones seem more popular or less popular among participants in the focus groups or town meetings.

To begin creating a logic model, the partners can divide the activities into two columns, early activities and later activities. Then the group should discuss outcomes they can realistically expect from the proposed activities. One outcome that seems obvious is an increased level of physical activity of residents, but there are more immediate and targeted outcomes that may precede such a behavioral change, such as increased opportunities for physical activity, increased awareness of physical activity offerings, and limiting of the target population to those at risk for developing diabetes, diabetes complications, or both. For diabetes care there may be outcomes even later than increased levels of physical activity, such as decreased levels of hemoglobin A1c (HbA1c) and, eventually, decreased incidence of diabetes morbidity and mortality. It is the role of the health educator to insist that the group set short-term objectives that include measurable outcomes. Objectives should be SMART — that is, specific, measurable, achievable, relevant, and time-bound. (See Appendix 4 in the Physical Activity Evaluation Handbook [9].) After the activities and the outcomes have been placed in sequence, the logic model begins to take shape. Inputs (i.e., resources to carry out the activities) can be added to the far left, and an overarching goal can be added to follow from long-term outcomes, as shown in the Figure.

Logic model

Figure. Example of a logic model for an intervention to create or enhance access to physical activity (PA) combined with informational outreach activities. [A text description of this model is also available.]

Step 3: focus on evaluation

As a health educator, you want to be sure that the evaluation is useful not only to your health department but also to the partner organizations that help to implement the program. To ensure buy-in and later use of the evaluation, ask stakeholders to develop questions that they would like to have answered. For example, the YMCA staff may want to know whether their membership increases. Increased membership becomes an outcome in the logic model. Note that the logic model can be made from right to left, that is, by asking, “How will we increase membership?” and then proposing some activities that might lead toward that outcome. Or it can be made from left to right, by asking, “Why are we doing that?” and then stating the expected outcomes of such activities. Either way, a focused evaluation will be one that poses questions based on the program and one that results in answers that serve the purpose of the evaluation. The purpose often will be to improve the program; other purposes may include gaining insight and assessing program effects. Defining your purpose is an important component of this step.

In our example, the stakeholders have already agreed on a logic model in step 2 (Figure), and so they can use it to focus their questions. They might decide to ask both process and outcome evaluation questions. Process questions relate to the inputs and activities, and outcome questions relate to the expected outcomes. It is possible to generate a long list of possible questions from the logic model, but then the list needs to be prioritized. Evaluating all questions may not be essential or even feasible. The stakeholders should remember the purpose of the evaluation and decide what would be useful for decision makers in prioritizing the list of questions. Examples of process and outcome questions include the following:

Process

  • Are the proposed activities being carried out by the partners? If not, why not?
  • What seem to be the most popular activities, and why?
  • Are we reaching individuals at risk for diabetes?

Outcome

  • Did awareness of opportunities increase?
  • Did participation increase? Why or why not?
  • Did HbA1c levels decrease among the population with diabetes?

Step 4: gather credible evidence

To answer the questions posed in step 3, evidence needs to be collected. How much evidence (quantity) and what kind of evidence (quality) are central to feasibility and accuracy. There must be a balance between collecting enough data and assuring it is of high quality. Sometimes a mix of quantitative and qualitative data will help achieve that balance: quantitative data can provide the numbers you need to answer some questions (e.g., participation rates), and qualitative data can help you understand why you got those numbers (e.g., interview a few who participate and a few who do not to learn why). Data are available from people, documents, observations, and existing information.

The Table provides a guide to collecting data for process and outcome questions by indicators, data sources, and performance measures. Indicators are what answer the question, data sources are the methods by which you collect data about the indicators, and performance measures are the outcomes you would like to achieve. It is helpful to have more than one indicator and more than one data source to answer each evaluation question. Using multiple indicators and data sources is often called triangulation and is recommended to increase accuracy. There are many tools available for collecting physical activity data. (See Appendix 5 in the Physical Activity Evaluation Handbook [9].)

Step 5: justify conclusions

There are three parts to this step: 1) analyze the data, 2) interpret the results, and 3) make judgments about the program. Having the performance measures helps to justify your conclusions. Perhaps a community college student needs an internship. You can hire him or her (often without financial compensation) to help with the evaluation. With guidance from you and the supervising professor, the student can analyze the data. Analysis for some questions will be easier than for others. For example, the difference between participation rates preintervention and postintervention is simple math, whereas analyzing focus group and interview data takes more time because all of the text must be read and common themes identified to answer the appropriate evaluation question.

After the analyses, you should convene a meeting of stakeholders to go over the results. Talk about possible alternative explanations to the findings of the evaluation. Discuss the limitations. One common limitation is having no control community; if possible, use a selected community as a basis of comparison in a quasi-experimental design. (For more on experimental designs, see the Physical Activity Evaluation Handbook, p. 26 [9].) Compare the results with the performance measures, and make judgments based primarily on that comparison. If you realize you did not achieve a performance measure, decide if you are willing to say that the program failed. It could be that almost every indicator showed improvement. With the decision makers present, the group can decide which results matter most and use those to summarize their findings to share with the community at large.

Step 6: ensure use and share lessons learned

The findings can be printed in the local newspaper, which, in our case, has the beneficial effect of increasing awareness of physical activity opportunities even further. The best features of the program should be highlighted. You can send the findings electronically to the CDC, where they can be widely distributed through the Physical Activity Listserv; in addition, examples of state physical activity programs can be posted on the State-based Physical Activity Directory at http://apps.nccd.cdc.gov/DNPAProg/.

To ensure use of your evaluation findings, formulate action-oriented recommendations. To help share lessons learned, consider your audience: use appropriate communications strategies and consider the most effective format for information (e.g., report, fact sheet, oral presentation) and venue (e.g., Web site, television, news media, town hall meeting).

Back to top

Conclusion

Although the principles of evaluation transcend topical areas, we hope that this step-by-step guide provides insight and examples for evaluating physical activity programs. There are evidence-based strategies for promoting physical activity in a community setting, and there are diabetes programs across the nation that could implement these strategies by engaging partners and initiating systems change. In our examples for creating access to places for physical activity, systems were engaged, and the strategies depended on these system changes. Partner organizations benefit by learning principles of evaluation that they can use for continuous quality improvement. Program evaluation plays a key role in ensuring success and sustainability of these programs. 

Back to top

Author Information

Corresponding Author: Sarah Levin Martin, PhD, Division of Nutrition and Physical Activity, Physical Activity and Health Branch, National Center for Chronic Disease Prevention and Health Promotion, Centers for Disease Control and Prevention, 4770 Buford Hwy NE, Mail Stop K-46, Atlanta, GA 30341. Telephone: 770-488-5413. E-mail: sjl2@cdc.gov.

Author Affiliations: Gregory W. Heath, DHSc, MPH, Division of Nutrition and Physical Activity, Physical Activity and Health Branch, National Center for Chronic Disease Prevention and Health Promotion, Centers for Disease Control and Prevention, Atlanta, Ga. Dr Heath is now with the Department of Health and Human Performance, University of Tennessee at Chattanooga, Chattanooga, Tenn.

Back to top

References

  1. U.S. Department of Health and Human Services. Healthy People 2010. 2nd edition. With Understanding and Improving Health and Objectives for Improving Health. 2 vols. Washington (DC): U.S. Government Printing Office; 2000 Nov.
  2. Centers for Disease Control and Prevention. Physical activity and health: a report of the surgeon general. Atlanta (GA): U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Promotion; 1996.
  3. Schulze MB, Hu FB. Primary prevention of diabetes: what can be done and how much can be prevented? Annu Rev Public Health 2005;26:445-67.
  4. Knowler WC, Barrett-Connor E, Fowler SE, Hamman RF, Lachin JM, et al. Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. N Engl J Med 2002;346(6):393-403.
  5. Boule NG, Haddad E, Kenny GP, Wells GA, Sigal RJ. Effects of exercise on glycemic control and body mass in type 2 diabetes mellitus: a meta-analysis of controlled clinical trials. JAMA 2001;286:1218-27.
  6. Centers for Disease Control and Prevention. Framework for program evaluation in public health. MMWR Morb Mortal Wkly Rep 1999;48(No. RR-11);1-40.
  7. Increasing physical activity. A report on recommendations of the Task Force on Community Preventive Services. MMWR Recomm Rep 2001;50(RR-18);1-14.
  8. RE-AIM [homepage on the Internet]. Manhattan (KS): Kansas State University Research & Extension. Available from: URL: www.re-aim.org*.
  9. Centers for Disease Control and Prevention. The physical activity evaluation handbook. Atlanta (GA): U.S. Department of Health and Human Services, Centers for Disease Control and Prevention. Available from: URL: http://www.cdc.gov/nccdphp/dnpa/physical/ handbook/index.htm.

Back to top

*URLs for nonfederal organizations are provided solely as a service to our users. URLs do not constitute an endorsement of any organization by CDC or the federal government, and none should be inferred. CDC is not responsible for the content of Web pages found at these URLs.

 



Tables

Return to your place in the textTable. Guide to Data Collection for Sample Intervention to Create or Enhance Access to Physical Activity, Combined With Informational Outreach Activities

Evaluation Question

Indicators

Data Sources

Performance Measure

Process

Are the proposed activities being carried out?

If not, why not?

Presence of classes

Activity leaders’ impressions

Observation

Interview

100% of proposed activities happening

What seem to be the most popular activities?

Why?

Participation rates

Opinions of target population

Sign-in sheetsa

Focus group of potential participants

20 per swim session, 12 per volleyball game or practice, 10 visitors per week for historical walk map, etc.

Are we reaching those at risk for diabetes complications?

Names of participants

Sign-in sheets crossed with medical records

75% of participants for enhanced activities will be from the target population

Outcome

Did awareness of opportunities increase?

Percentage aware of opportunities

Survey of all persons at risk for and with diabetes seen in the clinics and the city hospital

80% awareness among target population by 6 months

Did participation increase?

Why or why not?

Number of participants

Preinformational outreach participation rates versus postoutreach rates

50% increase in participation rates per site (YMCA, tourism bureau, community college)

Did hemoglobin A1c (HbA1c) levels decrease among population with diabetes?

Results of finger stick

Blood test in HbA1c analyzera

10% reduction in mean HbA1c level among participants with diabetes within 1 year

aInformed consent obtained for participation through recruitment at local clinics and hospital.

Back to top

 



 



The opinions expressed by authors contributing to this journal do not necessarily reflect the opinions of the U.S. Department of Health and Human Services, the Public Health Service, the Centers for Disease Control and Prevention, or the authors’ affiliated institutions. Use of trade names is for identification only and does not imply endorsement by any of the groups named above.


 Home 

Privacy Policy | Accessibility

CDC Home | Search | Health Topics A-Z

This page last reviewed March 30, 2012

Centers for Disease Control and Prevention
National Center for Chronic Disease Prevention and Health Promotion
 HHS logoUnited States Department of
Health and Human Services