 |
 |
|
||||||||
|
|
|
|
|
|
|
||
|
|
|
||||||
|
|
|
Volume 3: No. 3, July 2006
COMMUNITY CASE STUDY
|
||||||||
|
|
|
Healthy Weigh (El camino saludable) Program Overview Child care for preschool-aged children, free family food bags, and transportation were provided in each phase. Services were offered in English and Spanish. Phase 1 (Summer and Fall 2003) Age-, gender-, and ability-appropriate physical activity classes Family meals with table talks Nutrition lessons for adults, adolescents, and school-aged children Phase 2 Pilot (Spring 2004) Physical activity classes and a healthy snack with an age-appropriate nutrition lesson Phase 2 (Fall 2004) Family exercise and nutrition classes Light, healthy family meals with nutrition and food demonstrations Post-Phase 2 Exercise Program (Spring 2005) Age-appropriate physical activity classes and a healthy family snack Family Cooking Course (Spring 2005) Series of four cooking classes in which families worked together to prepare nutritious meals |
|
|
|
|
|
|
Weight management requires a complex series of interactions of individual choices, behaviors, and environments (e.g., familial, socioeconomic, political, cultural, natural, built [manmade]) (1). Healthy Weigh (El camino saludable) is a community-based obesity prevention program for low-income, predominantly Hispanic and African American families in an urban community in Tarrant County, Texas. Phase 2 has already been completed, and in spring 2005, Healthy Weigh implemented post-phase 2 activities (sidebar). Healthy Weigh Phase 1, a community–campus (Texas Christian University, Fort Worth, Tex) partnership, was completed in summer (June–August) and fall (September–November) 2003. The program met stated program objectives and extensively engaged students from several health disciplines in service learning (2). As phase 1 concluded, Healthy Weigh evolved from research being conducted in a community to community-based participatory research. This article describes lessons learned from critically examining the phase 1 evaluation process.
Program evaluation is an essential part of community health practice and has four goals: to gain insight, change practice, assess effects, and affect people participating in the evaluation process (3). The request for proposals, grant preparation, and implementation of phase 1 occurred rapidly. The grant’s purpose, which was to reduce overweight and obesity, required a program evaluation focused on assessing effects. However, in this pilot intervention, we also wanted to gain insight about the most effective ways to help low-income, racially and ethnically diverse families adopt healthy eating and physical activity patterns and manage their weight. Formative evaluation suggested that adding questions to the phase 1 evaluation process could have improved the program's efficiency and effectiveness and provided more insight into interactions among the program, participants, and community environments. We critically examined the phase 1 evaluation process using the formative evaluation data, Donabedian’s structure–process–outcome framework (4), and the Centers for Disease Control and Prevention’s (CDC’s) Framework for Program Evaluation in Public Health (referred to as framework in this article) standards (3). This examination provided information to improve delivery and evaluation of subsequent Healthy Weigh interventions.
The prevalence of overweight and obesity in Texas is among the highest in the United States, where overweight and obesity rates have increased steadily in the past two decades (5-7). Sixty-four percent of adult Texans are overweight or obese, and more than 39% of fourth-graders, 37% of eighth-graders, and 29% of eleventh-graders are overweight or at risk for overweight (5). Obesity is more prevalent among women, African Americans, Hispanic Americans, and people with low education levels and income. Many racial and ethnic groups continue to face disproportionate risks for obesity-related chronic diseases and associated decreased life expectancy (8,9).
Recent speculation attributes the increasing obesity prevalence among low-income populations to socioeconomic status, personal choices, established cultural and family patterns, and environmental factors (10-13). Low-income communities are particularly vulnerable because nutritious foods can be expensive or difficult to find, whereas less healthy, energy-dense foods are readily accessible and affordable (11). Compared with higher income neighborhoods, poor neighborhoods may lack well-maintained sidewalks and streets, safe outdoor spaces, or exercise facilities and therefore be less conducive to physical activity (14).
Parents are primary sources of information for children who are learning about healthy eating practices, so family environments are important intervention settings for establishing lifelong dietary habits (14-16). Studies investigating familial aggregation of obesity show that family eating environments link parental adiposity and dietary intake with children’s adiposity and dietary intake (17).
A request for proposals to address overweight and obesity from United Way of Metropolitan Tarrant County prompted university faculty and community organizations to collaborate on designing and implementing Healthy Weigh. As phase 1 was ending, development of a participant leadership group led to more balanced partnerships among participants, community organizations, and investigators, and Healthy Weigh evolved into a community-based participatory research effort. Texas Christian University Human Subjects Institutional Review Board approved the Healthy Weigh program. The previous sidebar summarizes the Healthy Weigh interventions; participants helped design and evaluate phase 2.
Cornerstone Community Center (CCC), a faith-based community organization, provided the physical facility for Healthy Weigh, assisted with weekly reminder calls to participants, and was the primary source of participant referrals and community volunteers. CCC serves a community at high risk for overweight and obesity. Factors for overweight and obesity included the median household income ($20,000), federal poverty level (44.6% below the poverty level), and people older than 25 years lacking high school diplomas (42%) (18). The population in CCC’s catchment area is 35% African American and 26% Hispanic (18).
Phase 1 was offered twice in 2003 (summer and fall) and consisted of 12 weekly sessions, prescreening, and postscreening. Participants chose to enroll in the English- or Spanish-language version of phase 1. Community focus groups provided input on program design, evaluation, and recruitment. Recruitment flyers were distributed door-to-door in target neighborhoods and through schools and CCC.
Of 282 people screened for phase 1, most participants were female (72%) and Hispanic (82%) or African American (12%). Many of the participants were families; 46% of participants were younger than 19 years, and 9% were older than 60 years. Many participants had limited proficiency in English or Spanish. Based on body mass index, 84% of adults were overweight or obese, and 50% of children and adolescents were overweight or at risk for overweight.
The research design included evaluation of program process objectives and participant outcome objectives based on Healthy People 2010 (1), Recommendations to Increase Physical Activity in Communities (19), and Dietary Guidelines for Americans (20). Following are the outcome objectives (21) and program process objectives for Healthy Weigh Phase 1:
Outcome objectives
Process objectives
Program evaluation was determined by interview, direct measurement, and self-report (Table 1).
The university research team (two registered nurses and one registered dietitian) directed program implementation, which was carried out by registered nurses, a registered dietitian, paid staff, community volunteers, and supervised students from nursing, nutritional sciences, kinesiology, social work, and medicine programs. More than 160 undergraduate and graduate students from two universities served as educators, exercise leaders, meal preparation and service coordinators, table-talk leaders, child care workers, health screeners, and research assistants. Sixty percent of the students were volunteers, and most of the remaining students participated to meet course requirements. Many program participants also served as volunteers by helping with set up, clean up, and interpretation.
We analyzed evaluation data and a logic model (Figure) based on actual implementation of phase 1 (3). We used the analysis to identify evaluation questions that were missing from the phase 1 evaluation process. We categorized these questions as structure, process, and outcome and determined how including these questions in the phase 1 evaluation might have strengthened program implementation and outcomes (Tables 2, 3, and 4). Finally, we applied the following 30 framework standards to judge overall quality of the phase 1 evaluation process (3):
Utility standards: Does the evaluation serve the information needs of the intended users?
Feasibility standards: Is the evaluation realistic, prudent, diplomatic, and frugal?
Propriety standards: Is the evaluation legal, ethical, and considerate of the welfare of those involved and affected?
Accuracy standards: Does the evaluation reveal and convey technically accurate information?
[A full-size PDF of this logic model (104k) is also available.]
Figure. Healthy Weigh (El camino saludable) Phase 1 logic model.
Seven framework utility standards address information needs of evaluation users, who were defined as the funding agency and investigators in phase 1. Ways in which utility standards were met included the following:
There were several ways in which the evaluation could have been improved to meet utility standards (Table 2). Because other community stakeholders (CCC, program participants, and staff) were not identified as evaluation users and did not help design the evaluation plan, their information needs were unknown and not addressed. Monitoring the balance of needs is essential to developing effective community-based participatory research partnerships (22). Establishing evaluation criteria with partners could have promoted ongoing monitoring of relationships, frequent acknowledgment of strengths and growth, and rapid response to actual and potential problems.
Three framework feasibility standards were developed to ensure effective and practical use of program resources. Most phase 1 evaluation procedures were practical, adapted for the community setting, and politically viable, and they effectively used time and resources. Use of familiar equipment (e.g., scales, stadiometer, measuring tape, sphygmomanometer) and noninvasive methods for data collection fostered participant acceptance of measurements and made it easier for students to help conduct screenings. Concerted efforts to prevent phase 1 activities from interfering with CCC’s daily operations and early negotiation for using equipment and space helped gain and sustain CCC’s cooperation in evaluation procedures. Proactive communication with CCC’s staff and university, funding agency, and community–partner organizations also contributed to political viability. Ongoing budget management based on financial requirements of the program, university, and funding partner helped ensure that resources were used effectively to achieve program processes and outcomes.
Structure–process–outcome analysis highlighted ways the evaluation process could have better met the feasibility standards related to use of resources and political viability (Tables 2–4). Investigators frequently filled staff roles because of a shortage of staff members, lack of availability of staff members, or staff members’ lack of skills. Resolving staff structure issues would have permitted more consistent staff assignments and potentially increased skills and job satisfaction. Greater attention to staff structure would have contributed to more efficient and effective use of resources for participant screening and overall program administration.
Two process variables also affected feasibility. Knowing more about why participants attended some sessions and not others could have promoted more cost-effective use of human and material resources needed for program physical activity and nutrition classes, family meals, and food-bag preparation. Systematically collecting data about participants who dropped out of the program might have helped improve retention and program design. Early clarification and monitoring of community partners’ roles would have provided a framework for increased collaboration and communication among partners, making the evaluation process more politically viable.
Eight framework propriety standards address legal and ethical characteristics and how the evaluation process affects all the participants involved. Although the lack of formal agreements among all key parties involved in the evaluation was a weakness, overall the evaluation process met propriety standards. Concern for participant welfare was evident in design of evaluation tools and procedures, the willingness to adapt the research design to meet participant needs, the commitment to building relationships with participants and CCC staff, and adherence to financial accounting standards.
Informed consent and other screening tools were appropriate for participants with limited English proficiency. Documents were available in English and Spanish and were prepared at the fourth- to fifth-grade reading level in English, with reader-friendly formatting and adequate white space (23). Because one consent form per family was used, parents of large families and people with limited literacy did not have to handle multiple documents. Family members could choose to read the informed consent or have it read to them. Child care for preschool-aged children allowed families to concentrate on completing the informed consent.
Participant privacy was maintained at levels consistent with community expectations. Staff members explained screening results and gave families copies of results in appropriate languages. Referrals for follow-up health care were based on prescreening findings. Efforts were made to obtain health care for participants who lacked regular sources of care.
When formative evaluation showed that participants were unfamiliar with setting goals, we modified the screening protocol to include teaching participants about setting appropriate goals for healthy eating, physical activity, or weight management. Fifty summer participants asked to enroll in the fall program, so we revised the research design. Relationships with participants and their progress toward improved physical activity and healthy eating patterns were more important to overall program goals than strict adherence to the original research design.
All financial reporting requirements of the university and United Way were met. Maintaining and properly cleaning the CCC facility, equipment, and supplies during and after each Healthy Weigh meeting also addressed fiscal responsibility standards.
Twelve framework accuracy standards assess correctness of evaluation findings. The evaluation process for phase 1 grant objectives (see previous lists of outcome and process objectives) met these standards.
Structure–process–outcome analysis revealed how the phase 1 evaluation process could have measured the findings more completely and accurately (Tables 2–4). Collection of qualitative data was not as consistent and systematic as planned because of human resource problems. For example, although many table-talk conversations were recorded in log books, different staff members facilitated and documented the talks each week. Some weeks, too few staff members were available to document all talks, or available staff members lacked skills to document Spanish-language table talks. These factors contributed to uneven and inconsistent qualitative data collection. Most questions posed for improving the phase 1 evaluation process could have been answered by consistent, systematic collection and analysis of field observations, personal narratives, and table-talk conversations. Systematic identification, documentation, and analysis of best practices would have accomplished the following:
We undertook this critical examination of the Healthy Weigh Phase 1 evaluation to improve the evaluation design for future Healthy Weigh interventions. We learned that the phase 1 evaluation process successfully assessed program effects and generally met framework standards. Structure–process–outcome analysis highlighted additional evaluation factors related to structure and process that could strengthen subsequent Healthy Weigh interventions, community partnerships, and program outcomes.
We have already applied lessons learned to improve the phase 2 evaluation plan and process. An English-speaking health action group formed during phase 1 helped design the phase 2 pilot. Although this particular health action group disbanded, participants and students who had been consistently active in Healthy Weigh formed a new health action group. This group of monolingual English speakers, monolingual Spanish speakers, bilingual speakers (English and Spanish), adults, and youths acted as a true partner in the design, implementation, and evaluation of phase 2.
Another improvement was increased attention to monitoring the CCC–university partnership. Problems were avoided or addressed early in phase 2 by more frequent and intentional monitoring of the partnership. Contacting CCC after each Healthy Weigh meeting helped us be aware of and address actual or potential problems.
This critical examination was valuable and challenging. As is frequently true of community health programs, the opportunity and resources to address a significant community health issue emerged rapidly. Faced with limited development time, we focused on creating a phase 1 evaluation plan that would assess effects of an evidence-based program and would be designed to be culturally and linguistically appropriate for the target community. Recognizing Healthy Weigh as a pilot intervention, we wanted to gain insights about factors and relationships that would optimize community partnerships, program design, service learning, and improvements in participant families’ physical activity, healthy eating, and weight management behaviors. Formative evaluation data gathered during program implementation suggested that the investigators were frequently filling volunteer and paid staff roles and not obtaining all desired data.
Critical examination of phase 1 evaluation enabled us to identify structure and process evaluation questions to include in future evaluation plans. These questions will enhance our insight and strengthen programs and their evaluation processes. Healthy Weigh has evolved into a community-based participatory research intervention as community partnerships have been established and nurtured. This retrospective examination made clear that we began Healthy Weigh Phase 1 without the community relationships necessary to involve all stakeholders in the evaluation process as we now envision and practice it.
Healthy Weigh could not have been successful without the dedication of all its partners. CCC and its director, Jesse Vasquez, provided space, a connection to the community, grassroots expertise, and support for Healthy Weigh. Tarrant Area Food Bank provided food, particularly free fruit and vegetables, which minimized grocery costs. United Way of Metropolitan Tarrant County and Texas Christian University provided financial support. Student volunteers from Texas Christian University and University of North Texas Health Science Center donated numerous hours as critical staff members. Most importantly, we thank the program participants, who shared their stories and time, made life changes, exercised, helped themselves, and changed their community.
Corresponding Author: Pamela Jean Frable, Assistant Professor, Harris School of Nursing, Texas Christian University, Box 298620, Fort Worth, TX 76129. Telephone: 817-257-5840. E-mail: p.frable@tcu.edu.
Author Affiliations: Lyn Dart, Assistant Professor, Department of Nutritional Sciences, Texas Christian University, Fort Worth, Tex; Patricia J. Bradley, Associate Professor, Harris School of Nursing, Texas Christian University, Fort Worth, Tex.
|
|
|
|
|
|
|
|
The opinions expressed by authors contributing to this journal do not necessarily reflect the opinions of the U.S. Department of Health and Human Services, the Public Health Service, the Centers for Disease Control and Prevention, or the authors’ affiliated institutions. Use of trade names is for identification only and does not imply endorsement by any of the groups named above.
Privacy Policy | Accessibility This page last reviewed March 30, 2012
|
|
