A Cohort Review Approach Evaluating Community Health Worker Programs in New York City, 2015–2017

The objective of this study was to describe how a cohort review approach was applied as an evaluation framework for a community health worker intervention among adult residents in 5 public housing developments in New York City in 2015–2017. The cohort review approach involved systematically monitoring participants engaged in the Harlem Health Advocacy Partners program during a given time period (“cohort”) to assess individual outcomes and program performance. We monitored participation status (completed, still active, disengaged, on leave, or died) and health outcomes. In this example of a cohort review, levels of enrollment and program disengagement were higher in cohort 1 than in cohort 2. For 6-month health outcomes, the percentage of participants with hypertension who had controlled blood pressure was static in cohort 1 and improved significantly in cohort 2. The percentage of participants with diabetes who self-reported controlled hemoglobin A1c increased significantly in cohort 1 at 6-month follow-up. The cohort approach highlighted important outcome successes and identified workload challenges affecting recruitment and retention.


Introduction
Although evidence for the effectiveness of community health workers (CHWs) is mounting, reviews of interventions related to chronic disease prevention and management demonstrate inconsistent results (1-3). One key issue is that many CHW programs are funded through grants or operating budgets that are often unpredictable, unstable, and time limited (4). Such funding mechanisms pose unique challenges: with short-term funding, some health outcomes may not emerge within funded evaluation time frames, and positive benefits of programs, including the adoption and maintenance of behavior change, may not have the opportunity to accrue or be sustained. Another problem is inconsistency in how results are reported.
These challenges have affected other public health interventions focused on sustained patient interactions, and monitoring tools developed in response to these challenges can be adapted for CHW program evaluation. For example, the introduction of an annual review process known as a "cohort review" was an important innovation in the monitoring and evaluation of tuberculosis control efforts (5); it involved systematic monitoring of groups of patients beginning treatment within a given period ("cohort"). Structured indicators allowed local and national comparisons, as well as measurement against previous cohorts, to assess improvements in program recruitment, retention, and outcomes.
We adapted cohort review methods to the evaluation of a CHW program. By standardizing participant status definitions and tracking outcome milestones, CHWs and evaluators can develop an analytic framework to better monitor participation status, participant characteristics, and health outcomes. The cohort process also allows for the assessment of trends of program performance indicators that are actionable for decision makers, particularly when comparison groups are unavailable or are no longer supported by funding sources.

Purpose and Objective
The objective of this study was to describe how the cohort review approach was applied as an evaluation framework for a community-focused CHW intervention, the Harlem Health Advocacy Partners (HHAP) program, in New York City. HHAP is an ongoing municipal project that aims to improve the health of adults residing in 5 public housing developments in East/Central Harlem. Despite rich histories of community organizing, East/Central Harlem has been subject to policies and processes such as redlining, broken windows policing, and "benign neglect" that have contributed to high levels of poverty and poor health outcomes. HHAP was launched to address health and social conditions in the neighborhood, with the aim of closing racial/ethnic gaps in health and social outcomes between public housing residents in East/Central Harlem and other New Yorkers (6,7). We developed and applied the cohort review approach to the health coaching component of HHAP to assess program implementation and outcomes during the first 2 years of the program.

Intervention Approach
During the first year of HHAP, 224 participants were enrolled from February through August 2015 (cohort 1), and subsequent cohorts followed an annual enrollment cycle. Cohort 2 enrolled 348 participants from September 2015 through August 2016. Concurrent to cohort 1 enrollment, we recruited a 1-year comparison sample of 176 residents from 5 nearby developments, selected on the basis of frequency-matched sociodemographic characteristics and proximity to the intervention developments (8). After cohort 1, comparison groups were not available.
In addition to a residence requirement, eligibility criteria for health coaching and the comparison group included being aged ≥18 and having at least one of 3 self-reported chronic conditions (asthma, diabetes, or hypertension). Participants who reported ever having received a physician diagnosis of asthma, hypertension, or dia-betes were defined as adults with these conditions, on the basis of the following question: "Have you ever been told by a doctor, nurse, or other health professional that you have . . . ?" Both cohort 1 and cohort 2 participants were recruited primarily via community outreach conducted by CHWs, who canvassed the grounds of the selected public housing developments and collaborated with community and senior centers in each development to promote the program. The comparison group was recruited from a random sample telephone survey (9). CHWs attempted to deliver core intervention components within 6 to 12 months of enrollment.
The HHAP intervention includes 4 components: 1) health coaching, 2) navigation of the health care system, 3) wellness activities, including peer support and walking groups, and 4) advocacy to build leadership among residents to address community health needs and improve systems and conditions that influence neighborhood health. The health coaching provided by CHWs also included referrals, emergency interventions during acute-risk situations (eg, morbidly high blood pressure readings, mental health crises), and the setting of one or more SMART (specific, measurable, achievable, results-focused, and time-bound) goals. Additional health care navigation support was available through referrals to a partner organization that assists residents in obtaining medical services and ensures they receive the care to which they are entitled. A full description of the HHAP model is available elsewhere (8).

Evaluation Methods
For cohort 1, CHWs conducted intake assessments as part of participant enrollment (baseline), and an academic research team from the NYU-CUNY Prevention Research Center (PRC) conducted follow-up assessments. The academic research team conducted all comparison group assessments. Cohort 1 and comparison participants received a $20 cash incentive for completing surveys. For cohort 2, CHWs conducted baseline and follow-up assessments. Surveys were conducted at 3 months, 6 months, 9 months, or 12 months after enrollment. Among participants enrolled in cohort 1 and cohort 2, 209 of 224 (93.3%) in cohort 1 and 233 of 348 (67.0%) in cohort 2 completed any follow-up assessment survey. For this analysis, we tabulated data on 6-month follow-up from both years; the response rate was 85.7% (192 of 224) for cohort 1, 92.6% (163 of 176) for the comparison group, and 41.7% (145 of 348) for cohort 2.
We categorized all HHAP participants into mutually exclusive and exhaustive categories of participation in health coaching: completed, enrolled active, disengaged, on leave, or died (Box). CHWs assigned and updated participant status. The NYU-CUNY PRC collected data on health outcomes in the baseline surveys and PREVENTING CHRONIC DISEASE follow-up surveys. These outcomes were blood pressure control, blood pressure control among participants with hypertension, and self-reported hemoglobin A 1c (HbA 1c ) control among participants with diabetes. Blood pressure was the average of 3 measurements taken at each survey point, and we defined control as systolic blood pressure under 140 mm Hg or diastolic blood pressure under 90 mm Hg (10). We dichotomized self-reported status of glycemic control as controlled if a health professional told a participant their diabetes was within goal and as "uncontrolled or don't know" if they were told it was not within goal or if they were unaware of their status.

Results
A greater percentage of residents participating in HHAP health coaching than in the comparison group were aged 65 or older and self-reported hypertension (Table 1). Most participants were female and either Hispanic or non-Hispanic black, reflecting the population of the public housing developments (9). Participants were demographically similar to one another across cohorts, except that a greater proportion of cohort 2 participants than cohort 1 or comparison group participants were Hispanic.
The percentage of participants with self-reported hypertension in cohort 1 and controlled blood pressure did not change from baseline to 6-month follow-up (58.8% to 60.1%, P = .79) ( Table  2). Blood pressure control among residents with hypertension in the comparison group may have worsened from baseline to 6month follow-up (61.0% to 53.3%, P = .16). In cohort 2, the percentage of participants with diagnosed hypertension and controlled blood pressure increased significantly, from 57.7% to 73.9% (P = .002). The percentage of participants with self-reported diabetes who reported their HbA 1c as controlled increased significantly in cohort 1 (50.0% to 64.3%, P = .02), whereas self-reported HbA 1c control did not improve among comparison group participants (65.7% to 64.2%, P = .74). Although the change was not significant, we found improvements in HbA 1c control among cohort 2 participants (72.3% to 83.0%, P = .20).

Implications for Public Health
Our findings from the first 2 years of HHAP's health coaching component demonstrate the utility of the cohort review approach in providing a structure for evaluating a multiyear program, particularly when an ongoing comparison group is not available. The approach highlighted successes in health outcomes among participants retained in the program and challenges in program retention.
The assessment showed that more participants in cohort 2 than in cohort 1 disengaged from the program after 6 months. One reason for the higher level of disengagement in cohort 2 could be the challenge of maintaining health-coaching participants carried over from cohort 1 while recruiting for cohort 2, since CHWs in cohort 2 were also responsible for managing participants from the previous year. In addition, in the beginning of cohort 2, programmatic operations were transferred from an external organization to the New York City Department of Health and Mental Hygiene, which may have resulted in a disruption for some participants. Finally, the incentive offered in cohort 1 may have positively influenced program retention and the number of follow-up interviews. Because the cohort process cycle emphasizes continuous monitoring and improvement, the HHAP program addressed retention and workload issues in cohort 3. Our cohort assessment quantified improvements in key health outcomes shown in previous studies, namely in blood pressure (11) and glycemic control (2,12). The increase from cohort 1 to cohort 2 in the number of participants with controlled blood pressure suggests that the ability of CHWs to enhance care increases over time. This care includes efforts to keep participants connected with their primary care physician and to motivate participants to take all routine tests and medications for their conditions.

PREVENTING CHRONIC DISEASE
In planning for evaluating CHW programs using a cohort approach, metrics for the implementation process should be developed a priori and aligned with program objectives. Our analysis underscored the challenge of defining the participation status of a participant as complete. The definition was challenging because the criteria for program completion changed over 2 cohort years; awareness of this challenge helped formalize the definition of completion. Moreover, the program further disaggregated the disengaged group into 4 new categories: withdrew, lost to follow-up, transferred out of health coaching, and unavailable (ie, unable to fit health coaching into their schedule). It will be important to monitor these categories to assess whether participants are not interested or able to participate in the program, which would reflect a poor fit between the program and a participant's needs (withdrew), or the program is unable to maintain contact with participants because of other factors (lost to follow-up).
We found the cohort review approach adaptable to new program goals. For example, to better HHAP's efforts to address the social determinants of health in addition to disease management, we developed outcome metrics for social determinants of health for cohort 3, and the program will continue to monitor other variables that may contribute to health outcomes.
Our study has several limitations. Our findings in part reflect differences in HHAP programmatic operations between cohort 1 and 2. Cohort 1 data were collected by both CHWs and an academic research team and participants in cohort 1 received a cash incentive for completing surveys, whereas cohort 2 data were collected by CHWs only, often with fewer follow-up assessments, and cohort 2 participants did not receive an incentive. Differences in data collection may have biased comparisons between cohort 1 and cohort 2. Some health outcome data were self-reported; however, any bias introduced by self-report is unlikely to be differential across cohorts, except if selection bias was introduced because of higher loss to follow-up in cohort 2. Finally, given the large number of disengaged participants in cohort 2, we are not fully confident that improved outcomes were solely a function of programmatic improvements. Improved outcomes may reflect differential disengagement of participants who would have been less likely to improve.
Although previous CHW evaluations focused on individual-level outcomes, we found the cohort monitoring approach to be an effective method for evaluating the implementation process of CHW programs while also addressing issues associated with resource constraints and limited program duration (13). Adapting a cohort approach can begin to fill this gap (4,14). 14.