Overview

Purpose of this Chapter

The purpose of this chapter is to provide guidance for developing a solid base of data to be used for the evaluation of HIV prevention community planning core objective five: “Fostering strong, logical linkages between the community planning process, plans, applications for funding, and allocation of CDC HIV prevention resources.” In this sense, core objective five bridges evaluation of the community planning process (core objectives one through four) with intervention plan evaluation by stressing the importance of allocating funds for interventions that correspond to recommended interventions in the comprehensive HIV prevention plan. This link between the planning process and actual HIV prevention programming is an extremely critical component of HIV prevention efforts, and the data collected are necessary to stakeholders’ ability to account for the extent to which federal funds are allocated to interventions that respond to documented needs.

This chapter features discussion of reasons for evaluating achievement of core objective five and sets forth a basic methodology for determining if interventions funded by the health department correspond to the comprehensive HIV prevention plan and whether interventions recommended in the plan are included in the annual CDC funding application submitted by CDC grantee health departments. The methodology, which is not intended to be a stand-alone approach, was pilot-tested by five CDC health department grantees, whose experiences and lessons learned helped streamline the data collection and analysis process. (For further background on the Management and Operational Indicators project, see the reference for Macro International, 1998.) The process is based on the assumption that the plan is a valid depiction of HIV prevention needs in the jurisdiction. Given that many of CDC’s grantees have already instituted mechanisms for ensuring that funding allocation is linked to recommendations set forth in comprehensive HIV prevention plans, recommendations in this chapter are not meant to replace existing mechanisms or to duplicate efforts. Instead, the purpose is to provide basic guidance for assessing linkages that may be helpful to grantees at all stages of evaluation. This chapter is also intended to provide grantees with information about what other steps to take in evaluating core objective five.

For those needing assistance in the data collection and analysis process, the appendix of this chapter contains 1) an example format for abstracting plans, 2) an example of an abstracted plan, 3) an example completed worksheet for determining whether interventions identified through intervention plan evaluation (discussed in Chapter 3) correspond to strategies in the plan and interventions proposed in the funding application, and 4) CDC’s required reporting forms.

Back to top

Reasons for Evaluating Linkages between the Comprehensive HIV Prevention Plan, Application, and Resource Allocation

While achievement of the first four objectives of HIV prevention community planning is essential to implementing the planning aspects of the initiative, core objective five is a critical factor in ensuring that the plan becomes a reality. The most direct way to assess linkages is literally to match recommended interventions in the plan to interventions that are included in the application and to interventions that are funded to determine if the application and funding allocation processes have been true to the intent of the plan. By conducting a formal evaluation, a health department can systematically collect data toward four ends:

• To compare the plan to the CDC funding application
• To identify the set of HIV prevention interventions in the jurisdiction (funded completely or in part with CDC funds and, when possible, other funds)
• To determine which interventions in the plan are being enacted and which are not (see Figure 5.1)
• To identify and learn about interventions being implemented (especially with CDC 99004 funds) that do not correspond to the plan (see Figure 5.2)

These data can be used to strengthen the comprehensive HIV prevention plan to ensure it is made up of recommendations that are explicit, reasonable, and feasible and to identify gaps in implementation that can be addressed during future funding allocation decisions. In addition, the information facilitates the health department’s efforts to be accountable to the many stakeholders who have a vested interest in the effects of the planning process. These stakeholders include community members, community planning group members, health department staff, local and state legislatures, CDC, and federal policymakers.

Steps for Evaluating Linkages between the Comprehensive HIV Prevention Plan and the CDC Funding Application

Determining the extent to which interventions recommended in the plan are included in the annual CDC funding application is a straightforward activity. In fact, most health departments ensure that all interventions in the application directly correspond to those in the plan, facilitating the following steps.

1. Abstract the comprehensive HIV prevention plan as described in the next section. This will produce a list of all prioritized target populations and prioritized interventions intended to reach them.
2. List all of the interventions included in the CDC funding application.
3. Compare the list of interventions in the plan with the list of interventions in the application. Using these lists, create three more lists: 1) the interventions that are in the application and the plan; 2) the interventions that are only in the plan; and 3) the interventions that are only in the application.

Figure 5.1 Linkages Between the Comprehensive Plan and a Jurisdiction's HIV Prevention Programs

Figure 5.2 Jurisdiction's Use of CDC 99004 Funds for HIV Prevention Interventions With Respect to the Comprehensive Plan

Back to top

Steps for Evaluating Linkages between the Comprehensive HIV Prevention Plan and Resource Allocation

Below are the suggested steps for evaluating linkages between the plan and funding allocation for interventions. Before beginning, however, it is important to determine the scope of information that will be collected. For instance, an ideal effort to assess implementation of the plan across an entire jurisdiction would involve collection of information about all HIV prevention interventions in the jurisdiction, not just those funded by the health department. In fact, the comprehensive HIV prevention plan is meant to guide all HIV prevention activities in a jurisdiction, including activities related to community planning that are not actual interventions (such as capacity building and infrastructure development efforts). (For further description of community planning, see CDC’s Supplemental Guidance on HIV Prevention Community Planning, 1993.)

There are many situations that might prevent a health department from conducting such a comprehensive effort. However, grantees are expected, at minimum, to gather data on all interventions conducted or funded by the health department with CDC funds. Depending on resources, data on interventions not funded by the health department could be collected simultaneously or at a later date. Funders of these interventions include departments in the health department other than the one that houses HIV prevention activities, other government agencies, and private foundations.

The following steps are designed to guide activities leading up to the compilation of data that can be used to assess core objective five. In jurisdictions where funding allocation is explicitly linked to the comprehensive plan or where there is a mechanism in place for evaluating core objective five, some or most of the steps may not be necessary.

1) Abstract the Comprehensive HIV Prevention Plan

Because plans vary greatly, each grantee will need to assess the easiest way to identify recommended interventions in its plan(s) to obtain a reference point for determining if funded interventions match the plan. Identifying recommended interventions most likely will entail abstraction of the plan. However, it is important to note that some plans may not need to be abstracted because recommended interventions are laid out in an organized way; other plans may require more abstraction efforts because specific interventions may be difficult to identify. Still, there are many ways in which a plan could be abstracted. One way is set forth in the example abstraction format and example completed abstraction provided in the appendix of this chapter.

Although there is not much one can do to improve a plan during the first round of abstraction, it is suggested that the lessons learned be used to guide health department and community planning group discussions about the ways in which the plan could be modified to facilitate identification of recommended interventions and other HIV prevention strategies.

The most straightforward way to create a list of HIV prevention interventions funded by the health department is to review contracts and other documents describing health department grantees’ interventions. Some grantees may have this information available in a management information system or some other electronic format. Some grantees may find it useful to combine this step with Step 3, which involves collecting information about the interventions identified.

If resources are available for identifying interventions not funded by the health department, the most direct approach is to identify HIV prevention providers and to interview them about their interventions and funding sources. Sources for identifying providers include needs assessments, gap analysis documents, resource inventories, health department mailing lists, AIDS prevention service directories, CBO associations, social service associations, funders of HIV prevention activities, CPG members, and HIV prevention service providers.

Chapter 3 features guidance for collecting data on new and existing interventions funded by the health department (partially or completely) with CDC funds. That process is also designed to collect data needed for evaluating core objective five because it calls for information about the types of interventions provided and the populations targeted. This information can be used to determine whether there is correspondence between actual interventions and recommended interventions in the plan.

For health departments planning to collect information about interventions not funded with CDC resources (or not available through processes discussed in Chapter 3), it may be useful to use the strategies listed below singularly or in combination, depending on health department resources and needs.

• Drawing data from health department data bases
• Abstracting health department contracts with providers
• Interviewing health department and provider staff members
• Providing data collection forms to providers
• Identifying funding sources and their grantees

Matching funded interventions to recommended interventions in the plan requires categorizing the data into three types:

• Interventions that match recommended interventions in the plan
• Interventions that do not match recommended interventions in the plan
• Recommended interventions in the plan that are not implemented through funded interventions

The example completed worksheet for determining linkages between funded interventions and the comprehensive HIV prevention plan (in the appendix) demonstrates how data can be organized.

Back to top

Applying the Data

The data yielded by this evaluation can be used to identify factors affecting the application process and intervention implementation in order to respond to stakeholders’ inquiries about the ways in which community planning has affected HIV prevention programming across the country.

Additionally, grantees might use the data to determine next steps, such as inquiring further into the reasons for which matches are particularly high or low; for instance, low correspondence between recommended interventions in the plan and funded interventions might imply a need for modifying the plan and/or funding allocation processes. The major benefit of the data should be to provide grantees and their stakeholders with concrete information about the nature of HIV prevention activities in the jurisdiction relative to recommended interventions in the plan. Each grantee should determine how the data can be used to inform planning, allocation, and reporting processes.

More specifically, grantees may choose to use the data for the following purposes:

• To support requests for CDC funding that would enable grantees to respond to recommendations in comprehensive HIV prevention plans
• To modify plans, as needed, to render them easier to abstract and implement
• To modify RFPs and other funding processes and health department contracts with providers to build stronger linkages between the comprehensive plan and the types of activities proposed and funded; one way to facilitate this is to ensure that language used to describe interventions is clearly defined and used consistently across all documents
• To improve contract monitoring and evaluation of the implementation of plans by standardizing the format and content of proposals, contracts, progress reports, and other documents submitted by providers; in this way, health departments can ensure receipt of information about the characteristics of HIV prevention efforts
• To provide feedback to stakeholders in an effort to document, in a systematic manner, implementation of plans and the effects of the planning process; this can benefit community planning group members by showing them the effects of their work, which can sustain their interest in the process
• To determine which recommended interventions in plans could be supported by non-health department funds and identify means for encouraging other funders to fund interventions that correspond with recommended interventions in plans

Back to top

References and Resources

Centers for Disease Control and Prevention. Supplemental Guidance on HIV Prevention Community Planning for Noncompeting Continuation of Cooperative Agreements for HIV Prevention Projects. Atlanta: Centers for Disease Control and Prevention, 1993.

Centers for Disease Control and Prevention. Announcement 300: Cooperative Agreements for Human Immunodeficiency Virus (HIV) Prevention Projects Program Announcement and Availability of Funds for Fiscal Year 1993. Atlanta: Centers for Disease Control and Prevention, 1993.

Centers for Disease Control and Prevention. Announcement 99004: Cooperative Agreements for Human Immunodeficiency Virus (HIV) Prevention Projects Program Announcement and Availability of Funds for Fiscal Year 1998. Atlanta: Centers for Disease Control and Prevention, 1998.

Macro International Inc. Guide to Assessing Linkages Between Comprehensive HIV Prevention Plans and HIV Prevention. Atlanta: Macro International, Inc., 1998.

Back to top

Go to Monitoring Outcomes of Health Education and Risk Reduction Individual- And Group-Level HIV Prevention Interventions