Conceptual Framework for HIV Prevention and Evaluation
Contents of This Document
References and Resources
Increasingly, federal, state, and local agencies involved in
HIV prevention are recognizing the importance of evaluation for two primary
purposes: 1) to determine the extent to which HIV prevention efforts have
contributed to a reduction in HIV transmission and 2) to be accountable to
stakeholders by informing them of progress made in HIV prevention nationwide. In
response to this, CDC has identified the types of evaluation data needed to be
accountable for use of federal funds and to conduct systematic analysis of HIV
prevention to improve HIV prevention policies and programs. Evaluation data that
are needed include the types and quality of HIV prevention interventions
provided by CDC health department grantees and their grantees, the
characteristics of clients targeted and reached by the interventions, and the
effects of interventions on client behavior and HIV transmission.
These data needs correspond to CDC’s conceptual framework for HIV prevention
and evaluation (Figure 1.1), which sets forth the six evaluation activities that
health departments receiving CDC funding for HIV prevention might implement to
achieve a comprehensive understanding of the status of HIV prevention in their
The purpose of this document is to provide CDC health department grantees
with resources for implementing all six types of evaluation. While only some of
these have been put forth as requirements CDC (as outlined in
Guidance and requested in CDC Announcement 99004), all of the activities are
beneficial to the successful assessment of progress in HIV prevention. In fact,
grantees are encouraged to supplement the activities described herein with
additional evaluation activities tailored to local needs.
Variation in Evaluation Capacity Across Jurisdictions
This document takes into account the diversity of health departments’
capacities and experience in evaluation and seeks to provide assistance and
references useful across the spectrum of grantee experience. For instance,
health departments that already have designed and engaged in many of the
evaluation activities may find the guidance useful in determining how to use
existing data mechanisms to report the minimum set of data asked for by CDC
rather than duplicate efforts by creating a new system. For health departments
initiating some or all of the evaluation activities for the first time, this
guidance provides basic evaluation information, recommendations for developing
evaluation systems, and references to other evaluation texts and resources. CDC
also is developing mechanisms for providing technical assistance, training, and
other support to grantees as they implement evaluation activities.
Figure 1.1 CDC’s Conceptual Framework for HIV Prevention and Evaluation
CDC’s Framework for HIV Prevention Planning, Implementation, and
|Implementation of the Community Planning Process
|Development of Comprehensive HIV Prevention Plan
|Submission of Application to CDC by Health
|Development of Proposals with Intervention Plans
|Allocation of Funds by Health Department
|Implementation of Interventions
|Changes in Risk Determinants
|Changes in HIV Transmission
Participation in Evaluation Activities Not Required by CDC
CDC health department grantees are strongly encouraged to engage in
evaluation activities that exceed the expectations of this Guidance.
Staff members from the Program Evaluation Research Branch are available to
discuss ideas and projects. In some cases, special studies might be arranged in
collaboration with the Division of HIV/AIDS Prevention staff. For further
information or to initiate discussions about such projects, please direct your
Program Evaluation Research Branch
Division of HIV/AIDS Prevention--Intervention, Research, and Support
1600 Clifton Rd.
Atlanta, GA 30333
Limitations of this Document
By design, this document describes evaluation activities that are presented
primarily for the purposes of collecting and reporting data that correspond to
CDC’s conceptual framework for HIV prevention evaluation. Following are several
limitations of the information provided in this document:
- This document is not intended to be a comprehensive text on how to
conduct evaluation; rather, it is designed to assist grantees in
responding to CDC recommendations and requirements regarding the evaluation
of HIV prevention interventions supported with CDC funds.
- The methods suggested herein are sufficient for all CDC accountability
purposes and for some program improvement, but they are not sufficient
for use by health departments in the thorough evaluation of all their HIV
prevention activities. Grantees are strongly encouraged to supplement
the data with data gleaned through other evaluation activities.
- This document does not explain how health department grantees
should use the data for program improvement; grantees should consult
evaluation texts and experts for assistance in this area.
- This document provides guidelines for evaluating most but not all
types of HIV prevention interventions.
Development of this Document
This document was developed by the Program Evaluation Research Branch in
CDC’s National Center for HIV, STD, and TB Prevention, Division of HIV/AIDS
Prevention—Intervention Research and Support, with the assistance of Macro
International, Inc. The branch conducted literature reviews and solicited
materials from more than 50 individuals representing health departments, CBOs,
evaluation consulting firms, and academic institutions. Through this process,
CDC determined that the underlying principles of this document would be the
- Evaluation activities discussed should reflect the critical steps in
HIV prevention planning, implementation, and assessment of effectiveness.
- Data collected through implementation of the activities outlined in
this document will supplement qualitative data collected through case
studies and other local evaluation activities.
- Recommended data to be collected and methods for collecting them
should meet the requirements of scientific rigor, usefulness, and
- Evaluation activities should consist of a balance between process and
- The evaluation activities and discussions about them should set forth
the minimum amount of evaluation recommended rather than be the gold
standard for a comprehensive evaluation of HIV prevention.
The first draft of the guidance was provided to more than 20 CDC staff
members, state AIDS directors, state and local health department staff members,
HIV prevention provider staff members, and expert evaluators. Reviewers provided
comments to CDC in writing as well as during a meeting on June 4-5, 1998.
Written and verbal comments from all reviewers were incorporated into a revised
version of the draft, which was then provided to state AIDS directors before
they met with CDC on July 20-21, 1998, to discuss the content and format. Their
comments led to additional changes, which were discussed at a NASTAD-wide
conference on August 3, 1998. A meeting with CBO representatives on October 5,
1998, contributed to efforts by shedding light on the issues affecting health
departments’ grantees. A final meeting with NASTAD representatives on May 6,
1999, led to the decision to employ this document as a resource book that
complements requirements and activities featured in CDC’s companion document,
Evaluating CDC-funded Health Department HIV Prevention Programs—Volume 1:
Guidance and CDC Announcement 99004.
Comments received during and following all meetings guided the finalization
of this document and completion of forms for collecting HIV prevention community
planning, process evaluation, and outcome monitoring data. The forms were then
pilot-tested by six CDC-grantee health departments, whose feedback was
invaluable in the improvement of the forms and data collection instructions.
Additionally, five states provided CDC with in-depth information about their
evaluation data systems; this provided further insight regarding current
evaluation activities and implementation issues.
Back to top
Conceptual Framework for HIV Prevention and Evaluation
The HIV prevention community planning initiative is a participatory planning
process intended to document jurisdictions’ HIV prevention needs, prioritize the
needs, and identify ways for meeting them. The overarching intent of the
initiative is to lay a foundation for the funding and implementation of
interventions that are expected to reduce HIV risk behaviors among target
populations and, eventually, to reduce HIV transmission in each jurisdiction.
The framework for evaluation (see Figure 1.1) is based upon a conceptual
framework referred to as the theory-driven evaluation perspective (Bickman 1990;
Chen 1990; Chen and Rossi 1992; Weiss 1997), which holds that a successful
evaluation of an initiative or program needs to systematically examine the
crucial components and the linkages between them.
The eight components of HIV prevention programming shown in the framework
comprise the theoretical progression of events necessary to create a
comprehensive HIV prevention program. Each oval represents an evaluation
activity designed to evaluate one or more of the components (as delineated by
the arrows). The components are described in the following paragraphs, while the
evaluation activities are summarized in Table 1.1 and in the chapter
descriptions at the end of this chapter.
Component 1. All state, territorial, and city health departments that
receive funding from CDC use the monies to form and maintain one or more HIV
prevention community planning groups (CPGs) as required by CDC’s HIV prevention
community planning initiative, introduced in 1994. The CPG is to be
representative of the epidemic in the health department’s jurisdiction and,
according to the requirements of the initiative, is to implement five core
objectives (Table 1.2). The CPG draws on an epidemiologic profile, a needs
assessment, behavioral science, community norms, and other data to determine HIV
prevention needs and identify interventions for responding to them.
Component 2. The CPG develops a comprehensive HIV prevention plan based
on prioritized needs and interventions.
Component 3. The health department incorporates elements of the plan into
its annual application for CDC funding. CDC reviews the application and awards
funds to the health department, which draws on the funds for HIV prevention
planning activities as well as for HIV prevention interventions.
|COMPONENTS IN HIV PREVENTION PROGRAMMING SUPPORTED BY CDC FUNDS
||TYPES OF HIV PREVENTION EVALUATION ACTIVITIES
||Health department grantees should develop evaluation plans before
beginning evaluation activities (Chapter 8 of this document describes
this type of evaluation activity).
- Jurisdictions implement the HIV prevention community planning
|Evaluation of the community planning process and the
development of the comprehensive HIV prevention plan. (Chapter 2)
- The planning process results in a comprehensive HIV
prevention plan (or plans) for each jurisdiction.
- Health departments propose a budget based on the
comprehensive HIV prevention plan.
|(See type of evaluation activity for Component 5)
- Health departments develop HIV prevention
interventions that they will implement, and their potential grantees
develop proposals for HIV prevention interventions (all are based on
comprehensive HIV prevention plans).
|Evaluation of intervention plans to determine each
intervention’s soundness and feasibility of interventions and to assess
its correspondence to the comprehensive HIV prevention plan. (Chapter 3)
- Health departments allocate funds to CBOs and other
providers in the jurisdiction.
|Evaluation of linkages between comprehensive HIV
prevention plan, application for funds, and resource allocation.
- Health departments, CBOs, and other providers
implement HIV prevention interventions.
|Process monitoring and evaluation of implementation of
interventions to assess each program’s conformity to its design, program
implementation, or the extent to which it reaches its intended audience.
- Funded interventions result in short- and mid-term
behavioral, social, and environmental outcomes.
|Outcome monitoring of HE/RR (health education/risk
reduction) individual- and group-level interventions to assess
achievement of interventions’ outcome objectives. (Chapter 6)
- Funded interventions result in long-term behavioral
changes leading to a reduction in HIV transmission.
|Monitoring and evaluation of the overall impact of HIV
prevention activities in the jurisdiction. (Chapter 7)
|Five Core Objectives of HIV Prevention Community Planning
- Foster the open and participatory nature of the community
- Ensure that the community planning group(s) reflects the
diversity of the epidemic in the jurisdiction, and that experts in
epidemiology, behavioral science, health planning, and evaluation
are included in the process.
- Ensure that priority HIV prevention needs are determined based
on an epidemiologic profile and a needs assessment.
- Ensure that interventions are prioritized based on explicit
consideration of priority needs, outcome effectiveness, cost
effectiveness, social and behavioral science theory, and community
norms and values.
- Foster strong, logical linkages between the community planning
process, plans, applications for funding, and the allocation of CDC
HIV prevention resources.
Component 4. The health department, CBOs, and other provider agencies
develop intervention plans that respond to needs identified in the comprehensive
HIV prevention plan as well as those identified by the health department.
Component 5. The health department reviews proposals (which contain
intervention plans1; see definition below) submitted in response to
requests for proposals (RFPs) and other funding mechanisms and determines which
interventions to fund. The health department works with providers to ensure the
interventions selected will be sound and feasible.
||Components 2, 3, and 5 correspond with HIV prevention
community planning core objective 5, which states that there should be
clear correspondence between strategies and activities in the plan, the
health department’s CDC funding application, and interventions funded by
the health department with CDC funds.)
Component 6. The health department and its grantees implement HIV
Component 7. The implemented interventions are expected to lead to
changes in risk determinants among intervention participants.
Component 8. The changes in risk determinants among intervention
participants is expected to lead to a reduction in HIV transmission in the
health department grantee’s jurisdiction.
Ideally, to ensure that the components of HIV prevention are implemented with
the highest quality and contribute effectively to reducing HIV transmission,
each component would be evaluated and the findings used for program and policy
improvement as well as assessment of local and national progress. Table 1.2
lists the eight components and the six types of evaluation activities that
correspond to them. Each evaluation activity will be discussed in a chapter of
this document. Table 1.3 highlights the benefits of evaluation for various
|BENEFITS OF HIV PREVENTION EVALUATION
|Community & Provider Benefits Health
- Ensures the quality of service delivery
- Ensures that HIV prevention resources are successfully reaching
- Guides resource allocation
- Documents progress of programs
- Improves programs
- Identifies programs that are effective or ineffective
- Enables the application of findings, which enhances credibility
and increases community support
- Increases motivation among staff and volunteers
- Increases the likelihood CBOs will be viewed positively by
private and public funders
- Fulfills federal reporting
- Describes the status of HIV
- Provides the health
of HIV prevention service
- Assists HIV Prevention
Groups in assessing
statewide patterns of service
- Documents the need for HIV
prevention services to the
state legislature and
- Documents the need for HIV
prevention services to the
- Guides resource allocation
- Ensures that funds are being
used as intended
- Fulfills reporting requirements to federal policymakers
- Assists CDC project officers in providing necessary technical
assistance to health department grantees
- Improves policies regarding HIV prevention program
Back to top
Contents of This Document
Each chapter in this guidance addresses one type of evaluation activity that
corresponds to CDC’s conceptual framework. Although the format of chapters varies slightly to
accommodate variation in topics and emphasis, all the chapters include an overview describing the
purpose of the chapter, a discussion of reasons for conducting the particular type of evaluation, an
outline of data elements that could be collected, a list of references and resources, and, when applicable,
methodology and example forms for collecting and analyzing relevant data.
The order of the chapters does not correspond directly to the sequence of HIV
prevention components featured in Table 1.1. Instead, the chapters are ordered to
achieve a logical flow based on the types of data expected for each type of evaluation in the event that
grantees conduct each type. Consequently, the chapter on developing an evaluation plan comprising all
types of evaluation is provided at the end of the guidance so that grantees will understand the
evaluation activities they may be conducting before incorporating them into a plan.
Chapter 2, Evaluating the HIV Prevention Community Planning Process,
addresses the importance of evaluating implementation of the HIV prevention community planning
initiative as the first component in the HIV prevention continuum. The chapter outlines the core
objectives of the initiative and the steps involved in fulfilling the objectives. It then
discusses measures for determining fulfillment of each objective and makes recommendations for using evaluation
findings. The appendix features example surveys containing core questions for CPG co-chairs and
members as they evaluate the process.
Chapter 3, Designing and Evaluating Intervention Plans, highlights
steps health departments could take when selecting interventions and ensuring that they are designed to be
as effective as possible. To that end, the chapter describes desirable elements of an intervention
plan. As described in Footnote 1, an intervention plan is a description of the intended
intervention that justifies the elements chosen to achieve desired outcomes and ensures that resources and
logistics are adequate to meet the needs of the proposed intervention. It contains information about
the target population(s), process and outcome objectives, protocol, budget, and
staffing. It might be contained in a proposal for funding or other document describing a planned
The chapter also provides recommendations for evaluating and improving the
relevance and scientific soundness of each element so that health departments can assist providers in
revising intervention plans as needed and ensuring that they correspond to recommendations set
forth in the comprehensive HIV prevention plan for the jurisdiction.
The chapter concludes with a set of variables for describing each
intervention that will be funded by the health department and for determining the extent to which the
interventions correspond to strategies outlined in the comprehensive HIV prevention plan. Example forms
for collecting intervention plan data are in the chapter’s appendix.
Chapter 4, Monitoring and Evaluating the Implementation of HIV Prevention
Programs, is intended to provide health departments with a working knowledge
of process monitoring and evaluation that will enable them to evaluate
implementation of their own interventions as well as to assist health department
grantees in the evaluation of their interventions. For purposes here, process
monitoring and evaluation is the assessment of each program’s conformity to
its design, program implementation, or the extent to which it reaches its
intended audience. This includes routine documentation of characteristics of the
people served and the services that were provided. A description of
implementation should be evaluated on its own merits as well as compared with
the intervention plan, which describes the intended objectives of the
The chapter takes into account the importance of evaluating implementation in
general and of referring to intervention plans to determine if interventions are
implemented as intended. To facilitate both efforts, the appendix features
example data collection forms that guide providers in the collection of
monitoring data as well as in the comparison of actual activities to intended
ones. Ideally, one form would be completed by the health department or its
grantees for each intervention.
Chapter 5, Evaluating Linkages Between the Comprehensive HIV Prevention
Plan and Resource Allocation, responds to the need for evaluating
fulfillment of HIV prevention community planning core objective 5. To this end,
the chapter describes the ways in which health departments and CPGs can assess
whether there are clear and substantial linkages between the comprehensive HIV
prevention plan, the CDC funding application, and allocation of resources for
HIV prevention interventions. The recommended methodology includes several steps
that yield pertinent data that can be used to:
- identify HIV prevention activities in the health department’s
- determine which strategies in the plan are included in the annual CDC
- determine which strategies in the plan are being enacted in the
jurisdiction and which are not; and
- identify interventions that are being implemented but do not correspond
to the plan.
To facilitate this process, the chapter’s appendix includes several example
Chapter 6, Monitoring Outcomes of Health Education/Risk Reduction
Individual- and Group- Level HIV Prevention Interventions, describes ways of
monitoring the achievement of outcome objectives for individual- and group-level
counseling interventions. Outcome monitoring involves the measurement of
progress in achieving the goals and objectives set forth in intervention plans
for each intervention. These measurements assess the effects of counseling
interventions on client outcomes (i.e., knowledge, attitudes, beliefs, and
The chapter discusses the differences between outcome monitoring and outcome
evaluation (discussed in detail in Chapter 7), the development of tools to
collect outcome monitoring data, when to collect the data, and issues in
analyzing the data.
Chapter 7, Evaluating Outcomes and Monitoring Impact of HIV Prevention
Programs, examines the important and complex subject of program
effectiveness and balances the need to be “scientific” with outcome evaluation
recommendations that are straightforward and feasible. To that end, the chapter
proposes ways to select interventions for outcome evaluation and outlines steps
for conducting outcome evaluation. It also describes the advantages and
disadvantages of various research designs and discusses incorporation of process
data (discussed in Chapter 4) into outcome evaluation. Throughout this document,
outcome evaluation is defined as the use of rigorous methods and designs
to assess the effects of an intervention.
Chapter 7 also briefly discusses impact monitoring and evaluation,
which assesses the cumulative effects of all HIV prevention activities in a
Chapter 8, Developing an Evaluation Plan, focuses on the importance of
developing a concrete, comprehensive evaluation plan before embarking on
evaluation activities. The chapter points out that creating a plan requires the
identification of local capacity, experience, resources, and technical
assistance needs in an effort to set a realistic and worthwhile course for
evaluation that addresses both local issues and national requirements. The
chapter also describes the six types of evaluation that ideally would be
included in an evaluation plan and lists basic elements of a plan. Furthermore,
the chapter strongly encourages participation of stakeholders throughout the
evaluation process, beginning with creation of the evaluation plan.
Back to top
References and Resources
Bickman, L., ed. Advances in program theory. San Francisco: Jossey-Bass,
Centers for Disease Control and Prevention. Cooperative agreements for
human immunodeficiency virus (HIV) prevention projects program announcement and
availability of funds for fiscal year 1998 (Announcement 99004). Atlanta:
Centers for Disease Control and Prevention, 1998.
Chen, H.T. Theory-driven evaluations. Newbury Park, CA: Sage, 1990.
Chen, H.T. and Rossi, P.H., eds. Using theory to improve program and
policy evaluation. Westport, CT: Greenwood, 1992.
Weiss, C.H. How can theory-based evaluation make greater headway?
Evaluation Review 1997; 21(4):501-524.
Back to top
Evaluating the HIV Prevention Community Planning Process
||In this document, intervention plan refers to a
description of a planned intervention strategy for a target population. It
contains information about the target population(s), process and outcome
objectives, protocol, budget, and staffing. It might be contained in a proposal
for funding or other document describing a planned intervention. The minimum
report of intervention plans to CDC is described in
Chapter 3 of Volume 1.