Note that CDC’s “Evaluation Guidance” covers minimum
expectations for the evaluation of HIV/AIDS community
planning and prevention programming under Program
Announcement 99004. You may add evaluation activities beyond
those discussed in the “Guidance” based on recommendations
from your community planning group (CPG) or an evaluation
advisory committee you’ve set up to guide the development of
your evaluation plan and oversee evaluation activities.
Committee members could include CPG members, CBO staff, and
health department contracts and computer staff.
In addition to the recommendations in CDC’s “Supplemental
Handbook” on developing an evaluation plan, you may find it
helpful to consider some basic questions as the framework
for your plan. These questions are who,
why, and how.
“Why” refers to the reasons you’re developing and
carrying out the evaluation plan. Other than fulfilling CDC
requirements, there are things you want to accomplish or
questions you want to answer. For example, what you want to
accomplish could be stated as goals:
- assess the quality of interventions;
- improve the community planning core objective to
foster openness and participation;
- identify how resources are allocated among target
- use monitoring and evaluation findings to improve
You may want to answer certain questions, such as:
- how do CPG members rate the priority setting
- who are the clients that receive prevention case
- how effective are group education programs?
- in what ways should evaluation results be used to
improve community planning and service delivery?
Center your evaluation plan on goals or questions.
To operationalize the “whys” of your evaluation plan, you
need to propose actions. Actions consist of the objectives
or activities needed to address goals or answer questions.
There will probably be a number of actions under each goal
or question; for example:
- collect baseline data;
- review risk assessment form;
- analyze data;
- disseminate results.
It is up to you to decide how detailed your evaluation
plan should be. A more detailed plan could include steps
under each action.
“Who” refers to the resources you’ll use to carry out
actions (i.e., objectives or activities). For example,
resources could include:
- health department staff;
- grantee staff;
- CPG committee;
- university researcher.
“What” includes the items or materials you need in
addition to resources to carry out your objectives or
activities; for example:
- quarterly reports from grantees;
- current computer run;
- specific data collection tool, such as CDC-
adapted process monitoring form or
- questionnaire for CPG members.
Your evaluation plan may want to specify where activities
will take place (or from whom data will be collected); for
- all grantees funded under program announcement
- a sample of CBOs that target African-American MSM
(identify the agencies);
- final CPG meeting of the year.
It is important that you identify timeframes for your
activities. You want to start with CDC’s deadlines and “back
into” timeframes. For example,
- outcome evaluation is due September 2003;
- develop plan (or evaluation protocol) for outcome
evaluation by September 2001;
include a logic model for the intervention being
- identify data collection tools (measures,
instruments) by January 2002;
- pilot test data collection tools by July 2002;
- carry out data collection between September 2002
and March 2003;
- analyze data by August 2003;
- prepare report by September 2003.
“How” refers to ways in which actions will be carried out
and should include activities for data analysis. Examples of
- sort data by age, race, ethnicity and gender using
relational data base;
- use “t” test to assess statistical significance of
differences between pre- and posttest scores;
- reach consensus at evaluation advisory committee
TECHNICAL ASSISTANCE NEEDS
Remember that CDC’s “Evaluation Guidance” asks you to
indicate your evaluation technical assistance needs.
It is crucial that each health department have an
evaluation plan that satisfies CDC’s requirements and also
meets the evaluation needs of CPG members and providers and
consumers of prevention services. Therefore, each plan
should be custom designed for the jurisdiction. However, in
terms of format, you may want to include a table with
The following example may prove useful as you develop or
revise your evaluation plan.
EVALUATION PLAN EXAMPLE
Who are the clients that participate in health
education and risk reduction programs?
|Determine which data to collect
||Evaluation Coordinator and eval. advisory committee
||July 1 eval. advisory committee meeting
||Review quarterly report forms; review CDC guidance on
||Evaluation Coordinator; contracts staff; HD computer
||Revised quarterly report form that incorporates CDC
||All grantees will submit hard copies of the report to HD
||Reports are due April 15, July 15, Oct. 15, Jan. 15
||HD computer staff will enter data and generate quarterly
||Evaluation Coordinator; HD computer staff
||Aggregated quarterly reports
||One month after each quarterly report is due
||Using relational data base, sort data by risk
population, race, ethnicity, age, and gender; sort by agency, setting, #
||Evaluation Coordinator, CPG members
||Quantitative data analysis (see action above)
||CPG meeting each quarter
||Qualitative interpretation of data, with implications
for community planning