Evaluating HIV Education Programs

Even though the program's decision makers will ultimately decide the sorts of evaluative evidence you should collect, you should certainly encourage them to gather behavioral data in nearly all evaluations of HIV education. Many HIV education programs only attempt to influence students' knowledge regarding HIV. Yet ample evidence indicates that knowledge-only programs typically have scant influence on students' behaviors.

Ideally, you should encourage the use of an assessment strategy in which evidence is gathered about students' behavior, skills, affect, and knowledge. The nature of the HIV education program itself will prominently determine which outcomes you should measure.

Of the four types of outcome data from students, the most important is behavioral data. Strive to collect student behavioral data if at all possible. This will sometimes oblige you to provide education and information to local community groups to overcome obstacles on the collection of sensitive behavioral data. Try as hard as possible to assemble evidence of the HIV education program's impact on students' HIV-risk behaviors. Without such evidence, a misleading picture of the program's effectiveness can emerge. HIV education programs that enhance only students' knowledge or attitudes may be judged effective when behavioral data would indicate otherwise.

It is difficult, of course, to demonstrate that an HIV education program has produced genuine changes in young people's behaviors. Part of the difficulty lies in having a sufficient period of time to discern changes in behavior. Program effectiveness may not be apparent for six or more months following the intervention, particularly if rates of sexual intercourse are low. Nevertheless, the program's staff has a responsibility to judge its efforts according to the changes that take place in students' behaviors.

Return to top

How to acquire suitable assessment devices

Once you have decided to measure the four types of student outcomes we have been discussing, where do you get your assessment instruments? There are two possible ways to proceed. You can either construct the instruments yourself or use (perhaps adapt) existing instruments. In selecting, adapting, or constructing your assessment devices, it is extremely important to ensure that they match your program objectives. Measuring an attitude or behavior change that was not sought as part of your instructional program may set up the program for failure.

The problem with creating your own assessment devices is that wording questions to assess behavior, skills, and attitudes is exceedingly tricky. Most educators have substantial experience in developing knowledge tests, but those sorts of assessment instruments are far easier to create than the other three types. Unless you have training and experience in the development of assessment instruments, it makes much more sense to use existing ones.

A set of assessment instruments designed to evaluate HIV education programs for students in grades 5-7 and 7-12 is provided in other booklets contained in this handbook. Start first by carefully considering whether some of these assessment devices will meet your needs; if not, you may need to create or adapt your own instruments. Because the development of acceptable assessment instruments for HIV education evaluation is extremely difficult, however, try to enlist the assistance of experienced test-developers to ensure the quality of your instruments.

Return to top

Securing permission to gather data

Asking students questions about their sexual activities is considerably different from asking them about the Civil War. Because sexual activity is the most common way HIV is transmitted, your assessment instruments will often contain questions about students' sexual behaviors. It is essential that you clear your intended assessment instruments with appropriate school-district authorities. A special review group consisting of educators, parents, and other citizens will often have been established to judge the acceptability of HIV education materials and data-gathering instruments. The assessment instruments included in this handbook should be cleared by a local review group.

You should follow established district procedures in the use of assessment instruments dealing with sensitive subjects such as sexual conduct or drug use. Some districts require that either active informed consent or passive informed consent be secured from parents of students prior to the administration of such assessment devices. With active informed consent, a letter is sent to a student's parents or guardians describing the general nature of the intended data-gathering and asking permission for the student to complete the assessment instruments described. This letter must be signed by parents or guardians indicating their permission to have the data-gathering instruments administered to the student. With passive informed consent, a similar descriptive letter is sent to the student's parents or guardians. They are required to sign and return it, however, only if they do not wish the student to complete the assessment instruments. Obviously, because active informed consent requires the receipt of a signed authorization letter from parents, it is more difficult to implement. Most school districts already have policies in place regarding whether active or passive informed consent is required for data gathering.

The sorts of assessment instruments that might offend local citizens varies greatly among communities. This is an opportunity for you to play a significant educational role with local officials. If fears of citizen disapproval lead to the elimination of questions dealing with key HIV-risk behaviors (such as sexual behavior), you will be unable to discern whether the HIV education program is accomplishing some of the outcomes that it ought to accomplish. You may need to apprise local officials of the deadly threat to students engaging in HIV-risk behaviors and of the consequent peril to those students if educational programs to reduce HIV-risk behaviors are ineffective. When local officials, parents, and guardians are made aware of this serious potential, they will usually allow reasonable questioning about high-risk behaviors.

Return to top

Confidentiality considerations

Once you have secured approval to administer suitable assessment instruments as part of your HIV education evaluation, you must structure the data gathering to increase the likelihood of getting truthful responses from students. To promote this, you should employ as many procedures as possible to enhance anonymity. Any such procedures should be announced to students in advance to assure them you do not intend to associate them with their responses.

Students should complete all instruments anonymously. Moreover, to remove the possibility that an individual's handwriting can be recognized, students should not be asked to write any words on the instruments. Instead, have students use only checkmarks or similar sorts of responses to all items. Similar, nonidentifiable pencils or pens should be used by everyone. In addition, students should place their own completed instruments in large envelopes or opaque containers that help avoid identifying the respondent. If possible, arrange seating to make it difficult for students to see each other's answers. (Several of the evaluation instruments in this handbook employ a response scheme specifically designed to prevent students from "inadvertently" seeing how others respond to items dealing with sensitive subjects such as sexual behavior or the use of illegal drugs.)

In short, make sure that you have taken all reasonable steps to assure students of confidentiality and anonymity. Even then, of course, not all students will respond honestly to all questions. If, however, your efforts to ensure confidentiality boost the number of candid responses, your interpretations of the resulting data will obviously be more accurate. Fortunately, your quest is to evaluate program effectiveness rather than the status of individual students.

Return to top

What about qualitative data?

So far we have dealt with the sorts of data gathered via fairly traditional quantitatively based assessment instruments. There are also a number of more qualitatively oriented data-gathering procedures, such as focus group interviews or one-on-one interviews with students who have participated in an HIV education program. These types of procedures often provide a rich source of explanatory evidence to help decision makers better understand the nature of the evidence you supply to them. For example, a few focus group sessions with students who have completed an HIV education program can prove particularly illuminating if the evaluator is trying to figure out which parts of the program worked well and which parts did not.

Return to top

Final thoughts about Guideline 2

It is difficult to say that one guideline is more important than another, for all guidelines should play pivotal roles in your evaluation of an HIV education program. Guideline 2, however, leads directly to the assembly of the chief evidence you will use. To fail to identify appropriate assessment instrumentation, therefore, is to lose the whole evaluative ball game.

Few test developers are skilled enough to craft instruments that tease out subtle nuances in students' attitudes or garner honest answers to sensitive questions about sexual activities. The assessment instruments provided in this handbook were developed and field-tested by measurement experts and reviewed by specialists in the field of HIV prevention. You should review these instruments to see if they suit your needs. You should also consider the usefulness of qualitative data-gathering approaches, because schemes such as focus group interviews provide evidence that blends well with more quantitatively oriented evidence.

Return to top

Back to Booklet 1 Table of Contents

Back to Handbook for Evaluating HIV Education - Introduction

Healthy Youth Home | Contact Us

CDC Home | CDC Search | Health Topics A-Z

Privacy Policy | Accessibility

This page last updated April 29, 2005

United States Department of Health and Human Services
Centers for Disease Control and Prevention
National Center for Chronic Disease Prevention and Health Promotion
Division of Adolescent and School Health