II. Standards and Suggested Practices

H. Data analysis and utilization

After CTR data have been cleaned (resulting in a clean data set), the data analysis and utilization phase of the data life cycle begins. The analysis of CTR data most often involves converting record-level data into an aggregate summary of CTR program services and outcomes. The results of effectively designed and conducted analysis of CTR data can help program managers, staff, and stakeholders understand more about the CTR services that have been delivered within the jurisdiction over time. Program managers can use those results to make informed decisions about actions that may be needed and to design plans that better reflect the program data.

CTR data can also be used in conjunction with other community health data, including HIV surveillance data, to provide a more complete understanding of the HIV epidemic and our response to it. However, before CTR data can be useful as a decision tool, they must be appropriately analyzed, interpreted, and prepared for dissemination. Generally, CTR data analyses for monitoring and reporting purposes involve the use of relatively simple descriptive statistics, such as frequency counts and cross-tabulations of variables for a reporting period (e.g., test results, demographics, risk factors). CTR data staff should routinely document details of changes to the original data (e.g., recodes, statistical procedures, data transformations, imputation or exclusion of outliers or missing data).

Standard H1: Health departments should develop a CTR data analysis plan for use in monitoring and evaluating program services and programmatic outcomes.

Health departments should develop a CTR data analysis plan for monitoring and evaluating service delivery. The data analysis plan will inform the CTR variables that are collected by service providers (see Standard D1 in the Data Collection section), and in this process identify the variables needed to answer important programmatic questions. This plan should be updated at least annually to reflect the goals and objectives described in the CTR program plan and include the data needed to adequately inform standard performance indicators that are required by funders. The design of the data analysis plan should make it easy to compare data from various reporting periods, thus to identify testing trends and assess the impact of advances (e.g., changes in policy or testing technology) in HIV prevention. Health departments should share their data analysis plan with CTR service providers and encourage those providers to use their own data to answer monitoring and evaluation questions. Examples of monitoring and evaluation questions include the following:

• Number of HIV tests provided (e.g., how many HIV tests were provided in the jurisdiction during 2007? how many were provided during 2008?)
• Percentages, by race/ethnicity, of HIV test results that were positive (e.g., what percentage of HIV-positive test results were for African Americans?)
• Percentages, by race/ethnicity, of HIV test results received by clients (e.g., were Hispanics less likely than members of other races or ethnicity to receive their test results?)
• Number of referrals made to clients who tested HIV-positive (e.g., what percentage of positive test results were linked to referrals for treatment services? were HIV-positive men who have sex with men [MSM] more or less likely than HIV-positive persons with other risk factors to be referred for medical care and treatment?)
• Percentage of clients referred for medical care and treatment who attended their first appointment (e.g., what percentage of the HIV-positive persons who were linked to care actually received services? were MSM more or less likely than persons with other risk factors to receive medical care and treatment after referral?)

Top of Page

Standard H2: Health department staff should develop and adhere to policies for sharing and reporting data that prevent the disclosure of individual client data.

Health departments should develop policies for sharing and reporting data regarding the types of CTR data that can be shared, with whom, and procedures for sharing these data. The data-reporting policy should include instructions about reporting very small cell sizes (usually in a table). Typically, to preclude the identification of individuals, counts of five or fewer are suppressed.

A breach of confidentiality can occur when staff members release information that may identify an individual and reveal confidential information about that person. Data-reporting policies can be included in the data QA protocol and should provide staff with guidance for situations that may lead to a breach of confidentiality.

Health departments should maintain a system for tracking fulfilled data requests that includes the name and organization of the data requestor and the date of request. This will allow health departments to monitor who is using CTR data and how it is being used.

Resources

42. Web site with guidelines for working with small numbers (Colorado)

    This Web site features guidelines for addressing both confidentiality and statistical issues in working with small numbers; explains the risk in using small numbers in tables, what constitutes a breach of confidentiality, how to reduce the risk of a breach, the reliability of statistics based on small numbers, and how to address statistical issues.

43. Data release checklist (Colorado)

    One-page checklist to help staff decide whether data can be released in response to an internal or an external request; includes questions about the release of individual information, compliance with guidelines and statutes, and the need for data-use agreements.

• Guidelines for sharing confidential information (Florida) [cited previously as Resource 22]

  Guidelines for medical providers and AIDS service organizations that need to share information to facilitate medical follow-up; includes an example of a consent form, approved by legal counsel, which may be adapted for use in any community.

Top of Page

Standard H3: Health departments should disseminate annual CTR data reports that are clear, unbiased, and accessible to stakeholders.

Health departments should develop and distribute regular reports to stakeholders in an effort to inform them about current trends in CTR services. Although not all health department data reports are intended for public distribution (e.g., internal progress reports), summary reports should be disseminated at least annually. To facilitate the dissemination of CTR program information, health departments may also consider maintaining a Web site on which they post reports, updates, and fact sheets. This is effective for disseminating new data between publication cycles of annual reports.

Data summary reports of current CTR trends provide important information to service providers, health departments, and community members. These reports are critical in prevention planning and resource allocation and are most useful to data users when the reports are well organized and presented in a format that allows for meaningful interpretation and comparison. If standard data collection and reporting procedures are used, CTR data can easily be organized into tables to illustrate meaningful comparisons of data from one reporting period to the next, thus helping to identify trends. Table 7 shows HIV tests results, by characteristics of persons tested during a 6-year period.

Table 7. HIV test results during a 6-year period

	1999 No.	2000 No.	2001 No.	2002 No.	2003 No.	2004 No.
Age at test (yrs)
<5	4,490	4,245	2,281	1,692	1,327	1,144
5-12	4,167	3,348	3,308	2,834	2,230	2,215
13-18	228,160	221,092	215,666	206,743	189,964	190,791
19-24	514,922	529,900	548,987	548,109	627,908	644,765
25-34	538,044	536,682	546,184	541,040	523,182	530,509
36-44	362,647	374,250	383,742	375,445	357,510	352,890
45-54	141,071	154,013	167,313	173,425	175,146	183,374
55-64	36,267	39,508	43,269	45,870	46,889	49,684
≥65	12,722	13,248	15,522	15,324	14,577	14,286
Not reported*	27,029	28,767	35,821	35,539	29,359	23,076
Race/ethnicity
White, not Hispanic	803,432	771,686	750,317	732,050	687,988	687,649
Black, not Hispanic	641,580	677,469	715,401	697,064	686,336	735,115
Hispanic	338,111	370,979	400,789	421,462	397,617	372,239
Asian/Pacific Islander	30,318	30,014	31,159	32,349	31,298	31,237
American Indian/Alaska Native	12,075	11,880	12,245	11,934	12,039	11,205
Other	23,787	25,169	28,841	29,731	32,269	34,596
Not reported*	20,316	17,854	23,309	21,431	20,345	20,693
Sex
Male	871,545	910,604	950,274	944,762	909,341	932,579
Female	987,586	979,789	998,421	989,736	942,743	947,369
Not reported*	10,388	14,658	13,366	11,523	15,808	12,786
Total	1,869,519	1,905,051	1,962,061	1,946,021	1,867,892	1,892,734

*Includes records without a value for this variable.

Adapted from HIV Counseling and Testing at CDC-Supported Sites, United States, 1999–2004.

Including graphs in CTR data reports can be very helpful to data users, generate staff buy-in regarding the usefulness of the data, and highlight findings that might be obscured in a large, complex table. Data graphs should be straightforward and simple. As illustrated in Figure 12, graphs can be especially useful for demonstrating comparisons of testing over time. In this graph, U.S. testing trends from 1999 through 2004 are illustrated by plotting the percentage of HIV-positive test results for each transmission category (y axis) and the reporting period (x axis). Graphs should always be titled, and each axis should be clearly labeled. Graphs should also be labeled with the unit of measurement on each axis and should have the proper scale (with the appropriate high and low values) so that the magnitude of difference is not over- or underrepresented.

Figure 12. Percentage of HIV-positive test results, by risk category of persons tested, 1999–2004

^aIncludes persons who did not report a risk factor, whose record contained no data on risk factors, or who reported other risk factors (i.e., perinatal exposure, hemophilia, receipt of blood transfusions, or health care exposure).

^bIncludes persons who had sex with partner at risk, who had a diagnosis of a sexually transmitted disease, who exchanged sex for drugs or money, who had sex while using noninjection drugs, who were victims of sexual assault, or whose only risk factor was heterosexual contact.

Adapted from HIV Counseling and Testing at CDC-Supported Sites, United States, 1999–2004.

Before CTR data reports are disseminated, they should be reviewed by statistical or data analysis staff to ensure that the information is displayed appropriately and accurately. When data errors in reports are identified, health departments should act promptly to determine whether an error has been published; if yes, they should publish a correction as quickly as possible. In addition, data reports should include the terms and conditions under which data analyses are performed, as well as the limitations regarding interpretation of the data. The following are common examples of such limitations:

“Not every case represents one client: some clients are tested more than once within a year. Therefore, the term testers does not refer to individuals—rather, some of the testers are the same person.”

“These data do not include HIV tests performed at private clinics, doctor’s offices, or hospitals.”

Resources

44. Florida Department of Health Web site—trends and statistics

    Web site that features epidemiologic profiles, fact sheets, monthly surveillance reports, slides, HIV counseling and testing annual reports, quarterly HIV data, and many other resources.

45. Texas Department of State Health Services Web site—reports

    Web site that features epidemiologic profiles, HIV and STD surveillance reports, reports to stakeholders, and many other resources.

46. Colorado Department of Public Health and Environment—HIV/STD surveillance

    Web site that features epidemiologic profiles, HIV and STD surveillance reports, and many other resources.

47. HIV counseling and testing at CDC-supported sites, United States, 1999–2004 (CDC)

    Data (test-level or aggregate-level) on the number of tests, the results of those tests, and information about the demographics and behavioral risk factors of the persons tested; includes (see Technical Notes) the limitations and comparability of the data.

Top of Page

Standard H4: Health departments should serve as a resource to service providers for the analysis and utilization of their data.

Whenever possible, health departments should be available to provide data analysis support and technical assistance to service providers. This support could include guidance in developing local evaluation and monitoring plans, explanations of the effective use of CTR data, and assistance in responding to specific requests regarding funding applications or community reports.

Health departments should encourage service providers to use their own data (e.g., demographics and patterns of risk behaviors) to enhance the planning and the provision of services. For instance, health departments can encourage service providers to use data from CTR programs, in combination with data regarding risk behaviors, to help direct services to specific populations and to set priorities and goals for future planning periods. Service providers can compare this information with profiles of high-priority populations to determine whether they are reaching the types of clients they planned to reach and whether they should modify the range or type of services they offer. It is also important for service providers to discuss recent community HIV testing and risk behavior trends with health department staff. This information can help inform CTR data collection requirements, special studies, and future analyses.

Top of Page