Skip directly to local search Skip directly to A to Z list Skip directly to navigation Skip directly to site content Skip directly to page options
CDC Home

III. Appendixes

Appendix A. QA standards for HIV CTR data

A. Protocol for data QA

B. Staffing, training, and communication

C. Data security and client confidentiality

D. Data collection

E. Data entry

F. Data management and cleaning

G. Data submission

H. Data analysis and utilization

Top of Page

Appendix B. Resources, by standard

A. Data QA protocol

Resource Description
Standard A1
  1. 2001 Revised guidelines for HIV counseling, testing, and referral
Guidance to policymakers and service providers for developing HIV CTR services and policies in traditional and nontraditional settings.
  1. 2006 Revised recommendations for HIV testing of adults, adolescents, and pregnant women in health care settings
Recommendations to policymakers and health care providers for implementing HIV testing in health care settings.
  1. Quality assurance guidelines for testing using rapid HIV antibody tests waived under the Clinical Laboratory Improvement Amendments of 1988
Guidance for sites using or planning to use rapid test kits to detect antibodies to HIV.
Standard A2
  1. CTR data policy and procedures manual
A template, developed by state health department staff, which service providers can tailor in developing their own data policies and procedures.
  1. Counseling, Testing and Linkages (CTL) data quality assurance guidelines (San Francisco)
Detailed guidelines for service providers to help them complete data collection forms correctly; includes pictures, screenshots, and common mistakes.
  1. CTR program and procedures manual template (San Francisco)
An excellent example of a protocol template that can be easily tailored and that contains comprehensive information on CTR programs as well as data collection and recordkeeping requirements.
Standard A3
  1. National Alliance of State and Territorial AIDS Directors (NASTAD) HIV counseling, testing, and referral services self-assessment tool for state and territorial health departments
Comprehensive information developed by NASTAD to help health departments assess their responsiveness to 2001 CDC’s Revised Guidelines for HIV Counseling, Testing, and Referral; includes information about collecting and entering data.

B. Staffing, training, and communication

Resource Description
Standard B1
  1. Sample position descriptions (Ohio)
Includes examples of position descriptions for an epidemiology supervisor, an epidemiologist, and a data entry operator.
Standard B3
  1. AIRS [AIDS Institute Reporting System] software basic training guide (for data entry)
Detailed example of a health department’s training manual designed for data-entry personnel; provides step-by-step instructions and screenshots for entering data into a local software system.
  1. Memo on training on new HIV test form (Louisiana)
Excellent example of clear communication between the health department and testing sites about the provision of follow-up training on the data collection guidelines for new HIV test forms.
  1. Training and technical assistance log
Document to help health department CTR staff track annual training and technical assistance for service providers to enhance quality assurance of CTR data.
  1. Web site with information on agency forms and materials (San Francisco)
This Web site features complete, updated list of CTR templates, forms, policies and procedures, and other materials that service providers need to fulfill their data requirements: a model of the kind of easy-to-access and easy-to-use information that health departments can make available to service providers via a Web site.

C. Data security and client confidentiality

Resource Description
  1. Web site with the state HIV testing laws compendium
This Web site features a compendium of key state HIV testing laws and policies. State HIV testing laws vary, and many have been revised since the release of CDC’s 2006 Revised recommendations for HIV testing.
  1. Web site with HHS information security program policy (HHS)
This Web site outlines the policy requirements and staff responsibilities of the department-wide information security program; includes management, operational, and technical information.
  1. CDC. Data Security and Confidentiality Guidelines for HIV, Viral Hepatitis, Sexually Transmitted Disease, and Tuberculosis Programs: Standards to Facilitate Sharing and Use of Surveillance Data for Public Health Action. Atlanta, Georgia: Centers for Disease Control and prevention; 2011.
Comprehensive guidance to help state and local health department HIV/AIDS surveillance programs ensure the security and confidentiality of their data. Surveillance programs, compared with most HIV testing programs, have strict guidelines for managing variables because they are more likely to have name-based data or other personally identifying information, but these guidelines offer useful information for CTR programs that are developing, reviewing, or updating security protocols.
Standard C1
  1. HIV/AIDS confidentiality agreement (California)
One-page agreement, including a summary of statutes pertaining to confidential public health records and penalties for unauthorized disclosure and an employee confidentiality pledge.
  1. Web site with privacy and security tool box (Colorado)
This Web site features tools (an incident reporting form, a sensitive information inventory, a facilities security plan, guidelines for workstation and wireless security) that can be used by health departments to develop security and confidentiality training for new employees.
Standard C2
  1. Memorandum of understanding (MOU) for the use of PEMS (CDC)
Agreement between CDC and PEMS users, which outlines specific agreements regarding data collection, entry, storage, use, sharing, retention and disposal, system security, and access privileges; a good example of a written agreement between a central organization and multiple grantees with respect to security and confidentiality.
  1. PEMS security summary (CDC)
Information about the security components of PEMS, oversight of those components, activities related to certification and accreditation, user responsibilities, and requirements for the execution of security agreements between CDC and PEMS users.
  1. Rules of behavior for PEMS agency system administrator (CDC)
This document specifies the formal rules of behavior expected of system administrators and communicates policies and procedures to be followed. These include legal and regulatory requirements, authentication management, and information management.
Standard C3
Comprehensive guidance to help state and local health department HIV/AIDS surveillance programs ensure the security and confidentiality of their data. Surveillance programs, compared with most HIV testing programs, have strict guidelines for managing variables because they are more likely to have name-based data or other personally identifying information, but these guidelines offer useful information for CTR programs that are developing, reviewing, or updating security protocols.
  1. Use of CDC information technology resources CDC-IS-2005-03 (CDC)
Policy for the use of all IT resources owned by, or operated on behalf of, CDC; describes appropriate personal use of resources, prohibited uses, privacy expectations, and responsibilities.
Standard C4
  1. Guidelines for sharing confidential information (Florida)
Guidelines for medical providers and AIDS service organizations that need to share information to facilitate medical follow-up; includes an example of a consent form, approved by legal counsel, which may be adapted for use in any community.
Standard C5
  1. HIV surveillance program confidentiality and data security policy (Michigan)
Surveillance policy, including relevant security and confidentiality requirements, such as maintenance of information, security investigations, release of information, oath of confidentiality, data transfers, and access to information.
  1. Retention guidelines for HIV medical records (Michigan)
This document provides counselors and administrators with guidelines for maintaining HIV/AIDS medical records and describes records retention schedules.
Comprehensive guidance to help state and local health department HIV/AIDS surveillance programs ensure the security and confidentiality of their data. Surveillance programs, compared with most HIV testing programs, have strict guidelines for managing variables because they are more likely to have name-based data or other personally identifying information, but these guidelines offer useful information for CTR programs that are developing, reviewing, or updating security protocols.
This Web site outlines the policy requirements and staff responsibilities of the department-wide information security program; includes management, operational, and technical information.
  1. Sensitive but unclassified information CDC-IS-2005-02 (CDC)
Policy and procedures for safeguarding (storing, disseminating, and protecting) data and documents that are sensitive enough to require protection but that may not be designated as classified information.
  1. American Medical Association (AMA) guiding principles for electronic medical records
Principles of collection, use, and warehousing of electronic medical records and claims data; includes summary of related AMA policy.

D. Data collection

Resource Description
Standard D2
  1. NHME data set (CDC)
Complete list of National HIV Prevention Program Monitoring and Evaluation (NHME) variables (variable number, variable name, value choices [if applicable], definition, instructions, and CDC’s reporting requirements for each variable).
  1. HIV test form and variables manual (CDC)
Detailed explanation of the CTR variables that providers must collect to meet CDC requirements. This manual is focused on the client- and agency-level variables related to CTR services and based on CDC’s revised HIV Test Form (released November 2007). Although not all agencies will use the new HIV Test Form to collect CTR data, the rationales and definitions in this manual are relevant to all health departments funded by CDC to provide CTR services and collect testing data.
  • Counseling, Testing, and Linkages (CTL) data quality assurance guidelines (San Francisco) [cited previously as Resource 5]
Detailed guidelines for service providers to help them complete data collection materials correctly; includes pictures, screenshots, and common mistakes.
  1. Guidance for completing the HIV counseling information form (HIV7) (California)
Guidance developed by the California Department of Public Health to help HIV counselors better understand the HIV Counseling Information Form and thus to record client information consistently; provides a detailed explanation of each item on the form and describes the role of the form in HIV counseling.
  1. Table of HIV agency ID numbers (Louisiana)
Table of agency ID numbers and agency names: an example of clear guidance concerning variables.
Standard D3
  1. Revised HIV test form (released November 2007) (CDC)
Health departments funded by CDC to provide HIV CTR services are required to collect data about these activities. This form is designed to follow the flow of a typical CTR session which helps CTR counselors systematically collect the required CTR variables.
  1. HIV testing encounter form (Maryland)
User-friendly form designed to reduce errors in documenting or entering data.
Standard D5
Detailed guidelines for service providers to help them complete data collection materials correctly; includes pictures, screenshots, and common mistakes.
  1. Tracking spreadsheet for clients who test HIV positive
Excellent example of a tracking spreadsheet to facilitate follow-up with clients who test HIV positive. It includes sections to track preliminary positive rapid test results and confirmed/conventional test results.
  1. Behavioral details consistency checklist (CDC)
Form designed to collect behavioral details while checking for internal consistency of responses provided.

E. Data entry

Resource Description
Standard E1
Complete, updated list of CTR templates, forms, policies and procedures, and other materials that service providers need to fulfill their data requirements: a model of the kind of easy-to-access and easy-to-use information that health departments can make available to service providers.
Standard E2
  1. Data entry protocol for HIV counseling information form (HIV6) (San Francisco)
Detailed instructions and graphics for CTR data entry staff of how to process forms, what information should be on forms, what errors can be found on forms, and what information should be entered in the database.
  1. PEMS user manual (CDC)
Manual developed to help PEMS users understand and use the application (inclusive of all major releases of PEMS as of April 2008, as well as associated patches up to 3.3.1); includes hyperlinks that allow the document to be used as an online tool.
Standard E3
  1. Example of a CTR data error report
Example of an error report from the health department to the service provider (including errors identified on each form to assist in correction and resubmission) and a cover letter to service providers.

F. Data management and cleaning

Resource Description
Standard F1
  • NHME data set (CDC) [cited previously as Resource 27]
Complete list of National HIV Prevention Program Monitoring and Evaluation (NHME) variables (variable number, variable name, value choices [if applicable], definition, instructions, and CDC’s reporting requirements for each variable).
Standard F2
  1. Cody’s data cleaning techniques using SAS Software
Cody, R. (1999). Cody’s data cleaning techniques using SAS software. Cary, NC: SAS Institute, Inc.
Step-by-step instructions for developing data cleaning programs and macros; includes validation checks (e.g., character data, numeric data, missing values, and date values), searching for duplicate records, double data entry and verification by using the COMPARE procedure, SQL-based solutions, and creating data sets by using general-purpose cleaning and validation programs.
  • Example of a CTR data error report [cited previously as Resource 37]
Example of an error report from the health department to the service provider (including errors identified on each form to assist in correction and resubmission) and a cover letter to service providers.

G. Data submission

Resource Description
  1. CT XML File Validation (CDC)
Instructions designed for XPEMS sites using xpems.exe, a simple XML syntax and data validation tool which helps ensure that data conforms to CDC’s requirements; includes screenshots of different errors and reports.
Standard G1
  1. CDC unified process guide—SDN overview (CDC)
Guide to help project teams comply with CDC and federal requirements; describes five services offered by SDN: file transfer, application hosting, user enrollment, digital certificate management, and authentication and authorization.
  1. Using the secure data network to access PEMS program activities (CDC)
Specific information for using SDN, such as enrolling for, and installing, a digital certificate; accessing SDN and PEMS activities; and downloading information from CDC.

H. Data analysis and utilization

Resource Description
Standard H2
  1. Web site with guidelines for working with small numbers (Colorado)
This Web site features guidelines for addressing both confidentiality and statistical issues in working with small numbers; explains the risk in using small numbers in tables, what constitutes a breach of confidentiality, how to reduce the risk of a breach, the reliability of statistics based on small numbers, and how to address statistical issues.
  1. Data release checklist (Colorado)
One-page checklist to help staff decide whether data can be released in response to an internal or an external request; includes questions about the release of individual information, compliance with guidelines and statutes, and the need for data-use agreements.
  • Guidelines for sharing confidential information (Florida) (cited previously as Resource 22)
Guidelines for medical providers and AIDS service organizations that need to share information to facilitate medical follow-up; includes an example of a consent form, approved by legal counsel, which may be adapted for use in any community.
Standard H3
  1. Florida Department of Health Web site—trends and statistics
Web site that features epidemiologic profiles, fact sheets, monthly surveillance reports, slides, HIV counseling and testing annual reports, quarterly HIV data, and many other resources.
  1. Texas Department of State Health Services Web site—reports
Web site that features epidemiologic profiles, HIV and STD surveillance reports, reports to stakeholders, and many other resources.
  1. Colorado Department of Public Health and Environment—HIV/STD surveillance
Web site that features epidemiologic profiles, HIV and STD surveillance reports, and many other resources.
  1. HIV counseling and testing at CDC-supported sites, United States,
    1999–2004
    (CDC)
Data (test-level or aggregate-level) on the number of tests, the results of those tests, and information about the demographics and behavioral risk factors of the persons tested; includes (see Technical Notes) the limitations and comparability of the data.

Appendix C. Common CTR data errors

Type of Data Error

Examples

Possible Resolutions

Missing data are the result when a respondent declines to answer a question, a data collector fails to ask a question or record a respondent’s answer, or a data entry staff member skips the entry of a response.

A CTR client answered 20 questions, but responses for only 15 variables were recorded on the form.
A CTR client declines to answer the question and the field is left blank even when there is a value that can be marked for “client declined to answer.”

Assign value for incomplete or missing data on the basis of known information (e.g., enter or correct site ZIP code information on the basis of the service provider’s location).
Communicate with CTR service provider
to retrieve missing data or update record.

Transcription or transposition errors occur when the data entry person misreads the answer recorded on the form and enters a number or letter that looks like the correct one (but is, in fact, incorrect).

The number:
“1” is entered as “7.”
“0” is entered as the letter “O.”
“39” is entered as “93.”
“1234” is coded as “1324.”

Conduct random spot checks of records
to identify data entry errors.

Out-of-Range errors occur when a number or date lies outside the span of probable or possible values. Many data entry software packages have built-in systems that check whether entered values are out of range.

The variable is the date of most recent HIV test, but the answer is a future date (e.g., 3/3/2025). This response would be considered outside the acceptable range.
The date of provision of results is before the date of the test.
A yes/no question has predetermined numerically assigned values so that “yes” is coded as “1” and “no” is coded as “2.” Thus, coding “7” for this question would be a range error since “7” is not an acceptable value choice.

Correct value for out-of-range data field.
Communicate with CTR service provider to collect accurate data or update record.

Duplication errors occur when the same information is entered in the database more than once.

A CTR client is tested once, and data about that session are entered. The hard-copy data are not clearly marked as “Entered,” and a second person enters those same data.

Determine whether the record is a duplicate or the ID number has been entered incorrectly.
Remove duplicate records or correct the ID number.

Incomplete errors result when a field is not filled out completely.

The five-digit ZIP code field only contains three digits.
The month and year for the date of visit are filled out, but the day is missing.

Communicate with CTR service provider
to complete data or update record.
Review supplemental documentation in client files to obtain missing data.

Interpretation errors occur when a counselor or client misinterprets the meaning of a variable and records incorrect data.

For “self-reported result,” the client misinterprets “positive” to mean a good test result and “negative” to mean a bad result, and reports invalid answer.

Communicate with CTR service provider
to clarify meaning of a variable and correct data or update record.

Inconsistent errors require a comparison of more than one data variable to determine whether there are inconsistencies in the data collected.

If the date of provision of results is before the date of the test, we know that one of those dates must be incorrectly recorded. However, you would not be able to determine that by looking at only one of the data values. Therefore, certain values must be reviewed together in order to reveal errors.

Communicate with CTR service provider
to collect information to resolve inconsistencies.
Assign a value based on other client information.

Top of Page

Appendix D. Methods used to develop the 2009 QA Standards

Developing the 2009 QA Standards for CTR data required several steps. (See Figure 13.) The 2009 QA Standards represent the best available evidence from the public health data literature, expert opinions, document reviews, and interviews with key staff members of health departments.

Figure 13. Steps in the development of the 2009 QA Standards

Step 1: Planning and coordination

To develop standards that CTR staff and service providers would adopt, it was important to establish the topics to be addressed and how the CTR staff and service providers would use the document. A comprehensive review of the literature was important in identifying resources that could be used to improve the quality of CTR data. After reviewing and summarizing the literature, we shared an outline of the standards with internal and external stakeholders so that they could review and provide feedback. To better understand the issues confronted by health departments, we also reviewed health department materials (e.g., progress reports, CTR QA plans, and CTR data submitted to CDC).

Step 2: Assessing current health department CTR data QA practices

Site visits, including group interviews with CTR staff, helped us document health department policies and practices during each step in the CTR data life cycle. The selection of sites entailed consideration of various factors (e.g., history of timely submission of data, completeness of data, and innovative data use practices). Several health departments were identified as sites with a history of submitting high-quality CTR data. Of these, three sites were invited (and agreed) to participate in a 4-day site visit.

Next, we developed a detailed data-collection protocol, which specified each step in preparation for the site visit, the plan and procedures for the site visit, and the procedures to be followed after the visit. All site visitors were trained in the use of the interview guides and onsite procedures. We conducted group interviews with CTR staff involved in each of the stages of the data life cycle. The interviews were audio-taped, and verbatim transcripts were produced for qualitative coding and analysis. We used the findings of this analysis in developing a draft of the 2009 QA Standard

Step 3: Translating information into user-friendly standards

In the next step, we obtained feedback from internal subject matter experts and external stakeholders during a 1½-day consultation. This consultation involved facilitated discussions designed to ensure that the draft addressed the needs of health departments and CTR programs and that it provided practical guidance for improving the quality of CTR data. Stakeholder feedback from the consultation was incorporated into the initial draft of the 2009 QA Standards, along with resources, tools, and templates. One of the primary objectives of this round of revisions was to further develop the document as a user-friendly and practical resource for health department staff.

Step 4: Piloting the transfer of the 2009 QA Standards into practice

To learn from health departments about the usefulness and applicability of the 2009 QA Standards and to facilitate the transfer of the standards to practice at the site level, CDC collaborated with three health departments to pilot the 2009 QA Standards. Health departments for this pilot were selected on the basis of their interest in participating, their geographic location, and the characteristics of their CTR data system (e.g., method of data entry). Each participating health department worked with an assigned project team member to plan, develop, and document policies and procedures. Project team members documented this process in order to understand how effectively the 2009 QA Standards met health departments’ needs as they developed and applied QA practices to CTR data processing.

The pilot process began with a 2-day introductory site visit to each of the three selected health departments. Site visits were followed by a series of seven weekly conference calls, each of which was focused on a specific aspect of the 2009 QA Standards (e.g., data collection, entry, management, submission, analysis, utilization, and security and confidentiality). Team members facilitated the conference calls with health department CTR and service provider staff. In addition to the weekly conference calls with individual sites, a cross-site conference call was conducted so that staff at each of the health department sites could share information and feedback about the 2009 QA Standards and the process of developing their protocol.

Step 5: Finalizing and disseminating the 2009 QA Standards

After the pilot, the document was revised on the basis of feedback from the health departments and lessons learned from the pilot. The document was also reviewed for clearance by the Division of HIV/AIDS Prevention and revised accordingly. The final step in the PADQUA project was the dissemination of the 2009 QA Standards, in both hard-copy and electronic formats, to health departments and CTR service providers.

Top of Page

Resources will be made available in the future

 

National HIV/AIDS Strategy Updates

AIDS.gov: Access to U.S. Government HIV / AIDS information

USA.gov: The U.S. Government's Official Web PortalDepartment of Health and Human Services
Centers for Disease Control and Prevention   1600 Clifton Road Atlanta, GA 30329-4027, USA
800-CDC-INFO (800-232-4636) TTY: (888) 232-6348 - Contact CDC–INFO
A-Z Index
  1. A
  2. B
  3. C
  4. D
  5. E
  6. F
  7. G
  8. H
  9. I
  10. J
  11. K
  12. L
  13. M
  14. N
  15. O
  16. P
  17. Q
  18. R
  19. S
  20. T
  21. U
  22. V
  23. W
  24. X
  25. Y
  26. Z
  27. #