A. Planning, Prioritizing, and Monitoring

The Planning Panel was briefed on the processes by which DHAP plans programs and activities, including the analysis conducted to determine the cost-benefit of the implementation of various activities. The panel also reviewed information on budget allocations, the decision-making process within the Division, and the National Program Monitoring and Evaluation process currently underway. The panel provided extensive input on the planning role of DHAP, not only for its own activities within CDC, but also in the future as the administration prepares to develop a National HIV/AIDS Strategy (NHAS).

Overarching recommendations were identified by the Planning Panel that do not relate directly to any particular review question but are important to include. The panel felt strongly that CDC DHAP leadership on domestic HIV prevention is needed now and that as the lead public health, science-based prevention agency in the nation, CDC DHAP should be the nation’s voice on HIV prevention. The panel emphasized the expectation that CDC DHAP will play a leadership role in the development of a National HIV/AIDS Strategy (NHAS) as well as a significant monitoring role that the panel recommends include making public how we are doing as a nation in meeting the objectives and goals of the NHAS (e.g., a national report card, the HIV state of the union).

A second significant recommendation from the panel is that DHAP be forward-thinking and futureoriented in its planning, resource allocation, and evaluation of its efforts with the goal of preventing as many new HIV infections as possible as the primary guiding principle. The concept of “combination prevention” was endorsed along with the recognition that to be successful, the nation needs to scale up significantly the coverage, scope, and intensity of HIV prevention in the United States. For planning purposes, the panel recommends an approach which includes the development of several different funding scenarios—what is likely to be achieved with current resources, with increased resources, and with decreased resources. The cost of failure to act must be considered.

The importance of greater transparency on planning and prioritization processes was stressed. Prior strategic planning efforts have not been well understood, and it was unclear to some panelists how they came to be, what data informed them, and who did and did not have influence on their development. In addition to greater transparency on future planning efforts, the panel recommends that the expertise of CDC staff be used more effectively in future planning. It was also noted that the goals and objectives in prior plans have not always been specific, measurable, achievable, and/or realistic within the specified time frame and with actual resources in mind.

The panel questioned whether or not DHAP used its own data to plan and evaluate its efforts effectively at the Division level and across all of the various Branches. The panel recommends that the Division develop clear ways of taking stock of what is happening internally, externally, and relative to set objectives on an ongoing basis. DHAP should develop data-utilization plans and look at what is needed to make decisions at the Division level. The panel recommends one simple, integrated nimble (prepared to act quickly) system that is geared toward strategic decision-making and the dissemination of appropriate and timely information for actionable use internally and externally.

Greater efforts should be made to maximize learning via the Division’s monitoring and evaluation activities. DHAP should develop evaluation questions that are better tied to strategic decisions and initiatives. Questions should change over time as appropriate, reflect goals and objectives and, where possible, be prospective. Attention must be made to improve the quality of the indicators and the data collected to reflect those indicators. DHAP’s indicators should be adequately specific, capturing important pieces of information (cost, unintended consequences), or clearly linked to process and outcome objectives at the division and program level. Information should be used to determine success and improve decision-making and implementation of DHAP efforts.

The panel felt strongly that CDC DHAP needed to have a stronger voice in policy beyond its accepted processes of guidance and recommendation development. DHAP needs to become an astute user of the existing public policy mechanisms, including budget, legislation, regulations, and public affairs, and assist state, local, and directly funded grantees in translating policy data and scientific analysis into action. DHAP should provide the public with easy access to information, translated in a user-friendly way. Finally, the panel acknowledged the need for CDC to be both nimble and transparent in order to ensure stakeholder buy-in and support.

Topics to be addressed:

• Strategy development and planning
• Priority setting (including Resource Allocation Model)
• Results monitoring (esp. DHAP monitoring and evaluation plan)

1. To what extent are DHAP’s processes for planning, strategy development, and priority setting explicit and technically valid?

• The panel requested clarification on how the strategic plan translated into an operational plan. Not all the panel members were aware the HIV Strategic Plan had been extended to 2010.
• The panel also wanted clarification on the distinction between the Strategic Plan and the Professional Judgment Budget, the latter’s development, who will be served, and how progress will be measured.
• The panel requested specific information on DHAP’s policymaking role, including how processes work within CDC for engagement with HHS, the Office of Management and Budget (OMB), and Congress and what limitations and boundaries exist, with a particular focus on the budget process. The panel further inquired as to what authority, including statutory authority, DHAP would require to be better engaged in policymaking processes. The panel recommended DHAP focus on its own organizational functions as a basis for future plans, avoid duplication of effort, and provide rapid response to emerging issues. DHAP can also have a broader scope for planning and use input from within the Division.

2. To what extent do these processes make adequate and appropriate use of data?

• It was not entirely clear to the panel what data were used to inform planning and whether or that not all of the data that might inform planning were being brought to bear on planning The panel suggested that CDC take stock of its potential data sources and their potential value for planning and evaluation at the Divisional level.
• The panel noted the lack of sufficient evaluation data on capacity-building efforts, including evaluation data on the tailoring and adaptation of the evidence-based interventions.

3. How might DHAP best incorporate external input into its planning, strategy development, and priority setting?

• The panel recommended that stakeholder involvement occur at various stages in order to achieve buy-in into the planning outcomes. External nongovernmental stakeholders expect CDC to lead a planning effort that includes stakeholder input with the optimal goal of gaining external support for DHAP programs and priorities.
• On a wider plane, the panel emphasized DHAP’s role in fostering a better understanding of HIV prevalence and its impact on future infections (e.g., via changing HIV transmission rates which depend on both incidence and prevalence). The panel stated the importance of CDC making strong leadership statements that would garner community support. The panel indicated that the purpose of the plan must be clearly stated.
• The panel underscored the need to have members of the CDC/HRSA Advisory Committee on HIV and STD Prevention and Treatment better understand the structure and operations of DHAP in order to be better prepared to monitor and make recommendations regarding DHAP activities.

4. What are the strengths and weaknesses of these processes and what are your recommendations for improving them?

• The panel emphasized the need to anticipate future needs in HIV prevention with planning based on surveillance and epidemiology data, prevalence and transmission rates. Planning efforts can be organized according to prevalence, with high-, moderate-, and low prevalence jurisdictions, and with special consideration of populations and jurisdictions with estimated low prevalence of HIV (Asian/Pacific Islanders, Native Americans/Alaska Native, U.S. Virgin Islands and Pacific Territories). The panel recommended creating a working group to provide input on future challenges. This panel could be charged with creating a policy- and program-relevant “dashboard” of the HIV epidemic suitable for timely and comprehensive program planning and evaluation.
• The panel provided several recommendations regarding planning with a particular focus on funding allocations.
  • The panel recommended that several effectiveness analyses be used in planning including, the number of new infections that could be averted with a) current resources, b) incremental increases in resources, c) increases based on documented need, d) level funding, e) no change (or the consequences of failing to act). Planning can also look at the marginal utility of additional funding to maximize its impact.
  • The analyses can support funding in other agencies and for activities not covered elsewhere. Planning can be based on overall funding for HIV activities (i.e., treatment and research). Planning may result in goals that go beyond the current parameters, while other goals may be more achievable.

5. How appropriate and relevant are DHAP’s strategic priorities to the current epidemic with respect to populations and strategies?

• The panel recommended an internal review of CDC HIV prevention funding at the various organizational unit levels (CDC OD, Coordinating Center, Center, Division, Branch) and a review of historical information, and for DHAP to gain control over all available HIV prevention resources.
• The panel also emphasized the need for greater transparency in the planning process so external sources could learn where funds are being directed. In particular, the panel recommended a measure of the cost of delivering services to clients and to examine these efforts at the national level.
• Data on cost effectiveness can be used to report to Congress and OMB to provide information on the relative efficiency of CDC’s HIV prevention efforts relative to other investments of federal resources.
• The panel recommended that the economic reality be considered, including survival of CBOs and the cost of prevention that can be sustained over time.

6. How appropriate is the relative mix of prevention efforts directed toward persons living with HIV and those not yet infected?

• Programmatically, the panel stated that DHAP may want to focus on some key strategies for HIV prevention and evaluate them thoroughly. Examples of these would include increased efforts within prisons and other corrections facilities, mental health and substance abuse treatment providers, and supported housing.

7. How appropriate are the substance and scope of DHAP’s strategic priorities relative to the Division’s mission?

• The panel recommended developing the framework of “highly active HIV prevention” (HIHP) in order to articulate the need to scale up prevention efforts at the national, state, and local levels. A comprehensive framework would enable jurisdictions to better target and tailor their HIV prevention response to their local epidemics. Efforts can also be expended to increase understanding of HIV prevention, foster improved messaging and marketing, and provide more information on planning efforts.

8. How clear and focused are DHAP’s strategic priorities, and have they been articulated and communicated adequately?

• The panel recommended a higher level of policy leadership, beyond funding, to support strategies such as needle exchange programs (NEPs) and to ensure adequate support for existing and new initiatives. Prevention activities can also be supported by seeking synergies with other activities and other federal agencies, including testing expansion, screening, and linkage to care, and by treating HIV treatment as prevention.

9. How well do DHAP’s strategic priorities support 1) collaboration among HIV, STD, viral hepatitis, and tuberculosis programs and 2) integration of HIV, STD, viral hepatitis, and tuberculosis prevention services at the client level?

• A plan must include comprehensive program and policy alternatives that are prioritized and rely on synergy among the national, state and local levels; be adjusted as needed; and be based on the data collected.
• A model can be used to quantify the priority-setting process and the selection criteria, especially to determine the priority for activities for certain populations (transgendered) and to determine the impact of new strategies.
• The panel recommended that DHAP develop a formula for prioritizing activities based on the epidemic burden. The formula should include incidence and prevalence rates and use data from DHAP and other sources (e.g., foundations, national organizations).
• The panel requested that DHAP develop a more clearly articulated policy agenda along with an analysis of how policy changes might impact program effectiveness and, ultimately, the trajectory of the epidemic. At present, policy may include DHAP guidelines, reporting requirements, state and local laws, and policies that are outside of CDC’s purview. The panel emphasized the need for transparency, including at the Financial Management Office (FMO).
• Some barriers were cited by the panel, including delays in obtaining input from the chain of command, and the role of Coordinating Centers within CDC, and the Office of the Director and communications with Congress.

10. How well do DHAP’s strategic priorities support reduction of health disparities?

• The panel did not have an explicit conversation about this question, but the theme of health disparities was woven throughout many of the conversations. However, an underpinning of the panel’s discussion was the need to address HIV/AIDS disparities and ensure that DHAP planning and prioritization processes resulted in better targeting of resources to address disparities based on race, ethnicity, and sexual orientation.

11. What are the principal gaps in DHAP’s strategic priorities?

• The Panel emphasized the need for a National HIV Prevention Plan—likely to be part of the National HIV/AIDS Strategy—in which CDC can play a pivotal role for the federal government, providing a national public health vision, leading by example the rest of HHS and the Executive Branch, including the White House. The panel members indicated that a leadership model was required to include input from within DHAP and other governmental and non-governmental entities that have valuable input on the development of this National Plan. DHAP can provide leadership in this process and encourage participation from frontline organizations.
• The panel indicated that it is DHAP’s role to lead the HHS HIV prevention effort, collaborate with other federal agencies, both within HHS and with other departments. DHAP’s unique role is to prevent as many HIV infections as possible, considering all available opportunities, and report annually on the state of the epidemic so that future strategies will be based on the likely trajectory of the epidemic.

Results Monitoring

12. To what extent is DHAP’s national monitoring and evaluation plan explicit and technically valid?

• Monitoring efforts can begin with determining a standard set of questions to be addressed, including input from grantees, and one standardized system that can incorporate all data collected, ensure there is not an overwhelming amount of data, and consider unintended consequences. In developing this centralized system, the lessons learned of the Program Evaluation and Monitoring System can be used. The process to develop and revise the questions should be flexible to respond to the reality of government.

13. To what extent are the data sources included in the plan adequate and appropriate, and what other data sources should be included?

• The panel recommended the standardization of data collection efforts across all federally funded testing initiatives, uniformity of data reporting and coding requirements among jurisdictions and the validation of data. The social impact of the testing activities within the various populations can also be explored.
• Data collection efforts can go beyond CDC to include other HHS agencies that can report standardized demographic information on populations.

14. How adequate and appropriate are the outcome and impact measures described in the plan in terms of their ability to assess DHAP’s programs and their public health impact?

• The panel raised the issue of who would have ultimate responsibility for monitoring and accountability, what data should be collected, and what would be measured. The panel stressed the use of various types of data, including core surveillance and grantee evaluation data, and lessons learned and the need to adequately fund the data collection efforts and meta-evaluation.

15. What are the strengths and weaknesses of the plan and what are your recommendations for improving it?

The panel concurred with the following statements:

• “To achieve optimal public health impact, the appropriate combination of evidence-based HIV prevention strategies must achieve sufficient coverage, intensity, and duration.”²
• Combination HIV Prevention: “Effective HIV prevention involves the simultaneous use of diverse and integrated prevention strategies—programs that help individuals prevent transmission, broader-based initiatives that alter the norms and behaviors of social groups, and increased access to tools that reduce the biological likelihood of transmission.”
• Analogy to HIV Treatment: “The parallels between HIV prevention and treatment are striking. Like antiretroviral therapy, HIV prevention is life-long, and its impact must be continually monitored and the prescribed regimens revised as circumstances and needs change. Just as a single pill cannot eradicate HIV, one-shot prevention efforts will not achieve the magnitude or sustainability of behavior change required to alter the epidemic’s course…Like treatment, effective HIV prevention requires a combination of strategies.”

The panel made the following issue-specific recommendations:

• DHAP should continue conversations around needle exchange and preparation for the possible lifting of the ban on the use of federal funds.
• DHAP’s sero-sorting position statement is an opportunity to identify and respond to a evolving issue in the future course of the epidemic.

² Source: The Global HIV Prevention Working Group, Behavior Change and HIV Prevention: (Re)Considerations For The 21st Century, August 2008

Top of Page

B. Surveillance

The Surveillance Panel reviewed written materials and heard presentations from DHAP personnel on the various components of the surveillance system and explored the sufficiency of CDC’s efforts in this area. The panel also considered the extent to which the data gathered through core and other surveillance activities are used to inform the decision-making process internally and externally and the interaction of program and surveillance.

Topics addressed:

• HIV case surveillance
• Incidence surveillance
• Drug resistance surveillance
• Behavioral surveillance
• Clinical surveillance

The Surveillance Panel recognized that DHAP and the jurisdictions are at a turning point in the ability to monitor, characterize, and impact the epidemic: for the first time in the history of the epidemic there is a national confidential name-based HIV reporting system; DHAP, in conjunction with the jurisdictions, has developed a rigorous Technical Guidance document that details required activities and best-practices; universal process and outcome performance standards have been published and are beginning to be monitored; training on the Technical Guidance and performance standards has occurred and is ongoing; and by the end of 2009 all jurisdictions will have implemented a new data management system that will allow jurisdictions and CDC to analyze surveillance data more sensitively for quality assurance and monitoring purposes as well as to support the increasing volume of electronic reporting technologies. In 2008, two other significant developments related to HIV surveillance were realized by CDC: a revised national HIV incidence estimate was derived from the young HIV incidence surveillance system and the CDC released guidance to promote the integration of HIV surveillance with HIV partner services activities. The current system, which includes core surveillance as well as HIV incidence, resistance, behavioral and clinical surveillance components, also has many challenges which are discussed in the following sections.

1. To what extent are the surveillance methods and resulting data of high scientific quality?

Core Surveillance

The panel members concurred that the Core Surveillance methods are robust and the system is becoming stronger with nationwide implementation of confidential name-based HIV reporting, documented technical guidance, and standard evaluation criteria. The quality of data from core surveillance systems, however, vary from jurisdiction to jurisdiction depending on the extent to which sites have implemented the required activities such as birth and death ascertainment activities, Routine Interstate Duplicate Review (RIDR), laboratory reporting (including CD4 counts and viral loads), document-based surveillance, and name-based reporting. As more people are living with HIV and living longer, and as the surveillance system continues to evolve in ways that can facilitate monitoring and identification of prevention opportunities along the spectrum of disease, the demands on the fundamental “core” system are expected to continue to increase. The panelists identified the following issues that impact the system’s ability to produce data of high scientific quality:

• There is a need to strengthen the core surveillance infrastructure, potentially as a priority over other ancillary surveillance activities, so that sites are able to conduct the required activities and meet the performance standards.
• The increasing volume of laboratory reports and other electronic reports is a significant challenge to the system. Receiving and managing electronic data require sites to adopt major data processing capabilities. More robust standard data matching and processing tools and expertise are needed in many areas.
• Transition to the new data management system (eHARS) is not yet complete at the jurisdiction level or within CDC’s data management and analysis processes. In addition, important corrections are still needed in the data system, such as for HIV-2 cases to be recorded and the calculation of variables for perinatal exposure cases.
• Surveillance programs and data management systems need to be prepared to incorporate the evolving testing technologies in data collection, management, and dissemination activities. Such advances may also necessitate timely changes to case definitions.
• Participants noted the need for an updated case residency consultation, reconsideration of the hierarchical approach to classification of risk, and guidance related to implementing surveillance of acute infections.
• Panelists observed that gaps in reporting from federal programs, such as Veteran’s Health Administration facilities, affect many areas. Intervention is needed by CDC to ensure that programs administered by federal counterparts are cooperative with national disease reporting standards.
• Capacity for data analysis and dissemination has challenges locally and within CDC (see Question 3: Constituent Needs and Question 5: Resources and Capacity).

HIV Incidence Surveillance

HIV Incidence Surveillance is still a young surveillance system and its overall functioning and quality has yet to be assessed. The system is complicated to implement, and panelist expressed some reservations about the scientific quality of the incidence estimation methodology given that not all labs participate, the inaccuracies inherent in Testing Treatment History (TTH) information, and whether the fundamental assumptions of the model will continue to be appropriate over time with changes in testing patterns and technologies. In addition, the extent to which the incidence estimates are representative of the nation will be better understood once there is a complete national HIV case surveillance dataset. Other specific observations and recommendations:

• Panelists were cautious about the use of data from the current system as the sole marker for national prevention goals. Locally, estimates do not appear sufficiently stable at this time to inform prevention activities in most jurisdictions.
• Panelists recognized the importance of measuring incident infections as a part of understanding the epidemic. Towards that end, some recommended that CDC work with FDA and manufacturers for faster approval of 4th generation HIV diagnostic tests and new tests for incident infections that could be conducted directly on specimens by the laboratory/testing entity avoiding the need to secure remnant specimens.
• In order to evaluate the strength of data used in the current estimation model, CDC is currently funding a special project to assess the accuracy of TTH data. Panelists suggested that CDC consider whether data collected through core surveillance nationwide might be able to assist with this validation, such as comparing self-reported testing information from Counseling/Testing programs to surveillance.
• Some panelists also recommended that incidence data, in particular infections identified in the acute phase, represent an opportunity to intervene and interrupt transmission. As CDC moves forward with the development of approaches to monitor incident infections, panelists were supportive of exploring surveillance/prevention collaborations to identify clusters and attempt to stop transmission. The group supported the approach that CDC is advancing through the “STOP” Project.

Drug Resistance Surveillance

Panelists recognized the potential value in monitoring medication-resistant strains of HIV nationwide. The current system, however, is in the early stages of development and is transitioning from an approach that requires acquisition of remnant specimens to a more standard laboratory reporting surveillance method. Panelists noted that this activity is logistically and scientifically complicated, and additional guidance is needed to ensure that the resulting data are appropriately managed and interpreted. The system has not yet been evaluated.

• Guidance is needed for jurisdictions to assess if they have sufficient coverage of resistance testing in clinical practice to discontinue the cumbersome process of obtaining remnant specimens that meet the rigorous handling requirements to be tested.
• Establishing reporting of results from laboratories is challenging, even for sites with permissive regulations and advanced technical capabilities. The capacity of laboratories to provide the data electronically is variable.
• Panelists identified many outstanding issues related to analysis and use of data, including the appropriate timeframe for analyzing results (within three months of diagnosis versus one year); the appropriate use of data for identifying clusters and informing prevention or clinical activities; and submission of genotype results to public-access scientific databanks such as GenBank. Input from sites and field experts are needed to inform additional guidance and technical assistance to ensure appropriate and meaningful use of data from this system.

Behavioral Surveillance

The panelists concluded that the National HIV Behavioral Surveillance (NHBS) is a strong project with good collaboration and analysis. The panelists had the following considerations related to ensuring the validity and utility of NHBS data:

• Eligibility for funding is based on the 2002 AIDS case data; panelists recommended considering if the project is appropriately representative now that universal HIV reporting has been implemented and more recent data are available.
• The strengths and weaknesses of respondent-driven sampling (RDS) should be assessed.
• Panelists recommended that CDC consider more flexible models for the cycles or location activity (e.g., consider single survey in some areas that have relatively homogeneous populations; consider expanding activities to lower morbidity areas to ensure representation and to gather information on non-urban populations, perhaps using a regional model to achieve sufficient sample size).
• Reviewers encourage CDC to explore opportunities to incorporate behavioral surveillance activities (not necessarily using the NHBS model) into TB, hepatitis, and STD program activities as a means of adding to the portfolio of monitoring risk behaviors. Some panelists also felt that there was an opportunity to conduct behavioral surveillance focused on acute HIV infection (AHI). If nucleic acid amplification testing (NAAT) and surveillance for clinical syndromes consistent with AHI were to become more widespread, there may be an opportunity to interview a sizable number of persons with AHI to better understand their risk behaviors and “thinking” (knowledge, beliefs and attitudes) at the approximate time they were infected.

Clinical Surveillance

The Medical Monitoring Project (MMP) is important as the only present-day study that attempts to assess a representative sample of HIV-infected persons in care to understand their care utilization, clinical characteristics, and risk behaviors. The panel agreed that 2009 is a critical year for MMP, which still has not been fully implemented after five years of funding. The panel recommended that CDC evaluate the program to identify roadblocks (e.g. OMB delays, stipends for and randomization of providers, capacity and resources to implement, real time sampling); determine if these roadblocks are adequately addressed; answer questions; and establish specific performance indicators. After completion of 2009 data collection, panelists recommended that the feasibility of the methodology be reviewed.

In addition, some panel members felt that some outcomes being monitored by the MMP (e.g., continuity of care) might be more easily monitored through measuring regularity of getting viral load tests and CD4 counts as part of core surveillance. Where the MMP is in place, comparison of outcomes using surveillance and the MMP should be done.

2. To what extent are the objectives of each DHAP surveillance system adequately described in, clearly linked to and consistent with the Division’s mission, goals, and priorities?

The materials provided to the reviewers did not specifically articulate how the objectives of each of the presented surveillance activities are linked to specific Division goals and priorities. However, the panelists did consider the stated the HIV Incidence and Case Surveillance Branch (HICSB) and Behavioral and Clinical Surveillance Branch (BCSB) goals of implementing surveillance and research activities that guide public health action at federal, state, and local levels. Overall, jurisdictions likely do not understand how their activities are linked to specific DHAP goals and priorities. With surveillance data essential to monitoring many goals established through various national strategic planning processes, such as CDC’s HIV Prevention Strategic Plan and the expected development of a National AIDS Strategy, it would be important for DHAP to ensure that surveillance jurisdictions understand how the DHAP goals tie into larger national goals and how the local level activities support those goals. In addition, the panelists had the following observations and recommendations:

• As noted in the summary, the case surveillance system is getting stronger with the realization of universal HIV reporting in 2008 and the implementation of common standards for the completeness, timeliness, and accuracy of jurisdiction-level surveillance systems. Given the strengthening foundation and potential capabilities of surveillance, CDC should re-examine the strengths of and demands on the surveillance system and reconsider what surveillance should be measuring and doing as well as assess the resources that are needed to support those activities as a partner in HIV prevention.
• The panelists were vigorously supportive of the inclusion of milestones in CDC’s Extended Plan that are specific to strengthening the capacity nationwide to monitor the epidemic. A robust surveillance system is essential to supporting sound programmatic decisions nationally and locally.
• A stated common goal of CDC’s surveillance branches is to conduct activities that will guide public health action at the federal, state, and local levels. Over the course of the review, the panel discussed several challenges for surveillance to action:
  • Cooperative agreements with the jurisdictions generally support data collection and not data analysis. The Epidemiologic Capacity Building Technical Assistance funding is an example of dedicated analysis effort and information sharing available to some areas, supported by both Prevention and Surveillance resources, that could be further developed to help ensure that surveillance data are analyzed and effectively incorporated into local planning and evaluation processes.
  • Some surveillance activities are not currently producing data that are able to be used at the local level (see details in state/city/jurisdiction constituency issues identified in Question 3).
  • Stronger tools and approaches are needed to help jurisdictions ensure that surveillance data and information from other studies and evaluation activities are utilized in a way that is most likely to impact the epidemic.
• The newly released HIV Partner Services guidance recognizes the important prevention opportunities that can be realized when surveillance programs collaborate with prevention programs. Panel members supported CDC’s development of this guidance and noted the ongoing need for CDC to incorporate rigorous security and confidentiality requirements in cooperative agreements across the agency’s divisions.

3. To what extent do the surveillance systems and products adequately meet the needs of CDC’s constituencies?

The panel at different points in the review discussed several constituencies of DHAP surveillance systems and products. In particular, the participants considered how surveillance activities impact programs and processes within CDC, jurisdiction-level prevention and surveillance programs, HIV Partner Services programs, and HRSA Ryan White CARE Act programs. An overarching recommendation from the panelists, also articulated in the response to Question 2, was for CDC to look outward to reconsider the scope of surveillance activities and assess the increasing demands on the surveillance system.

CDC Prevention and Epidemiology programs as constituents

• The extent to which CDC surveillance personnel are incorporated into the CDC prevention and evaluation planning processes was not clear to the reviewers. Given there are particular strengths and weaknesses of the surveillance system activities and the resulting data, surveillance expertise would be valuable both to interpret surveillance results and to guide decisions being made within CDC for developing prevention strategies, prioritizing interventions, developing campaigns, and establishing evaluation criteria or indicators.
• The panel participants identified some instances of gaps and other instances of duplication across CDC divisions and DHAP’s branch activities where data that are already being collected in one activity might be useful for another and might help support DHAP’s evaluation of progress towards attaining goals. In particular, there may be a benefit in evaluating the data collected through Core Surveillance, Medical Monitoring Project (MMP), National HIV Behavioral Surveillance (NHBS), Enhanced Perinatal Surveillance (EPS), Fetal and Infant Mortality Review (FIMR), and STD, TB, and other infectious disease surveillance activities to identify any unnecessary redundancy as well as additional available data that can help measure progress towards the Division’s goals overall.

States/cities/jurisdictions as constituents

• Although most CDC surveillance activities are intended to provide locally useful data for monitoring the epidemic as well as guiding and evaluating prevention efforts, the benefit to local sites is often not fully realized. For example:
  • Core Surveillance appears to be funded primarily for data collection and management activities, but not for local data analysis. DHAP should provide adequate funding and/or technical assistance to ensure that data are analyzed to inform local program needs.
  • Some jurisdictions funded for HIV Incidence Surveillance remain unclear about how best to use the data to inform program or if that system will produce sufficiently stable estimates to be meaningful locally.
  • Currently many jurisdictions lack the capacity to analyze resistance sequence data. DHAP should provide adequate funding and/or technical assistance to ensure that data are analyzed to inform local program needs as there is no consensus from CDC on analysis criteria. (CDC personnel noted during the discussions that guidance and technical assistance was anticipated later this year).
  • Some areas relay that behavioral surveillance data are inadequate for genuine evaluation of local prevention efforts.
  • The respondent sample size for many MMP sites is small and could limit locally useful data unless high patient participation rates are achieved by allowing and/or developing a more successful sampling methodology, such as Real Time Sampling. Data has thus far not been returned in a timely manner for analysis.
  • Lower morbidity areas generally do not qualify for “supplemental” surveillance activities, resulting in a less than complete characterization of the epidemic and of the behaviors of persons in those areas and nationally.
  • Jurisdictions are eager to have additional information to inform prevention. CDC’s recent back-calculation of case surveillance data provided rich national data complementing the new incidence estimation methodology and allowed for the estimation of transmission and unknown status. Panelists encouraged that technical assistance be available if such approaches could be applied locally.
• In order to assist surveillance programs with the provision of data to local prevention and care programs, CDC developed technical guidance and analysis programs for jurisdictions to produce a local Epidemiologic Profile document. While a very rich resource, many jurisdictions have found the recommended document cumbersome both to produce and for local planning bodies to use. The panelists recommended that CDC partner with the jurisdictions’ surveillance and prevention programs to assess how the Profile is (or is not) used and reconsider best practices for ensuring the appropriate incorporation of surveillance and other data in local planning processes. Smaller, more frequently produced Profiles may be more responsive to planning group needs. Panelists also noted that there is little funding to specifically support the development of the profiles.
• The use of GIS mapping to better correlate HIV infection with other socio-economic and geographic characteristics may provide information useful for planning; a recommendation was made for CDC to consider supporting the technical capacity for GIS mapping and small-area analysis of HIV by socio-economic status (e.g., percentage of persons living below poverty in the “neighborhood of residence”).
• The panel acknowledged the Division’s production of several summary products that accompanied the release of the incidence estimate in 2008 that areas found particularly helpful and encouraged CDC to consider developing similar materials to accompany other routine and non-routine surveillance reports. Often local jurisdiction staff, even those fairly knowledgeable, find that some CDC surveillance reports currently are not at all easy to understand.

Ryan White CARE Act programs as constituents (federal and local programs)

• HRSA programs are major “consumers” of surveillance data—national funding allocations presume an equitable and high-performing surveillance system across the country. Also, local care programs rely on surveillance programs for accurate, clear, simple summary data products in order to guide allocations, planning, and grant applications. Appropriate determination of unmet need for HIV primary medical care requires timely and complete local surveillance systems with comprehensive laboratory reporting and strong analysis capacity. Overall, panelists reiterated the importance of a strong collaborative relationship between HRSA and CDC, as surveillance capacity has a significant impact on HRSA programs both locally and nationally. Reviewers also noted the following:
  • With confidential name-based reporting now implemented in all jurisdictions, panelists urge CDC to reconsider its recommendation to HRSA on how best to utilize HIV morbidity data for HRSA funding, specifically noting that CDC evaluate the feasibility of switching from residence at diagnosis to most current residence for assigning state “ownership” of cases.
  • Grant applications to HRSA for the various Ryan White program “parts” require extensive local analysis of surveillance data. Reviewers requested that CDC work with HRSA to specify and consolidate the type and source of information needed for federal funding applications and progress reports.
  • Panelists also identified opportunities for collaboration and consolidation around the data collected to monitor and evaluate Ryan White programs. With HRSA’s development of a code-based client-level data system (a type of system just abandoned by CDC), Ryan White programs are required to collect and record information that may already have been collected and recorded in the de-duplicated surveillance system. While not advocating for a merging of the two systems, panelists urged the agencies to consider the efficiencies that could be gained with greater coordination around data needs.

Partner Services programs as constituents

• The current surveillance system functions as a look-back system that is not particularly timely for intervention. In the panel’s discussions related to HIV incidence surveillance, participants noted the potential increasing importance of a more timely surveillance system with the opportunities to interrupt transmission among reported acute infections, specifically noting that CDC evaluate the feasibility of switching from residence at diagnosis to most current residence for assigning state “ownership” of cases. As technologies to detect acute and recent infections evolve, the importance of surveillance working closely with Partner Services programs could be critical. Panelists encouraged that

CDC consider the potential role of surveillance as a part of the prevention portfolio and the resources that would be needed for the system to serve in this capacity.

4. To what extent are the surveillance systems adequately evaluated and the results effectively disseminated? To what extent are the data used by prevention activities for program development and evaluation purposes?

Evaluation of surveillance systems and dissemination of results

• For Core Surveillance, HICSB has made a significant investment in preparing jurisdictions for the implementation of rigorous, ongoing evaluation activities through the development of process and outcome standards, publication of standards in the Technical Guidance and articles, ongoing training, and incorporation of key standards in grant applications and progress reports. In addition, by the end of 2009, HICSB will have completed the deployment of the data management system, programs, and technical guidance needed to implement national evaluation activities. The plan appears rigorous. Participants noted that HIV surveillance seems to be one of the most comprehensive surveillance systems existing for any condition. While CDC has established performance standards, a thorough evaluation of the system has not yet been done nationally or locally in a uniform way. Preliminary results of the informal reporting by the sites of required process and outcome standards (from the 2007 APR and 2008 APR) have, to date, not been disseminated. As a result, the panelists could not definitively speak to the quality of the data. Additional observations:
  • Panelists asserted that some components of the evaluation plan are very complicated and may need to be reconsidered in light of what is practical in a non-research setting and what is most important or meaningful for improving the system given the available resources.
  • With laboratory reporting becoming such a major component of the surveillance system, there might be a benefit to developing process standards that prompt jurisdictions to monitor and evaluate the laboratory-reporting component specifically. Although sites ultimately need to meet the outcome standards and CDC does not necessarily want to dictate how sites need to achieve those standards, additional guidance and recommended standards would appear to be helpful.
  • Feedback to jurisdictions about how they compare to other programs with the implementation and achievement of standards would be useful.
• For HIV Incidence Surveillance, performance standards are defined, but some appear unattainable in the current model of incidence surveillance that depends upon the availability of remnant specimens. The panelists also noted that incidence data are part of several national prevention monitoring and evaluation goals, but question if the system is or will be sufficiently robust to provide those measurements alone.
• No formal evaluation has been conducted to date for the Variant, Atypical, and Resistant HIV Surveillance (VARHS) program. This is a developing system that is complicated and achievement of the defined outcome standard will be difficult, especially as long as sites rely on testing of remnant specimens.
• The panel recommended that CDC evaluate National HIV Behavioral Surveillance (NHBS) once two complete cycles have been completed to assess the strengths and weaknesses of respondent-driven sampling (RDS) and consider a more flexible model for the cycles or location activity (e.g., consider single survey in some areas). For example, in areas whose epidemic is almost entirely MSM, it may not make sense to conduct the IDU or HET cycles.
• The panelists strongly recommended the CDC evaluate the Medical Monitoring Project (MMP) in early 2010 to assess if the protocol can be implemented as designed and deliver the intended results. And if not, that its methods be changed to be more easily meet the goals of collecting representative clinical surveillance data. If the goals are assessed to be unachievable, then serious consideration should be given to either eliminate MMP or redesign it with a more workable methodology that produces useable data.

Application of surveillance data for prevention program development and evaluation

• The panel discussed the need for tools to ensure that surveillance data are utilized appropriately to align resources and interventions for the areas of greatest burden or areas, populations, or groups that are disproportionately affected, within CDC and at the jurisdiction-level. The materials provided for the review cited a resource allocation model under development to assist with optimal allocation of funds. CDC personnel clarified that such a model is not currently available for jurisdiction-level planning processes.

5. To what extent does DHAP have adequate capacity and sufficient resources devoted to surveillance, consistent with the Division’s mission, goals, and priorities?

• Overall the panel members concurred that DHAP does not currently have adequate capacity or resources devoted to core surveillance activities, which not only have dramatically increased in volume and complexity, but also serve as the foundation upon which prevention and care programs are based. As described in the responses to the other sections, the reviewers urge CDC to consider the demands on and the deliverables of the surveillance system activities, present and future.
• The reviewers recognized the value of each of the activities in the current surveillance portfolio but asserted that strengthening the national HIV reporting system (Core Surveillance) and ensuring rigorous national and local analysis capacity should be priority activities when resources are limited.
• The method of determining allocations across surveillance programs was not well understood by the reviewers. The panel recommends assessing funding levels across jurisdictions to ensure appropriate parity.
• CDC staffing and capacity did not seem at all sufficient to ensure meaningful support to or monitoring of sites. Some CDC epidemiologists providing support are new to the Branch and do not have enough familiarity with the complexities and intricacies of HIV surveillance activities. Provision of in-depth technical assistance and trainings—such as the beneficial trainings and meetings related to incidence estimation—is putting a strain on CDC program personnel resources. Lower morbidity areas in particular may require additional technical assistance for data analysis that CDC staff may be unable to provide.
• The needs for technical assistance are great and continue to increase, whether it be ensuring the effective use of surveillance data for programs; implementing HIV Partner Services collaborations; transitioning data management systems to more complicated SQL platforms; implementing and managing electronic reporting from laboratories, facilities, and other reporting systems; implementing, managing, and analyzing genotype/phenotype data in particular; changing the surveillance system with advances in testing technologies and interventions; or ensuring nation-wide achievement of performance standards.
• Comprehensive laboratory reporting in all jurisdictions demands technical expertise for establishing and maintaining rigorous local systems, but the benefits are significant locally and nationally. Such reporting offers complete and timely data for: monitoring trends in newly diagnosed HIV infection and prevalence of HIV infections; entry, continuity, and retention in care; and timely referral of cases for partner services and linkage to care and prevention services.
• Information Technology resources within CDC do not seem sufficient as evidenced by long-standing corrections needed for eHARS (e.g. perinatal, HIV-2, others). Greater coordination of IT solutions is needed across HIV surveillance components, prevention components, and perhaps even the prevention and surveillance of other diseases, such as STD, hepatitis, and TB.

6. To what extent is the mix and balance of surveillance activities relevant to the current epidemic?

While the panelists did not specifically address the question of the “mix and balance” of current surveillance activities, over the course of the review various recommendations emerged that speak to this issue:

• CDC’s efforts to support rigorous, complete, core HIV surveillance systems at the jurisdiction level is the fundamental activity required to ensuring appropriately balanced and representative surveillance activities.
• Over the past several years, CDC has moved towards incorporating incidence and resistance surveillance activities into the “core” surveillance processes and systems. With the anticipated advances in testing technologies and clinical practices, incorporation of these components into “core” activities seems reasonable but should be evaluated as the incidence and resistance systems mature.
• Panelists recognized the need for and value of behavioral and clinical monitoring components as a part of national HIV surveillance efforts, but asserted that these activities should not supplant core surveillance infrastructure. It may be that the core surveillance system, which now includes longitudinal clinical data, may be able to provide some of the information needed for clinical surveillance. Behavioral data may also be able to be appropriately monitored through more effective coordination with other prevention and disease surveillance programs.

7. What are your recommendations for changes (e.g., enhancements, additions, activities that can be de-emphasized), future directions, and priorities for the program?

Many recommendations for changes, enhancements, de-emphasis, future directions and priorities are addressed in the preceding questions of the Surveillance Panel Summary as well as in the brief panel summary in section IIB of the report. The recommendations not already addressed primarily relate to the need for greater efficiencies and collaboration within DHAP and across CDC programs:

• Closer integration and collaboration between the prevention, research, evaluation, and surveillance programs is needed to eliminate unnecessary duplication of efforts and to maximize the use of available data to advance prevention efforts.
• Greater coordination between DHAP prevention and surveillance programs could help to ensure jurisdiction-level collaboration with key shared activities and performance measures.
• Across CDC’s disease surveillance programs, crosscutting issues such as poor reporting from the Veteran’s Health Administration facilities might be more effectively addressed as a coordinated effort.
• The new HIV Partner Services guidelines represent a critical milestone in recognizing the valuable prevention opportunities that can occur when surveillance supports public health action. Many barriers to implementation remain, however. While technical challenges exist, the greater challenge will likely prove to be in the navigation of organizational “cultures” and approaches. The extent to which CDC actively holds all programs and grantees accountable to similar security and confidentiality standards will directly impact how quickly programs move towards collaboration and integration of shared functions.

Top of Page

C. Biomedical Interventions, Diagnostics, Laboratory, and Health Service Research

Topics to be addressed:

• Biomedical interventions
• Diagnostics and testing
• Laboratory research
• Health services research (including cost-effectiveness)
• Research-to-program translation and dissemination

The Biomedical Panel had an opportunity to review the current status of research conducted on biomedical interventions, such as the development of new assays, surveillance of acute HIV infection, and the use of various research methodologies. The Panel was presented information on current research conducted by the various branches, both internally and externally, and reviewed the data gathered from research and was able to question how the priorities for research are determined within DHAP and possible synergies with other CDC components.

1. To what extent are the biomedical, laboratory, and health services research projects and resulting data of high scientific quality?

• The panel was clear that they believed the projects and resulting data were of the highest scientific quality. They affirmed that the investigators were also of the highest quality.

2. To what extent are the objectives of each biomedical, laboratory, and health services research project adequately described and clearly linked to and consistent with the Division mission, goals, and priorities?

• The panel noted DHAP’s unique role was in translational research, helping to disseminate the findings of clinical trials into public health practice.
• The panel discussed that the process for setting priorities, and the criteria used, including level of effort and research required was unclear. This included the following specific domains: o Pre-Exposure Prophylaxis (PrEP), microbicides, vaccine research
  • Diagnostics o Activities performed in consultation with other DHAP branches and external partners
  • Resistance testing (done in collaboration with industry)
• The panel questioned the process by which Branches determine what activities they carry out and who makes the final decision. Whereas priorities are determined at the Division level, branch chiefs can determine if the activities will be carried out and these may or may not match the Division level priorities.
• The panel questioned whether some legacy programs could be reprogrammed to other more effective initiatives.
• The panel discussed whether DHAP’s goals needed to be expanded so that CDC can be more agile and flexible in responding to the epidemic. They discussed the role of the CDC Washington, D.C. Office.
• There was discussion between allowing the CDC to conduct investigator-driven research versus selecting research based on cost effectiveness.
• The panel discussed CDC’s role in the development of diagnostics and research on incident and prevalent infections. Though this research has been historical and informative, it is not always unique to CDC and is occasionally investigator driven.
• The panel discussed CDC’s role in finding diagnostics to determine incidence, which will be helpful in determining program effectiveness.
• The panel felt that early detection is secondary to primary prevention, but clearly acknowledged that incidence data are important to target the work to the need.
• The panel discussed both the challenges and occasional necessity of combining behavioral and biomedical intervention development. They recognized that behavioral and biomedical research appears to occur in separate tracts and that little effectiveness research exists that studies the impact of combining behavioral with biomedical interventions. They observed that CDC would be ideally poised to conduct such research.
• The panel recommended the development of more innovative study designs, and better integration of behavioral and biomedical interventions as appropriate.

3. To what extent do the biomedical, laboratory, and health services research activities adequately meet the needs of CDC’s constituencies?

• As discussed above, the panel observed a poor correlation between the prioritization of research activities and broader prioritization processes within DHAP.
• The panel discussed the significant cultural and morbidity differences between white and black MSM and questioned whether or not adequate research had been conducted that took into account these differences.
• The panel questioned whether or not research has adequately addressed women who are infected while pregnant or how pregnant women who are infected became pregnant.
• Again, in order to better align research with DHAP priorities and goals including prioritized populations, the panel recommended establishing both an internal and an external scientific review panel (or process for periodic review) to assess existing and future projects so that only programs that are aligned with the mission and goals of DHAP are funded and implemented.

4. To what extent are the biomedical, laboratory, and health services research products adequately evaluated and the results effectively disseminated?

• The panel questioned how data is validated and how research models can be applied to the real world.
• The panel questioned how data are disseminated to decision makers. Though this is commonly done via published papers or Morbidity and Mortality Weekly Reports (MMWRs), the panel recognized that as a public health agency, this was not always the most rapid of mechanisms by which data could be shared. The recent HIV incidence figures were discussed as an example.
• The panel noted that CDC could do more to strengthen the communication of CDCapproved recommendations and guidelines.
• The panel questioned how the implementation of CDC guidelines is evaluated once they are released to the public. The panel noted that releasing guidelines is not enough to effectively disseminate new information. The CDC needs to also evaluate whether their recommendations are being carried out, and if they aren’t, what societal or legal factors need to be addressed or changed first to ease the implementation of the guidelines. In addition, the panel felt that guidelines need to be evaluated to determine whether they are realistic given the current health system infrastructure.
• The panel recommended that DHAP review its process for sharing data and guidelines to assure that it meets the needs of its constituents.

5. To what extent does DHAP have adequate capacity and sufficient resources devoted to biomedical, laboratory, and health services research consistent with the Division’s mission, goals and priorities?

• The panel discussed the impact of other entities, most importantly, the Global AIDS Program (GAP), on staffing levels. It was clear that many excellent DHAP staff had relocated to GAP, draining DHAP of personnel resources.
  • Specifically, the laboratory branch has lost several key staff to GAP and as a result have lost some capacity to carry out branch functions.
• The panel discussed the utility of cross-branch workgroups. The HIV Testing in Medical Settings workgroup was noted as effective in coordinating efforts across multiple branches.
• The panel questioned the prioritization of some biomedical interventions including PrEP, microbicides, circumcision, and vaccines. The panel recognized the CDC’s scientific contribution to these fields, but questioned whether there might be opportunity for greater collaboration with other partners including GAP, industry, and the National Institutes of Health (NIH).
• The panel discussed the importance of health services and cost-effectiveness research in the CDC scientific portfolio and the relative lack of prioritization these areas seem to be given. They discussed the few staff devoted to this type of research.
• The panel recommended that the DHAP Laboratory Branch and the GAP laboratory engage in more extensive collaborative activities in order to increase efficiencies and avoid duplication.
• The panel suggested DHAP explore expanding their health services and cost-effectiveness research portfolios by bringing in additional scientists and experts in these fields.

6. Are the Laboratory Branch’s diagnostics reference functions appropriate in scope, consistent with DHAP’s mission and goals, and adequately staffed?

• The panel noted the importance of the Laboratory Branch’s role in developing diagnostic assays for monitoring acute and recent HIV infection and confirming HIV infection.
• The panel discussed the need to develop an assay to detect non-clade B infections, which have been surging lately.
• The panel indicated that DHAP must be flexible enough to address changing diagnostic demands and that the research budget for biomedical interventions is not sufficient.
• The panel recommended that the Laboratory Branch continue to play a role in developing diagnostic assays and that this should be done in discussion with other federal agencies and industry that may be doing the same.
• The panel recommended that the Branch continue to serve as a reference laboratory.
• The panel recommended that DHAP engage external partners in an ongoing basis to ensure that they remain vigilant to the trajectory of diagnostic research.

7. Do DHAP’s biomedical, laboratory, and health services research activities reflect the appropriate mix of basic and applied research, and to what extent is the mix and balance of activities relevant to the current epidemic?

• As discussed previously, the panel observed an occasional disconnect between the research portfolio and DHAPs broader priorities. In addition, it was not clear how these broader priorities helped to inform the research agenda.

8. To what extent does the biomedical, laboratory, and health services research portfolio adequately advance the current state of science?

• The panel was clear that they believed the projects and resulting data was of the highest scientific quality. They affirmed that the investigators were also of the highest quality.
• The panel noted DHAP’s unique role was in translational research, helping to disseminate the findings of clinical trials into public health practice.
• The panel discussed the important role of the Laboratory Branch in conducting animal model research because it is a niche no one else can currently fill.
• The panel emphasized the importance of matching the scientific agenda with the current state of the epidemic.

9. What are your recommendations for changes (e.g., enhancements, additions, activities that can be de-emphasized), future directions, and priorities for the biomedical, laboratory, and health services research programs?

• The panel clearly recommended a review of the scientific portfolio as it relates to the priorities and goals of DHAP and the current state of the HIV epidemic as reflected in surveillance, programmatic, health services, and cost-effectiveness research.
  • The panel suggested that this review be recurrent, or a process, to allow for reevaluation of legacy projects and ongoing research.
  • The panel suggested that the review take into account research being performed by external partners including GAP, industry, and NIH.
• The panel reiterated its recommendation that DHAP collaborate more with GAP, industry, and with NIH to find synergies.

Top of Page

D. Behavioral, Social, And Structural Interventions Research

The Behavioral Panel was able to review information on the current status of behavioral and structural intervention research, the different types of research the CDC carries out, and the interaction of expressed need for new and innovative behavioral interventions with the programs being implemented at the national, state, and local levels. The panel also reviewed the scientific quality and appropriateness of research being conducted and the process for determining the subject and scope of research being conducted.

Topics to be addressed:

• Behavioral, social, and structural interventions
• Communications and social marketing research
• Operational research (including cost-effectiveness)
• Linkage to care, retention in care, ART adherence
• Research-to-program translation and dissemination

1. To what extent is the behavioral research portfolio of high scientific quality?

• The panel questioned the definition of science and the purpose of DHAP’s activities, which is to reduce incidence and prevalence not improve science. Science should have an ultimate programmatic outcome in the areas of capacity, technical assistance, science of training, and quality improvement and assurance. The panel agreed that DHAP should focus on applied research and real world issues, but questioned whether their current resources allowed for this type of paradigm shift.
• The panel recommended an expanded definition of behaviors to also include organizations, individuals, and social groups.
• The panel questioned whether DHAP’s focus on populations and demographics of race, gender, and sexual orientation translates effectively into intervention designs.

2. To what extent are the objectives of the behavioral research portfolio adequately described, clearly linked to, and consistent with the Division’s mission, goals, and priorities?

• The panel emphasized that research efforts should reflect science and philosophy and engage in innovative methodologies and give more emphasis to practice-based research.
• The panel came to consensus that while it remains important to acknowledge the disparate impact of HIV on specific populations, institutionalizing the response by population group has not proven to be effective and has had some unintended consequences. In particular, attention has been shifted from the social and individual practices conferring vulnerability and influences that increase the risk of HIV transmission to compartmentalizing individuals based on race/ethnicity; sex/gender, which generally is not appropriate or effective for intervention.
• The panel recommended that DHAP engage exclusively in Phase 1 and Phase 4 research activities, as those are the ones that can be more completely carried out within DHAP’s existing structure, and the current research model is not adequate. The Capacity Building Branch, given support in the development of study design, can carry out some research.
• The panel questioned if a logic model for developing social marketing campaigns exists. For example, when questioning CDC staff about why one program is considered and another is not, there was no clear answer provided.
• The panel noted that the theoretical models that drive program development, implementation and evaluation do not reflect a unified conceptualization of behavior change. For instance, in three projects presented to a group that focused on increasing HIV testing, six different models were used.
• The panel questioned CDC’s focus on medication adherence and whether this was an appropriate topic for CDC’s prevention research portfolio. It was pointed out that other federal agencies (National Institute of Mental Health, HRSA) may have asked CDC for assistance in dissemination of adherence interventions, and further information is needed to better understand the rationale for a focus on antiretroviral therapy (ART) adherence.

3. To what extent do the behavioral research findings/products meet the needs of CDC’s constituencies and inform programmatic efforts?

• The panel recommended that research take into account the needs at the community level and bridge research to practice with community-centered models. The panel emphasized that in communities there is no demographic distinction of individuals and questioned the meaningfulness of research conducted along strict demographic lines to CBOs.
• The panel recommended DHAP review the process for translating research findings into programs for implementation by CBOs. The analysis should include the cost of implementation and barriers commonly reported to implementation, including staff turnover at CBOs.
• The panel encouraged CDC to work with health departments and CBOs in operational research to ensure that evidence-based and data-supported linkage to care strategies are used in conjunction with HIV testing. The goal is to make sure that persons newly diagnosed with HIV and persons not connected to care are linked and retained in care.

4. To what extent are the prevention interventions and strategies within our research portfolio adequately evaluated (adequacy of research methods) and effectively disseminated?

• The panel discussed structural interventions and their role in the DHAP research agenda, which should come secondary to operational effectiveness research and is closer to DHAP’s role as an implementer of public health programs.

5. To what extent does DHAP have adequate capacity and sufficient resources devoted to behavioral research, consistent with the Division’s mission, goals, and priorities, and consistent with the mission of our other federal partners?

• The panel recommended that DHAP work with the NIH to address research gaps and avoid duplication of effort, including knowledge about treatment and adherence, and it should be DHAP’s role to serve as conduit to health departments and communities.
• The panel emphasized the need for collaboration among the research and program branches.

6. To what extent are the strategies, and the mixture and balance of these strategies, within the behavioral research portfolio appropriate and relevant to the current epidemic (with respect to various strategies or tactics, various populations or risk groups, and individual, social, structural, or other risk determinants)?

• The panel recommended that research projects be done in collaboration with CBOs funded to implement programs and that DHAP’s approach to gathering best practices be more global.
• The panel observed that currently DHAP’s operations research is focused on adaptation, effectiveness/replication studies, and improving Effective Behavioral Intervention (EBI) implementation. While these activities are appropriate, the panel recommended a redefinition of applied research in DHAP that maximizes opportunities to solve the real world problems of dissemination and implementation. The adaptation process, for example, is not the subject of current research. The panel recommended that the research focus on the processes of a dissemination science that emphasizes scientific studies of the training procedures, end-user response to guidance, implementation strategies, contract manager effectiveness, enhancers and barriers to scale up, evidence in the real world for effectiveness, and capacity building. This paradigm shift is needed to conduct organizational and system-level research to evaluate the ultimate effectiveness and cost effectiveness of interventions as implemented by real-world service providers. The panel also recommended a community-driven research agenda at the CDC that takes advantage of CDC’s unique relationships with CBOs and health departments and encourages input in the identification of potentially effective “home-grown” interventions.

7. To what extent does the behavioral research portfolio advance the current state of science?

• The panel stated that DHAP has a measurable impact among emergency rooms and health departments, but the link between science and activities implemented is not clearly stated, and priorities are not specified.
• The panel stated that DHAP’s research focuses on HIV-testing strategies and behavioral interventions, which does not appear guided by research findings. Therefore, linkages between research and program and how research findings impact incidence and prevalence are not apparent.
• The panel discussed the impact of conducting population-based research. They stated that focusing on populations resulted in research being directed at populations regardless of epidemiological burden. The panel emphasized that research should be based on the science and not other considerations. Research based on behaviors would be more realistic and would be easier to disseminate as it is what the practitioners are seeking.
• The panel stated DHAP should consider the literature about what is happening globally to guide the creation of models for the United States.
• The panel recommended the use of social marketing theory in addition to the use of traditional behavior change theory when developing programs and interventions.

8. To what extent is the research translation model effective and efficient in moving proven interventions and strategies into practice?

• The panel recommended that a social-marketing approach should be integrated into more behavioral research activities. The use of this approach in developing public communication programs is commendable, but researchers need to shift focus from individual-level behaviors to more social and structures issues (e.g., from testing to stigmas, access to service). They also recommended that health departments and CBOs be linked into campaign and program development early on to assure that local roll-outs and implementation capacities are most effectively and efficiently deployed. Social marketing also has a central role to play in scaling up intervention dissemination and translation of research to practice.
• The panel stated that the findings of research can be disseminated faster.

9. What are your recommendations for changes (e.g., enhancements, additions, activities that can be de-emphasized), future directions, and priorities for the behavioral research portfolio?

• The panel recommended that CDC should engage in behavioral and social research— formative (Phase 1) and operations/effectiveness (Phase 4)—that target practices and behaviors that fuel HIV epidemics across populations and settings; and CDC should work with CBOs and health departments to determine how best to focus and tailor these responses in the particular communities. (For example, at the individual level—research on programs and interventions that address concurrent sexual partnerships, unprotected anal intercourse, non-disclosure of HIV status. At the societal level—research on CBO and health department capacity to deliver HIV prevention services [including, but not limited to EBIs]; scale-up of access to and uptake of male and female condoms, etc.)
• The panel recommended that DHAP review data on epidemiology and varying levels of risk at the individual level and racial and ethnic disparities to develop and implement program approaches, interventions, and public health strategies.
• The panel recommended the use of more extensive applied research methodologies to analyze the implementation of interventions in real-world conditions and the actual cost of program implementation.
• The panel recommended that research be expanded to include mass communication (especially agenda setting) and sociological theories and models (scaling social innovations, diffusion of innovations) to better capture both the complexities of the challenges and also illuminate new solutions and innovative approaches to craft them.

Top of Page

E. Prevention Programs, Capacity Building, and Program Evaluation

The Program Panel reviewed information on HIV-prevention programs funded at the national, state, local, and community-based levels, including primary prevention, behavioral interventions (DEBIs), public health strategies, capacity-building activities and public information campaigns. The Panel was able to review the types of programs and activities being implemented, monitoring and evaluation data currently available, and plans for enhancing the prevention programs in the future.

Topics to be addressed:

• Community planning
• Counseling and testing
• HIV Screening
• Partner services
• Prevention interventions (behavioral, structural, biomedical)
• Communications and social marketing
• Capacity building
• Program evaluation

1. To what extent are the Division’s prevention programs, capacity-building activities, and program monitoring and evaluation approaches evidence-based, of high scientific quality, and consistent with the current state of HIV prevention science?

• DHAP lacks a mechanism for programs to explicitly interface with science and integrate various sources of data (e.g., surveillance, research agenda, evaluation) to ensure bilateral priority setting and exchange.
• DHAP should strongly promote evidence-based approaches that work, including syringeaccess interventions, the safe disposal of syringes, comprehensive sex education, and condom availability and effectiveness.
• We do not know whether the national collection of HIV intervention are “working,” addressing populations at greatest risk or reducing health disparities, because evaluation data are not yet available.
• The Program Evaluation Monitoring System (PEMS) used now does not meet our needs. We are stuck with legacy elements of PEMS (i.e., the information technology infrastructure and the inability to accept and analyze data and to generate reports).
• The panel recommends that PEMS be replaced. Until a new data system is implemented, CDC needs to continue to work with state and local partners on collecting data through PEMS.
• The committee acknowledges the strengths of DEBIs in the prevention portfolio; however, DEBIs are not in and of themselves the sum total of an effective, comprehensive prevention portfolio. There are shortcomings that need to be addressed.
  • There needs to be an increase in DEBIs targeted to gay men and gay men of color, so as to limit adaptation and translation. There needs to be a synthesis of core concepts of similar DEBIs.
  • There needs to be increased attention to and resources for the use of novel technologies for prevention interventions, especially technologies used by gay and bisexual men to meet partners.
  • CDC should continue to focus on the effectiveness and cost effectiveness of its programs.
  • CDC needs to balance the limited population reach of DEBIs with the need to scale up to reach larger populations. To the extent possible, CDC should link broad-scale activities with greater reach to more specialized behavioral interventions for higher-risk individuals.
  • CDC should continue to support the evaluation of “locally grown” interventions through expertise and resources.
• The prevention communication group should review and apply the science of strategic message framing and risk communication.
• CDC should be concerned about the effectiveness and cost effectiveness of their programs.
• CDC should provide tools and technical assistance to allow for health departments to calculate their incidence estimates using the back-calculation methodology and to implement behavioral surveillance.

2. To what extent are the objectives of the Division’s prevention program, capacity building, and program monitoring and evaluation portfolios adequately described, clearly linked to, and consistent with the Division mission, goals, and priorities?

• They are not described. CDC needs to communicate its frameworks and priorities more succinctly. We need a more comprehensive presentation of DHAP’s portfolio.
• There needs to be a mapping and modeling of interventions, programs, and resources to more clearly describe how the portfolio will meet the goals. Without such mapping and modeling, the determination of the mission appropriateness and linkage (goals, etc.) cannot be accomplished.
• DHAP and NCHHSTP should revisit their mission and goals, particularly in the context of the social determinants of health discussion. There needs to be more specificity, targeting and prioritization of activities to address these overarching risk determinants.

3. To what extent do the Division’s prevention program, capacity-building, and program monitoring and evaluation activities meet the needs of CDC’s constituencies (e.g., training and technical assistance activities sufficiently address growing and evolving needs of public health workforce; program monitoring and evaluation activities result in sufficient accountability, least burden to constituents, and best utilization of data to inform programmatic efforts)?

• The panel defined constituencies as CBOs, national organizations, health departments, policy makers, communities, any grantee, and the general public (re: testing as well as social marketing).
• Evaluation data needs of local areas are not being met. CDC should start with needs assessment at the local level then go up rather than in the opposite direction. (See evaluation comments throughout). To the extent possible, CDC should move beyond process monitoring (aka PEMS) to increase program capacity for quality assurance and outcome monitoring.
• The panel recommended an increase in cultural competency of capacity building and technical assistance by implementing technical assistance programming more flexibly. Currently there is a need for more culturally matched and locally appropriate technical assistance providers.
• The panel recommended giving CDC funded organizations the opportunity to select from identified providers of capacity building and technical assistance or allow these providers to identify their own providers of technical assistance. To implement this now, CDC should allow CBA providers to use their grant resources to purchase the needed services from organizations that can provide needed and culturally competent services (if CBA providers cannot do so).
• Regarding the public health workforce, the panel recommended increasing the focus of the capacity-building program on workforce retention, public health core competencies, and capacity. These efforts should include all levels of the public health workforce (community based, health departments, and CDC).
• CDC should ensure through recommendations and funding that research institutions funded by CDC collaborate and disseminate information and findings with health departments and community-based organizations.

4. To what extent are the Division’s prevention program, capacity-building, and program monitoring and evaluation activities adequately evaluated, and to what extent are the interventions and tools within the capacity-building portfolio effectively disseminated?

• While the panel recognizes the importance of this question, it could not respond to this question due to lack of data.
• DHAP should initiate a comprehensive, ongoing, and proactive review and evaluation of the portfolio and dissemination.
• DHAP needs to strengthen mechanisms to share lessons learned from evaluation activities and successful practices.

5. To what extent does DHAP have adequate capacity and sufficient resources devoted to prevention programs, capacity building, and program monitoring and evaluation, consistent with the Division mission, goals, and priorities?

• The panel concurs with CDC’s professional judgment and its budget request to Congress; however, we cannot be entirely sure whether this budget will assure the capacity to meet DHAP’s goals.
• CDC should maintain the programmatic priorities irrespective of budget increase and align the budget with these priorities.
• The panel recommends an increase in full-time equivalents (FTEs) and funding for programs and evaluation. (See other evaluation comments throughout the report).
• There should be a dedicated proportion of resources reserved for core functions such as evaluation. CDC should review other agency models for examples.
• The panel acknowledges the need to halt some activities throughout DHAP, such as legacy projects that are no longer appropriate.
• CDC should redirect discretionary resources (FTEs and funding) and scientific capacity toward the prevention programs.

6. To what extent is the mix and balance of prevention program, capacity building, and program monitoring and evaluation activities appropriate and relevant to the current epidemic?

• There is a need for a combination of effective approaches in the fight against HIV. DHAP should develop a comprehensive prevention portfolio, and it should include structural interventions reflecting a social-determinants-of-health framework and biomedical interventions.
• The intervention mix does not adequately meet the needs of gay and bisexual men of all race/ethnicities or all MSM.
• Surveillance/Data Collection does not currently represent the epidemic with the current surveillance variables (e.g. MSM versus gay-identified men; non U.S. born individuals).
• The panel recommends updating the social marketing portfolio to represent the epidemic, the state of program science, and the needs of populations. Not all social marketing materials should be translated, but many should be developed in native languages.
• Establish a well-defined, well-described portfolio and associate with it an evaluation framework and data sources that help monitor implementation and measure effectiveness.

7. To what extent are the current nine essential components of comprehensive state and local HIV prevention programs still appropriate, based upon current surveillance, research, laboratory, epidemiological, and program evaluation data?

• The panel recommended a redefinition of what are the essential elements of a Comprehensive HIV Prevention Program, which are defined in the Supplemental Guidance for HIV Prevention Community Planning, to incorporate activities currently carried out by non-governmental organizations at the national and community-based level.

8. To what extent does HIV prevention community planning effectively inform programmatic efforts and meet the needs of constituents, affected populations and other partners?

• The panel recognized the importance of community planning and its ability to increase accountability, transparency, community participation, community leadership capacity, and policy capacity.
• The panel also acknowledged the high and disproportionate level of resources allocated to community planning in many jurisdictions and the need to balance the valued process with the need for local flexibility and appropriate resource allocation to these functions
• CDC should explore new, flexible, and more resource-efficient models for maintaining community input, accountability, and transparency. The guidance should allow for a variety of models that assure these elements (input, accountability and transparency).
• CDC needs to use the HIV Prevention Leadership Summit (HPLS) as a means of reinvigorating the energy around community planning, providing the opportunity for information and innovation transfer among jurisdictions, and developing community planning leadership. CDC should also explore regional meeting opportunities.

9. To what extent do the Division’s HIV prevention programs address the goals of reducing health disparities; supporting program collaboration; and facilitating integration of HIV, STD, viral hepatitis, and TB services at the client level?

• Establish a prioritized effort to examine the impact of discrimination on health outcomes. (For example, there is a need to recognize the centrality of gay and bisexual men in the HIV epidemic who face tremendous health inequalities. There is a need for a lesbian, gay, bisexual, and transgender [LGBT] focus and an office to examine the general health of LGBT people; particularly if we move toward a social-determinants-of-health model and take seriously the impact of homophobia and gender bias, among other factors, on health outcomes.)
• The panel supports the prioritization of focus on blacks, African Americans, Latinos, Asian and Pacific Islander communities, Native Americans and immigrant populations who face tremendous health disparities.
• Regarding Program Collaboration/Integration, the panel recommends increasing collaboration/interface between surveillance, evaluation, and prevention programs to ensure bidirectional learning and planning.
• The panel recommends an increase in cross-Center and collaboration between health departments and HHS Program Collaboration and Service Integration (PCSI) and consistent promotion of collaboration among programs who receive funding from multiple federal sources (CDC, HRSA, NIH, etc.)
• The panel recommends increased flexibility in cooperative agreements (e.g. 10 percent flexibility of use allowance across CDC Cooperative Agreements) to allow program and service integration in order to meet the needs of the local epidemics (e.g. HIV, TB, STD, viral Hepatitis).
• The panel recommends increased resources for intra-agency support and collaboration, more FTEs across the board, but if the programs collaborated better, then maybe they could tap into each other’s resources. And surveillance should be part of the process as well.
• There is an opportunity for coordination/collaboration across CDC programs in the Center to collaborate on social determinants of health. As HIV moves toward biomedical interventions, the panel expressed hope that we can learn from successes and failures of the other disease areas (STDs, hepatitis, etc.)
• DHAP should look for opportunities to integrate HIV into the context of other programs (school health, social services, substance abuse services job corps, etc). CDC should develop a common language and public health framework across programs.

10. To what extent are effective mechanisms in place to ensure that programmatic needs and perspectives are informing the Division’s surveillance and research agenda?

• Mechanisms are not in place (or if they are, they are not adequate). CDC needs to assure program-science interface for the setting of the scientific agenda and to assure translation of science into interventions.
• CDC should establish a practice/program community review of the research portfolio on an ongoing basis. The review should examine research priorities and portfolio.
• CDC needs to increase operational, translational, and optimization research to define the optimal mix of prevention programs.
• See above comments for program collaboration and need for mechanism to assure bidirectional exchange between research and programs.

11. What are your recommendations for changes (e.g., enhancements, additions, activities that can be de-emphasized), future directions, and priorities for the prevention program, capacity building, and program monitoring and evaluation portfolios?

a) Move toward social determinants of health framework.

• Social determinants of health should at least include: poverty, racism, homophobia, incarceration (and mass imprisonment), homelessness, substance abuse, immigration, and power inequities.
• This broadened approach should extend throughout the DHAP portfolio and should become part of CDC’s conversation in the national health reform discussion.
• CDC should seek additional consultation about moving toward this framework.

b) Recognize the centrality of gay and bisexual men in the HIV epidemic.

• Given that the majority of cases of HIV remain among gay and bisexual men of all races and ethnicities, DHAP should refocus and emphasize gay, bisexual, MSM, and transgender people of all race and ethnicities facing tremendous health inequities in communications, research, and programs.
• DHAP should determine if the disparities framework will be expanded to include MSM or if a complementary inequities framework is more appropriate to facilitate goal setting and use of resources.
• DHAP should prioritize gay men and differentiate between gay men and the broader category of MSM.

c) Strategically allocate DHAP resources, and improve DHAP’s organization and functioning in order to respond to the epidemic, establish broad partnerships, and innovate.

• Given the changes afoot including health reform and the development of a national AIDS strategy, DHAP should help to create a framework for resource distribution, which reflects the epidemic, recognizing the need for base funding for performing core functions. DHAP should utilize the range of data for assessing the need and capacity.
• CDC should identify more useful ways to learn from information about what is happening in the epidemic (interventions, epidemiology, etc.) at national and local levels.
• CDC should streamline external and internal clearance and procurement processes to reduce barriers to prevention program implementation (e.g. OMB and internal CDC processes).
• DHAP should increase organizational flexibility to implement innovation and change and consider organizational restructuring to ensure better program collaboration.
• CDC should develop integrated models that facilitate effective integration among the centers and their funded programs.

d) Design, fund and implement a more strategic evaluation approach to facilitate learning across the organization and across a network of providers, to inform prevention and capacity-building needs, and to address the information needs of external stakeholders.

• DHAP should distinguish which data should be collected locally and which nationally.
• CDC should orient evaluation toward both process and outcome.
• CDC should prioritize the establishment of a different and more flexible prevention evaluation data system.
• There must be close collaboration with state and local partners in the solution for the interim and long term.

e) Reclaim prevention science, and use it actively to inform policy and program.

• CDC must provide scientifically accurate, culturally appropriate information and tools to the public (e.g., syringe exchange, condom use, promotion, comprehensive sex education). We need to evaluate the impact of the Helms Amendment on the provision of information and programs to our partners, particularly if we are prioritizing sexual minorities.
• Because the science of syringe exchange in reducing HIV is broad and compelling, CDC should request a lifting of the ban on the use of federal funds for syringe exchange.
• CDC should assert the efficacy of correct and consistent condom use.
• CDC should fully support (in program and funding), comprehensive sex education as part of an effective HIV prevention strategy.
• Social marketing and health communications should reach specific populations at greatest risk for HIV as well as stakeholders.

f) The framework of “essential elements,” while historically focused on health department cooperative agreements, should now describe the entire CDC HIV portfolio, not just what is done with and through health departments.

• The “Nine Essential Elements” need to be revisited, updated, and expanded to include the modes of operation and spheres of influence in which HIV prevention operates.
• CDC should convene a consultation to examine the elements.

Top of Page

Go to Section VI