A. Planning, Prioritizing, and Monitoring
The Planning Panel was briefed on the processes by which DHAP plans programs and activities,
including the analysis conducted to determine the cost-benefit of the implementation of various
activities. The panel also reviewed information on budget allocations, the decision-making process
within the Division, and the National Program Monitoring and Evaluation process currently
underway. The panel provided extensive input on the planning role of DHAP, not only for its own
activities within CDC, but also in the future as the administration prepares to develop a National
HIV/AIDS Strategy (NHAS).
Overarching recommendations were identified by the Planning Panel that do not relate directly to
any particular review question but are important to include. The panel felt strongly that CDC
DHAP leadership on domestic HIV prevention is needed now and that as the lead public health,
science-based prevention agency in the nation, CDC DHAP should be the nation’s voice on HIV
prevention. The panel emphasized the expectation that CDC DHAP will play a leadership role in
the development of a National HIV/AIDS Strategy (NHAS) as well as a significant monitoring role
that the panel recommends include making public how we are doing as a nation in meeting the
objectives and goals of the NHAS (e.g., a national report card, the HIV state of the union).
A second significant recommendation from the panel is that DHAP be forward-thinking and futureoriented
in its planning, resource allocation, and evaluation of its efforts with the goal of preventing
as many new HIV infections as possible as the primary guiding principle. The concept of
“combination prevention” was endorsed along with the recognition that to be successful, the nation
needs to scale up significantly the coverage, scope, and intensity of HIV prevention in the United
States. For planning purposes, the panel recommends an approach which includes the development
of several different funding scenarios—what is likely to be achieved with current resources, with
increased resources, and with decreased resources. The cost of failure to act must be considered.
The importance of greater transparency on planning and prioritization processes was stressed. Prior
strategic planning efforts have not been well understood, and it was unclear to some panelists how
they came to be, what data informed them, and who did and did not have influence on their
development. In addition to greater transparency on future planning efforts, the panel recommends
that the expertise of CDC staff be used more effectively in future planning. It was also noted that
the goals and objectives in prior plans have not always been specific, measurable, achievable,
and/or realistic within the specified time frame and with actual resources in mind.
The panel questioned whether or not DHAP used its own data to plan and evaluate its efforts
effectively at the Division level and across all of the various Branches. The panel recommends that
the Division develop clear ways of taking stock of what is happening internally, externally, and
relative to set objectives on an ongoing basis. DHAP should develop data-utilization plans and look
at what is needed to make decisions at the Division level. The panel recommends one simple,
integrated nimble (prepared to act quickly) system that is geared toward strategic decision-making
and the dissemination of appropriate and timely information for actionable use internally and
Greater efforts should be made to maximize learning via the Division’s monitoring and evaluation
activities. DHAP should develop evaluation questions that are better tied to strategic decisions and
initiatives. Questions should change over time as appropriate, reflect goals and objectives and,
where possible, be prospective. Attention must be made to improve the quality of the indicators and
the data collected to reflect those indicators. DHAP’s indicators should be adequately specific,
capturing important pieces of information (cost, unintended consequences), or clearly linked to
process and outcome objectives at the division and program level. Information should be used to
determine success and improve decision-making and implementation of DHAP efforts.
The panel felt strongly that CDC DHAP needed to have a stronger voice in policy beyond its
accepted processes of guidance and recommendation development. DHAP needs to become an
astute user of the existing public policy mechanisms, including budget, legislation, regulations, and
public affairs, and assist state, local, and directly funded grantees in translating policy data and
scientific analysis into action. DHAP should provide the public with easy access to information,
translated in a user-friendly way. Finally, the panel acknowledged the need for CDC to be both
nimble and transparent in order to ensure stakeholder buy-in and support.
Topics to be addressed:
- Strategy development and planning
- Priority setting (including Resource Allocation Model)
- Results monitoring (esp. DHAP monitoring and evaluation plan)
1. To what extent are DHAP’s processes for planning, strategy development, and priority
setting explicit and technically valid?
- The panel requested clarification on how the strategic plan translated into an operational
plan. Not all the panel members were aware the HIV Strategic Plan had been extended to
- The panel also wanted clarification on the distinction between the Strategic Plan and the
Professional Judgment Budget, the latter’s development, who will be served, and how
progress will be measured.
- The panel requested specific information on DHAP’s policymaking role, including how
processes work within CDC for engagement with HHS, the Office of Management and
Budget (OMB), and Congress and what limitations and boundaries exist, with a particular
focus on the budget process. The panel further inquired as to what authority, including
statutory authority, DHAP would require to be better engaged in policymaking processes.
The panel recommended DHAP focus on its own organizational functions as a basis for
future plans, avoid duplication of effort, and provide rapid response to emerging issues.
DHAP can also have a broader scope for planning and use input from within the Division.
2. To what extent do these processes make adequate and appropriate use of data?
- It was not entirely clear to the panel what data were used to inform planning and whether or
that not all of the data that might inform planning were being brought to bear on planning The panel suggested that CDC take stock of its potential data sources and their potential
value for planning and evaluation at the Divisional level.
- The panel noted the lack of sufficient evaluation data on capacity-building efforts, including
evaluation data on the tailoring and adaptation of the evidence-based interventions.
3. How might DHAP best incorporate external input into its planning, strategy development,
and priority setting?
- The panel recommended that stakeholder involvement occur at various stages in order to
achieve buy-in into the planning outcomes. External nongovernmental stakeholders expect
CDC to lead a planning effort that includes stakeholder input with the optimal goal of
gaining external support for DHAP programs and priorities.
- On a wider plane, the panel emphasized DHAP’s role in fostering a better understanding of
HIV prevalence and its impact on future infections (e.g., via changing HIV transmission
rates which depend on both incidence and prevalence). The panel stated the importance of
CDC making strong leadership statements that would garner community support. The panel
indicated that the purpose of the plan must be clearly stated.
- The panel underscored the need to have members of the CDC/HRSA Advisory Committee
on HIV and STD Prevention and Treatment better understand the structure and operations
of DHAP in order to be better prepared to monitor and make recommendations regarding
4. What are the strengths and weaknesses of these processes and what are your
recommendations for improving them?
- The panel emphasized the need to anticipate future needs in HIV prevention with planning
based on surveillance and epidemiology data, prevalence and transmission rates. Planning
efforts can be organized according to prevalence, with high-, moderate-, and low prevalence
jurisdictions, and with special consideration of populations and jurisdictions
with estimated low prevalence of HIV (Asian/Pacific Islanders, Native Americans/Alaska
Native, U.S. Virgin Islands and Pacific Territories). The panel recommended creating a
working group to provide input on future challenges. This panel could be charged with
creating a policy- and program-relevant “dashboard” of the HIV epidemic suitable for
timely and comprehensive program planning and evaluation.
- The panel provided several recommendations regarding planning with a particular focus on
- The panel recommended that several effectiveness analyses be used in planning
including, the number of new infections that could be averted with a) current
resources, b) incremental increases in resources, c) increases based on documented
need, d) level funding, e) no change (or the consequences of failing to act). Planning
can also look at the marginal utility of additional funding to maximize its impact.
- The analyses can support funding in other agencies and for activities not covered
elsewhere. Planning can be based on overall funding for HIV activities (i.e.,
treatment and research). Planning may result in goals that go beyond the current
parameters, while other goals may be more achievable.
5. How appropriate and relevant are DHAP’s strategic priorities to the current epidemic
with respect to populations and strategies?
- The panel recommended an internal review of CDC HIV prevention funding at the various
organizational unit levels (CDC OD, Coordinating Center, Center, Division, Branch) and a
review of historical information, and for DHAP to gain control over all available HIV
- The panel also emphasized the need for greater transparency in the planning process so
external sources could learn where funds are being directed. In particular, the panel
recommended a measure of the cost of delivering services to clients and to examine these
efforts at the national level.
- Data on cost effectiveness can be used to report to Congress and OMB to provide
information on the relative efficiency of CDC’s HIV prevention efforts relative to other
investments of federal resources.
- The panel recommended that the economic reality be considered, including survival of
CBOs and the cost of prevention that can be sustained over time.
6. How appropriate is the relative mix of prevention efforts directed toward persons living
with HIV and those not yet infected?
- Programmatically, the panel stated that DHAP may want to focus on some key strategies
for HIV prevention and evaluate them thoroughly. Examples of these would include
increased efforts within prisons and other corrections facilities, mental health and substance
abuse treatment providers, and supported housing.
7. How appropriate are the substance and scope of DHAP’s strategic priorities relative to
the Division’s mission?
- The panel recommended developing the framework of “highly active HIV prevention”
(HIHP) in order to articulate the need to scale up prevention efforts at the national, state,
and local levels. A comprehensive framework would enable jurisdictions to better target
and tailor their HIV prevention response to their local epidemics. Efforts can also be
expended to increase understanding of HIV prevention, foster improved messaging and
marketing, and provide more information on planning efforts.
8. How clear and focused are DHAP’s strategic priorities, and have they been articulated
and communicated adequately?
- The panel recommended a higher level of policy leadership, beyond funding, to support
strategies such as needle exchange programs (NEPs) and to ensure adequate support for
existing and new initiatives.
Prevention activities can also be supported by seeking synergies with other activities and
other federal agencies, including testing expansion, screening, and linkage to care, and by
treating HIV treatment as prevention.
9. How well do DHAP’s strategic priorities support 1) collaboration among HIV, STD, viral
hepatitis, and tuberculosis programs and 2) integration of HIV, STD, viral hepatitis, and
tuberculosis prevention services at the client level?
- A plan must include comprehensive program and policy alternatives that are prioritized and
rely on synergy among the national, state and local levels; be adjusted as needed; and be
based on the data collected.
- A model can be used to quantify the priority-setting process and the selection criteria,
especially to determine the priority for activities for certain populations (transgendered) and
to determine the impact of new strategies.
- The panel recommended that DHAP develop a formula for prioritizing activities based on
the epidemic burden. The formula should include incidence and prevalence rates and use
data from DHAP and other sources (e.g., foundations, national organizations).
- The panel requested that DHAP develop a more clearly articulated policy agenda along
with an analysis of how policy changes might impact program effectiveness and, ultimately,
the trajectory of the epidemic. At present, policy may include DHAP guidelines, reporting
requirements, state and local laws, and policies that are outside of CDC’s purview. The
panel emphasized the need for transparency, including at the Financial Management Office
- Some barriers were cited by the panel, including delays in obtaining input from the chain of
command, and the role of Coordinating Centers within CDC, and the Office of the Director
and communications with Congress.
10. How well do DHAP’s strategic priorities support reduction of health disparities?
- The panel did not have an explicit conversation about this question, but the theme of health
disparities was woven throughout many of the conversations. However, an underpinning of
the panel’s discussion was the need to address HIV/AIDS disparities and ensure that DHAP
planning and prioritization processes resulted in better targeting of resources to address
disparities based on race, ethnicity, and sexual orientation.
11. What are the principal gaps in DHAP’s strategic priorities?
- The Panel emphasized the need for a National HIV Prevention Plan—likely to be part of the
National HIV/AIDS Strategy—in which CDC can play a pivotal role for the federal
government, providing a national public health vision, leading by example the rest of HHS
and the Executive Branch, including the White House. The panel members indicated that a
leadership model was required to include input from within DHAP and other governmental
and non-governmental entities that have valuable input on the development of this National
Plan. DHAP can provide leadership in this process and encourage participation from
- The panel indicated that it is DHAP’s role to lead the HHS HIV prevention effort,
collaborate with other federal agencies, both within HHS and with other departments.
DHAP’s unique role is to prevent as many HIV infections as possible, considering all available opportunities, and report annually on the state of the epidemic so that future
strategies will be based on the likely trajectory of the epidemic.
12. To what extent is DHAP’s national monitoring and evaluation plan explicit and
- Monitoring efforts can begin with determining a standard set of questions to be addressed,
including input from grantees, and one standardized system that can incorporate all data
collected, ensure there is not an overwhelming amount of data, and consider unintended
consequences. In developing this centralized system, the lessons learned of the Program
Evaluation and Monitoring System can be used. The process to develop and revise the
questions should be flexible to respond to the reality of government.
13. To what extent are the data sources included in the plan adequate and appropriate, and
what other data sources should be included?
- The panel recommended the standardization of data collection efforts across all federally
funded testing initiatives, uniformity of data reporting and coding requirements among
jurisdictions and the validation of data. The social impact of the testing activities within the
various populations can also be explored.
- Data collection efforts can go beyond CDC to include other HHS agencies that can report
standardized demographic information on populations.
14. How adequate and appropriate are the outcome and impact measures described in the
plan in terms of their ability to assess DHAP’s programs and their public health impact?
- The panel raised the issue of who would have ultimate responsibility for monitoring and
accountability, what data should be collected, and what would be measured. The panel
stressed the use of various types of data, including core surveillance and grantee evaluation
data, and lessons learned and the need to adequately fund the data collection efforts and
15. What are the strengths and weaknesses of the plan and what are your recommendations
for improving it?
The panel concurred with the following statements:
- “To achieve optimal public health impact, the appropriate combination of evidence-based
HIV prevention strategies must achieve sufficient coverage, intensity, and duration.”2
- Combination HIV Prevention: “Effective HIV prevention involves the simultaneous use of
diverse and integrated prevention strategies—programs that help individuals prevent
transmission, broader-based initiatives that alter the norms and behaviors of social groups,
and increased access to tools that reduce the biological likelihood of transmission.”
- Analogy to HIV Treatment: “The parallels between HIV prevention and treatment are
striking. Like antiretroviral therapy, HIV prevention is life-long, and its impact must be
continually monitored and the prescribed regimens revised as circumstances and needs
change. Just as a single pill cannot eradicate HIV, one-shot prevention efforts will not
achieve the magnitude or sustainability of behavior change required to alter the epidemic’s
course…Like treatment, effective HIV prevention requires a combination of strategies.”
The panel made the following issue-specific recommendations:
- DHAP should continue conversations around needle exchange and preparation for the
possible lifting of the ban on the use of federal funds.
- DHAP’s sero-sorting position statement is an opportunity to identify and respond to a
evolving issue in the future course of the epidemic.
2 Source: The Global HIV Prevention Working Group, Behavior
Change and HIV Prevention: (Re)Considerations For The 21st Century, August
Top of Page
The Surveillance Panel reviewed written materials and heard presentations from DHAP personnel
on the various components of the surveillance system and explored the sufficiency of CDC’s
efforts in this area. The panel also considered the extent to which the data gathered through core
and other surveillance activities are used to inform the decision-making process internally and
externally and the interaction of program and surveillance.
- HIV case surveillance
- Incidence surveillance
- Drug resistance surveillance
- Behavioral surveillance
- Clinical surveillance
The Surveillance Panel recognized that DHAP and the jurisdictions are at a turning point in the
ability to monitor, characterize, and impact the epidemic: for the first time in the history of the
epidemic there is a national confidential name-based HIV reporting system; DHAP, in conjunction
with the jurisdictions, has developed a rigorous Technical Guidance document that details required
activities and best-practices; universal process and outcome performance standards have been
published and are beginning to be monitored; training on the Technical Guidance and performance
standards has occurred and is ongoing; and by the end of 2009 all jurisdictions will have
implemented a new data management system that will allow jurisdictions and CDC to analyze
surveillance data more sensitively for quality assurance and monitoring purposes as well as to
support the increasing volume of electronic reporting technologies. In 2008, two other significant
developments related to HIV surveillance were realized by CDC: a revised national HIV incidence estimate was derived from the young HIV incidence surveillance system and the CDC released
guidance to promote the integration of HIV surveillance with HIV partner services activities. The
current system, which includes core surveillance as well as HIV incidence, resistance, behavioral
and clinical surveillance components, also has many challenges which are discussed in the
1. To what extent are the surveillance methods and resulting data of high scientific quality?
The panel members concurred that the Core Surveillance methods are robust and the system is
becoming stronger with nationwide implementation of confidential name-based HIV reporting,
documented technical guidance, and standard evaluation criteria. The quality of data from core
surveillance systems, however, vary from jurisdiction to jurisdiction depending on the extent to
which sites have implemented the required activities such as birth and death ascertainment
activities, Routine Interstate Duplicate Review (RIDR), laboratory reporting (including CD4
counts and viral loads), document-based surveillance, and name-based reporting. As more
people are living with HIV and living longer, and as the surveillance system continues to
evolve in ways that can facilitate monitoring and identification of prevention opportunities
along the spectrum of disease, the demands on the fundamental “core” system are expected to
continue to increase. The panelists identified the following issues that impact the system’s
ability to produce data of high scientific quality:
- There is a need to strengthen the core surveillance infrastructure, potentially as a priority
over other ancillary surveillance activities, so that sites are able to conduct the required
activities and meet the performance standards.
- The increasing volume of laboratory reports and other electronic reports is a significant
challenge to the system. Receiving and managing electronic data require sites to adopt
major data processing capabilities. More robust standard data matching and processing tools
and expertise are needed in many areas.
- Transition to the new data management system (eHARS) is not yet complete at the
jurisdiction level or within CDC’s data management and analysis processes. In addition,
important corrections are still needed in the data system, such as for HIV-2 cases to be
recorded and the calculation of variables for perinatal exposure cases.
- Surveillance programs and data management systems need to be prepared to incorporate the
evolving testing technologies in data collection, management, and dissemination activities.
Such advances may also necessitate timely changes to case definitions.
- Participants noted the need for an updated case residency consultation, reconsideration of
the hierarchical approach to classification of risk, and guidance related to implementing
surveillance of acute infections.
- Panelists observed that gaps in reporting from federal programs, such as Veteran’s Health
Administration facilities, affect many areas. Intervention is needed by CDC to ensure that
programs administered by federal counterparts are cooperative with national disease
- Capacity for data analysis and dissemination has challenges locally and within CDC (see
Question 3: Constituent Needs and Question 5: Resources and Capacity).
HIV Incidence Surveillance
HIV Incidence Surveillance is still a young surveillance system and its overall functioning and
quality has yet to be assessed. The system is complicated to implement, and panelist expressed
some reservations about the scientific quality of the incidence estimation methodology given
that not all labs participate, the inaccuracies inherent in Testing Treatment History (TTH)
information, and whether the fundamental assumptions of the model will continue to be
appropriate over time with changes in testing patterns and technologies. In addition, the extent
to which the incidence estimates are representative of the nation will be better understood once
there is a complete national HIV case surveillance dataset. Other specific observations and
- Panelists were cautious about the use of data from the current system as the sole marker for
national prevention goals. Locally, estimates do not appear sufficiently stable at this time to
inform prevention activities in most jurisdictions.
- Panelists recognized the importance of measuring incident infections as a part of
understanding the epidemic. Towards that end, some recommended that CDC work with
FDA and manufacturers for faster approval of 4th generation HIV diagnostic tests and new
tests for incident infections that could be conducted directly on specimens by the
laboratory/testing entity avoiding the need to secure remnant specimens.
- In order to evaluate the strength of data used in the current estimation model, CDC is
currently funding a special project to assess the accuracy of TTH data. Panelists suggested
that CDC consider whether data collected through core surveillance nationwide might be
able to assist with this validation, such as comparing self-reported testing information from
Counseling/Testing programs to surveillance.
- Some panelists also recommended that incidence data, in particular infections identified in
the acute phase, represent an opportunity to intervene and interrupt transmission. As CDC
moves forward with the development of approaches to monitor incident infections, panelists
were supportive of exploring surveillance/prevention collaborations to identify clusters and
attempt to stop transmission. The group supported the approach that CDC is advancing
through the “STOP” Project.
Drug Resistance Surveillance
Panelists recognized the potential value in monitoring medication-resistant strains of HIV
nationwide. The current system, however, is in the early stages of development and is
transitioning from an approach that requires acquisition of remnant specimens to a more
standard laboratory reporting surveillance method. Panelists noted that this activity is
logistically and scientifically complicated, and additional guidance is needed to ensure that the
resulting data are appropriately managed and interpreted. The system has not yet been
- Guidance is needed for jurisdictions to assess if they have sufficient coverage of resistance
testing in clinical practice to discontinue the cumbersome process of obtaining remnant
specimens that meet the rigorous handling requirements to be tested.
- Establishing reporting of results from laboratories is challenging, even for sites with
permissive regulations and advanced technical capabilities. The capacity of laboratories to
provide the data electronically is variable.
- Panelists identified many outstanding issues related to analysis and use of data, including
the appropriate timeframe for analyzing results (within three months of diagnosis versus
one year); the appropriate use of data for identifying clusters and informing prevention or
clinical activities; and submission of genotype results to public-access scientific databanks
such as GenBank. Input from sites and field experts are needed to inform additional
guidance and technical assistance to ensure appropriate and meaningful use of data from
The panelists concluded that the National HIV Behavioral Surveillance (NHBS) is a strong
project with good collaboration and analysis. The panelists had the following considerations
related to ensuring the validity and utility of NHBS data:
- Eligibility for funding is based on the 2002 AIDS case data; panelists recommended
considering if the project is appropriately representative now that universal HIV reporting
has been implemented and more recent data are available.
- The strengths and weaknesses of respondent-driven sampling (RDS) should be assessed.
- Panelists recommended that CDC consider more flexible models for the cycles or location
activity (e.g., consider single survey in some areas that have relatively homogeneous
populations; consider expanding activities to lower morbidity areas to ensure representation
and to gather information on non-urban populations, perhaps using a regional model to
achieve sufficient sample size).
- Reviewers encourage CDC to explore opportunities to incorporate behavioral surveillance
activities (not necessarily using the NHBS model) into TB, hepatitis, and STD program
activities as a means of adding to the portfolio of monitoring risk behaviors. Some panelists
also felt that there was an opportunity to conduct behavioral surveillance focused on acute
HIV infection (AHI). If nucleic acid amplification testing (NAAT) and surveillance for
clinical syndromes consistent with AHI were to become more widespread, there may be an
opportunity to interview a sizable number of persons with AHI to better understand their
risk behaviors and “thinking” (knowledge, beliefs and attitudes) at the approximate time
they were infected.
The Medical Monitoring Project (MMP) is important as the only present-day study that
attempts to assess a representative sample of HIV-infected persons in care to understand their
care utilization, clinical characteristics, and risk behaviors. The panel agreed that 2009 is a
critical year for MMP, which still has not been fully implemented after five years of funding.
The panel recommended that CDC evaluate the program to identify roadblocks (e.g. OMB
delays, stipends for and randomization of providers, capacity and resources to implement, real
time sampling); determine if these roadblocks are adequately addressed; answer questions; and establish specific performance indicators. After completion of 2009 data collection, panelists
recommended that the feasibility of the methodology be reviewed.
In addition, some panel members felt that some outcomes being monitored by the MMP (e.g.,
continuity of care) might be more easily monitored through measuring regularity of getting
viral load tests and CD4 counts as part of core surveillance. Where the MMP is in place,
comparison of outcomes using surveillance and the MMP should be done.
2. To what extent are the objectives of each DHAP surveillance system adequately described
in, clearly linked to and consistent with the Division’s mission, goals, and priorities?
The materials provided to the reviewers did not specifically articulate how the objectives of
each of the presented surveillance activities are linked to specific Division goals and priorities.
However, the panelists did consider the stated the HIV Incidence and Case Surveillance Branch
(HICSB) and Behavioral and Clinical Surveillance Branch (BCSB) goals of implementing
surveillance and research activities that guide public health action at federal, state, and local
levels. Overall, jurisdictions likely do not understand how their activities are linked to specific
DHAP goals and priorities. With surveillance data essential to monitoring many goals
established through various national strategic planning processes, such as CDC’s HIV
Prevention Strategic Plan and the expected development of a National AIDS Strategy, it would
be important for DHAP to ensure that surveillance jurisdictions understand how the DHAP
goals tie into larger national goals and how the local level activities support those goals. In
addition, the panelists had the following observations and recommendations:
- As noted in the summary, the case surveillance system is getting stronger with the
realization of universal HIV reporting in 2008 and the implementation of common
standards for the completeness, timeliness, and accuracy of jurisdiction-level surveillance
systems. Given the strengthening foundation and potential capabilities of surveillance, CDC
should re-examine the strengths of and demands on the surveillance system and reconsider
what surveillance should be measuring and doing as well as assess the resources that are
needed to support those activities as a partner in HIV prevention.
- The panelists were vigorously supportive of the inclusion of milestones in CDC’s Extended
Plan that are specific to strengthening the capacity nationwide to monitor the epidemic. A
robust surveillance system is essential to supporting sound programmatic decisions
nationally and locally.
- A stated common goal of CDC’s surveillance branches is to conduct activities that will
guide public health action at the federal, state, and local levels. Over the course of the
review, the panel discussed several challenges for surveillance to action:
- Cooperative agreements with the jurisdictions generally support data collection and not
data analysis. The Epidemiologic Capacity Building Technical Assistance funding is an
example of dedicated analysis effort and information sharing available to some areas,
supported by both Prevention and Surveillance resources, that could be further
developed to help ensure that surveillance data are analyzed and effectively
incorporated into local planning and evaluation processes.
- Some surveillance activities are not currently producing data that are able to be used at
the local level (see details in state/city/jurisdiction constituency issues identified in
- Stronger tools and approaches are needed to help jurisdictions ensure that surveillance
data and information from other studies and evaluation activities are utilized in a way
that is most likely to impact the epidemic.
- The newly released HIV Partner Services guidance recognizes the important prevention
opportunities that can be realized when surveillance programs collaborate with prevention
programs. Panel members supported CDC’s development of this guidance and noted the
ongoing need for CDC to incorporate rigorous security and confidentiality requirements in
cooperative agreements across the agency’s divisions.
3. To what extent do the surveillance systems and products adequately meet the needs of
The panel at different points in the review discussed several constituencies of DHAP
surveillance systems and products. In particular, the participants considered how surveillance
activities impact programs and processes within CDC, jurisdiction-level prevention and
surveillance programs, HIV Partner Services programs, and HRSA Ryan White CARE Act
programs. An overarching recommendation from the panelists, also articulated in the response
to Question 2, was for CDC to look outward to reconsider the scope of surveillance activities
and assess the increasing demands on the surveillance system.
CDC Prevention and Epidemiology programs as constituents
- The extent to which CDC surveillance personnel are incorporated into the CDC prevention
and evaluation planning processes was not clear to the reviewers. Given there are particular
strengths and weaknesses of the surveillance system activities and the resulting data,
surveillance expertise would be valuable both to interpret surveillance results and to guide
decisions being made within CDC for developing prevention strategies, prioritizing
interventions, developing campaigns, and establishing evaluation criteria or indicators.
- The panel participants identified some instances of gaps and other instances of duplication
across CDC divisions and DHAP’s branch activities where data that are already being
collected in one activity might be useful for another and might help support DHAP’s
evaluation of progress towards attaining goals. In particular, there may be a benefit in
evaluating the data collected through Core Surveillance, Medical Monitoring Project
(MMP), National HIV Behavioral Surveillance (NHBS), Enhanced Perinatal Surveillance
(EPS), Fetal and Infant Mortality Review (FIMR), and STD, TB, and other infectious
disease surveillance activities to identify any unnecessary redundancy as well as additional
available data that can help measure progress towards the Division’s goals overall.
States/cities/jurisdictions as constituents
- Although most CDC surveillance activities are intended to provide locally useful data for
monitoring the epidemic as well as guiding and evaluating prevention efforts, the benefit to
local sites is often not fully realized. For example:
- Core Surveillance appears to be funded primarily for data collection and management
activities, but not for local data analysis. DHAP should provide adequate funding and/or
technical assistance to ensure that data are analyzed to inform local program needs.
- Some jurisdictions funded for HIV Incidence Surveillance remain unclear about how
best to use the data to inform program or if that system will produce sufficiently stable
estimates to be meaningful locally.
- Currently many jurisdictions lack the capacity to analyze resistance sequence data. DHAP should provide adequate funding and/or technical assistance to ensure that data
are analyzed to inform local program needs as there is no consensus from CDC on
analysis criteria. (CDC personnel noted during the discussions that guidance and
technical assistance was anticipated later this year).
- Some areas relay that behavioral surveillance data are inadequate for genuine evaluation
of local prevention efforts.
- The respondent sample size for many MMP sites is small and could limit locally useful
data unless high patient participation rates are achieved by allowing and/or developing a
more successful sampling methodology, such as Real Time Sampling. Data has thus far
not been returned in a timely manner for analysis.
- Lower morbidity areas generally do not qualify for “supplemental” surveillance
activities, resulting in a less than complete characterization of the epidemic and of the
behaviors of persons in those areas and nationally.
- Jurisdictions are eager to have additional information to inform prevention. CDC’s
recent back-calculation of case surveillance data provided rich national data
complementing the new incidence estimation methodology and allowed for the
estimation of transmission and unknown status. Panelists encouraged that technical
assistance be available if such approaches could be applied locally.
- In order to assist surveillance programs with the provision of data to local prevention and
care programs, CDC developed technical guidance and analysis programs for jurisdictions
to produce a local Epidemiologic Profile document. While a very rich resource, many
jurisdictions have found the recommended document cumbersome both to produce and for
local planning bodies to use. The panelists recommended that CDC partner with the
jurisdictions’ surveillance and prevention programs to assess how the Profile is (or is not)
used and reconsider best practices for ensuring the appropriate incorporation of surveillance
and other data in local planning processes. Smaller, more frequently produced Profiles may
be more responsive to planning group needs. Panelists also noted that there is little funding
to specifically support the development of the profiles.
- The use of GIS mapping to better correlate HIV infection with other socio-economic and
geographic characteristics may provide information useful for planning; a recommendation
was made for CDC to consider supporting the technical capacity for GIS mapping and
small-area analysis of HIV by socio-economic status (e.g., percentage of persons living
below poverty in the “neighborhood of residence”).
- The panel acknowledged the Division’s production of several summary products that
accompanied the release of the incidence estimate in 2008 that areas found particularly
helpful and encouraged CDC to consider developing similar materials to accompany other
routine and non-routine surveillance reports. Often local jurisdiction staff, even those fairly
knowledgeable, find that some CDC surveillance reports currently are not at all easy to
Ryan White CARE Act programs as constituents (federal and local programs)
- HRSA programs are major “consumers” of surveillance data—national funding allocations
presume an equitable and high-performing surveillance system across the country. Also,
local care programs rely on surveillance programs for accurate, clear, simple summary data
products in order to guide allocations, planning, and grant applications. Appropriate
determination of unmet need for HIV primary medical care requires timely and complete
local surveillance systems with comprehensive laboratory reporting and strong analysis
capacity. Overall, panelists reiterated the importance of a strong collaborative relationship
between HRSA and CDC, as surveillance capacity has a significant impact on HRSA
programs both locally and nationally. Reviewers also noted the following:
- With confidential name-based reporting now implemented in all jurisdictions, panelists
urge CDC to reconsider its recommendation to HRSA on how best to utilize HIV
morbidity data for HRSA funding, specifically noting that CDC evaluate the feasibility
of switching from residence at diagnosis to most current residence for assigning state
“ownership” of cases.
- Grant applications to HRSA for the various Ryan White program “parts” require
extensive local analysis of surveillance data. Reviewers requested that CDC work with
HRSA to specify and consolidate the type and source of information needed for federal
funding applications and progress reports.
- Panelists also identified opportunities for collaboration and consolidation around the
data collected to monitor and evaluate Ryan White programs. With HRSA’s
development of a code-based client-level data system (a type of system just abandoned
by CDC), Ryan White programs are required to collect and record information that may
already have been collected and recorded in the de-duplicated surveillance system.
While not advocating for a merging of the two systems, panelists urged the agencies to
consider the efficiencies that could be gained with greater coordination around data
Partner Services programs as constituents
- The current surveillance system functions as a look-back system that is not particularly
timely for intervention. In the panel’s discussions related to HIV incidence surveillance,
participants noted the potential increasing importance of a more timely surveillance system
with the opportunities to interrupt transmission among reported acute infections,
specifically noting that CDC evaluate the feasibility of switching from residence at
diagnosis to most current residence for assigning state “ownership” of cases. As
technologies to detect acute and recent infections evolve, the importance of surveillance
working closely with Partner Services programs could be critical. Panelists encouraged that
CDC consider the potential role of surveillance as a part of the prevention portfolio and the
resources that would be needed for the system to serve in this capacity.
4. To what extent are the surveillance systems adequately evaluated and the results
effectively disseminated? To what extent are the data used by prevention activities for
program development and evaluation purposes?
Evaluation of surveillance systems and dissemination of results
- For Core Surveillance, HICSB has made a significant investment in preparing jurisdictions
for the implementation of rigorous, ongoing evaluation activities through the development
of process and outcome standards, publication of standards in the Technical Guidance and
articles, ongoing training, and incorporation of key standards in grant applications and
progress reports. In addition, by the end of 2009, HICSB will have completed the
deployment of the data management system, programs, and technical guidance needed to
implement national evaluation activities. The plan appears rigorous. Participants noted that
HIV surveillance seems to be one of the most comprehensive surveillance systems existing
for any condition. While CDC has established performance standards, a thorough
evaluation of the system has not yet been done nationally or locally in a uniform way.
Preliminary results of the informal reporting by the sites of required process and outcome
standards (from the 2007 APR and 2008 APR) have, to date, not been disseminated. As a
result, the panelists could not definitively speak to the quality of the data. Additional
- Panelists asserted that some components of the evaluation plan are very complicated
and may need to be reconsidered in light of what is practical in a non-research setting
and what is most important or meaningful for improving the system given the available
- With laboratory reporting becoming such a major component of the surveillance system,
there might be a benefit to developing process standards that prompt jurisdictions to
monitor and evaluate the laboratory-reporting component specifically. Although sites
ultimately need to meet the outcome standards and CDC does not necessarily want to
dictate how sites need to achieve those standards, additional guidance and
recommended standards would appear to be helpful.
- Feedback to jurisdictions about how they compare to other programs with the
implementation and achievement of standards would be useful.
- For HIV Incidence Surveillance, performance standards are defined, but some appear
unattainable in the current model of incidence surveillance that depends upon the
availability of remnant specimens. The panelists also noted that incidence data are part of
several national prevention monitoring and evaluation goals, but question if the system is or
will be sufficiently robust to provide those measurements alone.
- No formal evaluation has been conducted to date for the Variant, Atypical, and Resistant
HIV Surveillance (VARHS) program. This is a developing system that is complicated and
achievement of the defined outcome standard will be difficult, especially as long as sites
rely on testing of remnant specimens.
- The panel recommended that CDC evaluate National HIV Behavioral Surveillance
(NHBS) once two complete cycles have been completed to assess the strengths and
weaknesses of respondent-driven sampling (RDS) and consider a more flexible model for
the cycles or location activity (e.g., consider single survey in some areas). For example, in
areas whose epidemic is almost entirely MSM, it may not make sense to conduct the IDU or
- The panelists strongly recommended the CDC evaluate the Medical Monitoring Project
(MMP) in early 2010 to assess if the protocol can be implemented as designed and deliver
the intended results. And if not, that its methods be changed to be more easily meet the
goals of collecting representative clinical surveillance data. If the goals are assessed to be
unachievable, then serious consideration should be given to either eliminate MMP or
redesign it with a more workable methodology that produces useable data.
Application of surveillance data for prevention program development and evaluation
- The panel discussed the need for tools to ensure that surveillance data are utilized
appropriately to align resources and interventions for the areas of greatest burden or areas,
populations, or groups that are disproportionately affected, within CDC and at the
jurisdiction-level. The materials provided for the review cited a resource allocation model
under development to assist with optimal allocation of funds. CDC personnel clarified that
such a model is not currently available for jurisdiction-level planning processes.
5. To what extent does DHAP have adequate capacity and sufficient resources devoted to
surveillance, consistent with the Division’s mission, goals, and priorities?
- Overall the panel members concurred that DHAP does not currently have adequate capacity or
resources devoted to core surveillance activities, which not only have dramatically increased in
volume and complexity, but also serve as the foundation upon which prevention and care
programs are based. As described in the responses to the other sections, the reviewers urge
CDC to consider the demands on and the deliverables of the surveillance system activities,
present and future.
- The reviewers recognized the value of each of the activities in the current surveillance
portfolio but asserted that strengthening the national HIV reporting system (Core
Surveillance) and ensuring rigorous national and local analysis capacity should be priority
activities when resources are limited.
- The method of determining allocations across surveillance programs was not well
understood by the reviewers. The panel recommends assessing funding levels across
jurisdictions to ensure appropriate parity.
- CDC staffing and capacity did not seem at all sufficient to ensure meaningful support to or
monitoring of sites. Some CDC epidemiologists providing support are new to the Branch
and do not have enough familiarity with the complexities and intricacies of HIV
surveillance activities. Provision of in-depth technical assistance and trainings—such as the
beneficial trainings and meetings related to incidence estimation—is putting a strain on
CDC program personnel resources. Lower morbidity areas in particular may require
additional technical assistance for data analysis that CDC staff may be unable to provide.
- The needs for technical assistance are great and continue to increase, whether it be ensuring
the effective use of surveillance data for programs; implementing HIV Partner Services
collaborations; transitioning data management systems to more complicated SQL platforms;
implementing and managing electronic reporting from laboratories, facilities, and other
reporting systems; implementing, managing, and analyzing genotype/phenotype data in
particular; changing the surveillance system with advances in testing technologies and
interventions; or ensuring nation-wide achievement of performance standards.
- Comprehensive laboratory reporting in all jurisdictions demands technical expertise for
establishing and maintaining rigorous local systems, but the benefits are significant locally
and nationally. Such reporting offers complete and timely data for: monitoring trends in
newly diagnosed HIV infection and prevalence of HIV infections; entry, continuity, and
retention in care; and timely referral of cases for partner services and linkage to care and
- Information Technology resources within CDC do not seem sufficient as evidenced by
long-standing corrections needed for eHARS (e.g. perinatal, HIV-2, others). Greater
coordination of IT solutions is needed across HIV surveillance components, prevention
components, and perhaps even the prevention and surveillance of other diseases, such as
STD, hepatitis, and TB.
6. To what extent is the mix and balance of surveillance activities relevant to the current
While the panelists did not specifically address the question of the “mix and balance” of current
surveillance activities, over the course of the review various recommendations emerged that
speak to this issue:
- CDC’s efforts to support rigorous, complete, core HIV surveillance systems at the
jurisdiction level is the fundamental activity required to ensuring appropriately balanced
and representative surveillance activities.
- Over the past several years, CDC has moved towards incorporating incidence and resistance
surveillance activities into the “core” surveillance processes and systems. With the
anticipated advances in testing technologies and clinical practices, incorporation of these
components into “core” activities seems reasonable but should be evaluated as the incidence
and resistance systems mature.
- Panelists recognized the need for and value of behavioral and clinical monitoring
components as a part of national HIV surveillance efforts, but asserted that these activities
should not supplant core surveillance infrastructure. It may be that the core surveillance
system, which now includes longitudinal clinical data, may be able to provide some of the
information needed for clinical surveillance. Behavioral data may also be able to be
appropriately monitored through more effective coordination with other prevention and
disease surveillance programs.
7. What are your recommendations for changes (e.g., enhancements, additions, activities
that can be de-emphasized), future directions, and priorities for the program?
Many recommendations for changes, enhancements, de-emphasis, future directions and
priorities are addressed in the preceding questions of the Surveillance Panel Summary as well
as in the brief panel summary in section IIB of the report. The recommendations not already
addressed primarily relate to the need for greater efficiencies and collaboration within DHAP
and across CDC programs:
- Closer integration and collaboration between the prevention, research, evaluation, and
surveillance programs is needed to eliminate unnecessary duplication of efforts and to
maximize the use of available data to advance prevention efforts.
- Greater coordination between DHAP prevention and surveillance programs could help to
ensure jurisdiction-level collaboration with key shared activities and performance measures.
- Across CDC’s disease surveillance programs, crosscutting issues such as poor reporting
from the Veteran’s Health Administration facilities might be more effectively addressed as
a coordinated effort.
- The new HIV Partner Services guidelines represent a critical milestone in recognizing the
valuable prevention opportunities that can occur when surveillance supports public health
action. Many barriers to implementation remain, however. While technical challenges exist,
the greater challenge will likely prove to be in the navigation of organizational “cultures”
and approaches. The extent to which CDC actively holds all programs and grantees
accountable to similar security and confidentiality standards will directly impact how
quickly programs move towards collaboration and integration of shared functions.
Top of Page
C. Biomedical Interventions, Diagnostics, Laboratory, and Health Service Research
Topics to be addressed:
- Biomedical interventions
- Diagnostics and testing
- Laboratory research
- Health services research (including cost-effectiveness)
- Research-to-program translation and dissemination
The Biomedical Panel had an opportunity to review the current status of research conducted on
biomedical interventions, such as the development of new assays, surveillance of acute HIV
infection, and the use of various research methodologies. The Panel was presented information on
current research conducted by the various branches, both internally and externally, and reviewed
the data gathered from research and was able to question how the priorities for research are
determined within DHAP and possible synergies with other CDC components.
1. To what extent are the biomedical, laboratory, and health services research projects and
resulting data of high scientific quality?
- The panel was clear that they believed the projects and resulting data were of the highest
scientific quality. They affirmed that the investigators were also of the highest quality.
2. To what extent are the objectives of each biomedical, laboratory, and health services
research project adequately described and clearly linked to and consistent with the
Division mission, goals, and priorities?
- The panel noted DHAP’s unique role was in translational research, helping to disseminate
the findings of clinical trials into public health practice.
- The panel discussed that the process for setting priorities, and the criteria used, including
level of effort and research required was unclear. This included the following specific
o Pre-Exposure Prophylaxis (PrEP), microbicides, vaccine research
o Activities performed in consultation with other DHAP branches and external partners
- Resistance testing (done in collaboration with industry)
- The panel questioned the process by which Branches determine what activities they carry
out and who makes the final decision. Whereas priorities are determined at the Division
level, branch chiefs can determine if the activities will be carried out and these may or may
not match the Division level priorities.
- The panel questioned whether some legacy programs could be reprogrammed to other more
- The panel discussed whether DHAP’s goals needed to be expanded so that CDC can be
more agile and flexible in responding to the epidemic. They discussed the role of the CDC
Washington, D.C. Office.
- There was discussion between allowing the CDC to conduct investigator-driven research
versus selecting research based on cost effectiveness.
- The panel discussed CDC’s role in the development of diagnostics and research on incident
and prevalent infections. Though this research has been historical and informative, it is not
always unique to CDC and is occasionally investigator driven.
- The panel discussed CDC’s role in finding diagnostics to determine incidence, which will
be helpful in determining program effectiveness.
- The panel felt that early detection is secondary to primary prevention, but clearly
acknowledged that incidence data are important to target the work to the need.
- The panel discussed both the challenges and occasional necessity of combining behavioral
and biomedical intervention development. They recognized that behavioral and biomedical
research appears to occur in separate tracts and that little effectiveness research exists that
studies the impact of combining behavioral with biomedical interventions. They observed
that CDC would be ideally poised to conduct such research.
- The panel recommended the development of more innovative study designs, and better
integration of behavioral and biomedical interventions as appropriate.
3. To what extent do the biomedical, laboratory, and health services research activities
adequately meet the needs of CDC’s constituencies?
- As discussed above, the panel observed a poor correlation between the prioritization of
research activities and broader prioritization processes within DHAP.
- The panel discussed the significant cultural and morbidity differences between white and
black MSM and questioned whether or not adequate research had been conducted that took
into account these differences.
- The panel questioned whether or not research has adequately addressed women who are
infected while pregnant or how pregnant women who are infected became pregnant.
- Again, in order to better align research with DHAP priorities and goals including prioritized
populations, the panel recommended establishing both an internal and an external scientific
review panel (or process for periodic review) to assess existing and future projects so that
only programs that are aligned with the mission and goals of DHAP are funded and
4. To what extent are the biomedical, laboratory, and health services research products
adequately evaluated and the results effectively disseminated?
- The panel questioned how data is validated and how research models can be applied to the
- The panel questioned how data are disseminated to decision makers. Though this is
commonly done via published papers or Morbidity and Mortality Weekly Reports
(MMWRs), the panel recognized that as a public health agency, this was not always the
most rapid of mechanisms by which data could be shared. The recent HIV incidence figures
were discussed as an example.
- The panel noted that CDC could do more to strengthen the communication of CDCapproved
recommendations and guidelines.
- The panel questioned how the implementation of CDC guidelines is evaluated once they are
released to the public. The panel noted that releasing guidelines is not enough to effectively
disseminate new information. The CDC needs to also evaluate whether their
recommendations are being carried out, and if they aren’t, what societal or legal factors
need to be addressed or changed first to ease the implementation of the guidelines. In
addition, the panel felt that guidelines need to be evaluated to determine whether they are
realistic given the current health system infrastructure.
- The panel recommended that DHAP review its process for sharing data and guidelines to
assure that it meets the needs of its constituents.
5. To what extent does DHAP have adequate capacity and sufficient resources devoted to
biomedical, laboratory, and health services research consistent with the Division’s
mission, goals and priorities?
- The panel discussed the impact of other entities, most importantly, the Global AIDS
Program (GAP), on staffing levels. It was clear that many excellent DHAP staff had
relocated to GAP, draining DHAP of personnel resources.
- Specifically, the laboratory branch has lost several key staff to GAP and as a result have
lost some capacity to carry out branch functions.
- The panel discussed the utility of cross-branch workgroups. The HIV Testing in Medical
Settings workgroup was noted as effective in coordinating efforts across multiple branches.
- The panel questioned the prioritization of some biomedical interventions including PrEP,
microbicides, circumcision, and vaccines. The panel recognized the CDC’s scientific
contribution to these fields, but questioned whether there might be opportunity for greater
collaboration with other partners including GAP, industry, and the National Institutes of
- The panel discussed the importance of health services and cost-effectiveness research in the
CDC scientific portfolio and the relative lack of prioritization these areas seem to be given.
They discussed the few staff devoted to this type of research.
- The panel recommended that the DHAP Laboratory Branch and the GAP laboratory engage
in more extensive collaborative activities in order to increase efficiencies and avoid
- The panel suggested DHAP explore expanding their health services and cost-effectiveness
research portfolios by bringing in additional scientists and experts in these fields.
6. Are the Laboratory Branch’s diagnostics reference functions appropriate in scope,
consistent with DHAP’s mission and goals, and adequately staffed?
- The panel noted the importance of the Laboratory Branch’s role in developing diagnostic
assays for monitoring acute and recent HIV infection and confirming HIV infection.
- The panel discussed the need to develop an assay to detect non-clade B infections, which
have been surging lately.
- The panel indicated that DHAP must be flexible enough to address changing diagnostic
demands and that the research budget for biomedical interventions is not sufficient.
- The panel recommended that the Laboratory Branch continue to play a role in developing
diagnostic assays and that this should be done in discussion with other federal agencies and
industry that may be doing the same.
- The panel recommended that the Branch continue to serve as a reference laboratory.
- The panel recommended that DHAP engage external partners in an ongoing basis to ensure
that they remain vigilant to the trajectory of diagnostic research.
7. Do DHAP’s biomedical, laboratory, and health services research activities reflect the
appropriate mix of basic and applied research, and to what extent is the mix and balance
of activities relevant to the current epidemic?
- As discussed previously, the panel observed an occasional disconnect between the research
portfolio and DHAPs broader priorities. In addition, it was not clear how these broader
priorities helped to inform the research agenda.
8. To what extent does the biomedical, laboratory, and health services research portfolio
adequately advance the current state of science?
- The panel was clear that they believed the projects and resulting data was of the highest
scientific quality. They affirmed that the investigators were also of the highest quality.
- The panel noted DHAP’s unique role was in translational research, helping to disseminate
the findings of clinical trials into public health practice.
- The panel discussed the important role of the Laboratory Branch in conducting animal
model research because it is a niche no one else can currently fill.
- The panel emphasized the importance of matching the scientific agenda with the current
state of the epidemic.
9. What are your recommendations for changes (e.g., enhancements, additions, activities
that can be de-emphasized), future directions, and priorities for the biomedical,
laboratory, and health services research programs?
- The panel clearly recommended a review of the scientific portfolio as it relates to the
priorities and goals of DHAP and the current state of the HIV epidemic as reflected in
surveillance, programmatic, health services, and cost-effectiveness research.
- The panel suggested that this review be recurrent, or a process, to allow for reevaluation
of legacy projects and ongoing research.
- The panel suggested that the review take into account research being performed by
external partners including GAP, industry, and NIH.
- The panel reiterated its recommendation that DHAP collaborate more with GAP, industry,
and with NIH to find synergies.
Top of Page
D. Behavioral, Social, And Structural Interventions Research
The Behavioral Panel was able to review information on the current status of behavioral and
structural intervention research, the different types of research the CDC carries out, and the
interaction of expressed need for new and innovative behavioral interventions with the programs
being implemented at the national, state, and local levels. The panel also reviewed the scientific
quality and appropriateness of research being conducted and the process for determining the subject
and scope of research being conducted.
Topics to be addressed:
- Behavioral, social, and structural interventions
- Communications and social marketing research
- Operational research (including cost-effectiveness)
- Linkage to care, retention in care, ART adherence
- Research-to-program translation and dissemination
1. To what extent is the behavioral research portfolio of high scientific quality?
- The panel questioned the definition of science and the purpose of DHAP’s activities, which
is to reduce incidence and prevalence not improve science. Science should have an ultimate
programmatic outcome in the areas of capacity, technical assistance, science of training, and
quality improvement and assurance. The panel agreed that DHAP should focus on applied
research and real world issues, but questioned whether their current resources allowed for
this type of paradigm shift.
- The panel recommended an expanded definition of behaviors to also include organizations,
individuals, and social groups.
- The panel questioned whether DHAP’s focus on populations and demographics of race,
gender, and sexual orientation translates effectively into intervention designs.
2. To what extent are the objectives of the behavioral research portfolio adequately
described, clearly linked to, and consistent with the Division’s mission, goals, and
- The panel emphasized that research efforts should reflect science and philosophy and
engage in innovative methodologies and give more emphasis to practice-based research.
- The panel came to consensus that while it remains important to acknowledge the disparate
impact of HIV on specific populations, institutionalizing the response by population group
has not proven to be effective and has had some unintended consequences. In particular,
attention has been shifted from the social and individual practices conferring vulnerability
and influences that increase the risk of HIV transmission to compartmentalizing individuals
based on race/ethnicity; sex/gender, which generally is not appropriate or effective for
- The panel recommended that DHAP engage exclusively in Phase 1 and Phase 4 research
activities, as those are the ones that can be more completely carried out within DHAP’s
existing structure, and the current research model is not adequate. The Capacity Building
Branch, given support in the development of study design, can carry out some research.
- The panel questioned if a logic model for developing social marketing campaigns exists.
For example, when questioning CDC staff about why one program is considered and
another is not, there was no clear answer provided.
- The panel noted that the theoretical models that drive program development,
implementation and evaluation do not reflect a unified conceptualization of behavior
change. For instance, in three projects presented to a group that focused on increasing HIV
testing, six different models were used.
- The panel questioned CDC’s focus on medication adherence and whether this was an
appropriate topic for CDC’s prevention research portfolio. It was pointed out that other federal agencies (National Institute of Mental Health, HRSA) may have asked CDC for
assistance in dissemination of adherence interventions, and further information is needed to
better understand the rationale for a focus on antiretroviral therapy (ART) adherence.
3. To what extent do the behavioral research findings/products meet the needs of CDC’s
constituencies and inform programmatic efforts?
- The panel recommended that research take into account the needs at the community level
and bridge research to practice with community-centered models. The panel emphasized
that in communities there is no demographic distinction of individuals and questioned the
meaningfulness of research conducted along strict demographic lines to CBOs.
- The panel recommended DHAP review the process for translating research findings into
programs for implementation by CBOs. The analysis should include the cost of
implementation and barriers commonly reported to implementation, including staff turnover
- The panel encouraged CDC to work with health departments and CBOs in operational
research to ensure that evidence-based and data-supported linkage to care strategies are
used in conjunction with HIV testing. The goal is to make sure that persons newly
diagnosed with HIV and persons not connected to care are linked and retained in care.
4. To what extent are the prevention interventions and strategies within our research
portfolio adequately evaluated (adequacy of research methods) and effectively
- The panel discussed structural interventions and their role in the DHAP research agenda,
which should come secondary to operational effectiveness research and is closer to DHAP’s
role as an implementer of public health programs.
5. To what extent does DHAP have adequate capacity and sufficient resources devoted to
behavioral research, consistent with the Division’s mission, goals, and priorities, and
consistent with the mission of our other federal partners?
- The panel recommended that DHAP work with the NIH to address research gaps and avoid
duplication of effort, including knowledge about treatment and adherence, and it should be
DHAP’s role to serve as conduit to health departments and communities.
- The panel emphasized the need for collaboration among the research and program branches.
6. To what extent are the strategies, and the mixture and balance of these strategies, within
the behavioral research portfolio appropriate and relevant to the current epidemic (with
respect to various strategies or tactics, various populations or risk groups, and individual,
social, structural, or other risk determinants)?
- The panel recommended that research projects be done in collaboration with CBOs funded
to implement programs and that DHAP’s approach to gathering best practices be more
- The panel observed that currently DHAP’s operations research is focused on adaptation,
effectiveness/replication studies, and improving Effective Behavioral Intervention (EBI)
implementation. While these activities are appropriate, the panel recommended a
redefinition of applied research in DHAP that maximizes opportunities to solve the real world
problems of dissemination and implementation. The adaptation process, for example,
is not the subject of current research. The panel recommended that the research focus on the
processes of a dissemination science that emphasizes scientific studies of the training
procedures, end-user response to guidance, implementation strategies, contract manager
effectiveness, enhancers and barriers to scale up, evidence in the real world for
effectiveness, and capacity building. This paradigm shift is needed to conduct
organizational and system-level research to evaluate the ultimate effectiveness and cost effectiveness
of interventions as implemented by real-world service providers. The panel
also recommended a community-driven research agenda at the CDC that takes advantage of
CDC’s unique relationships with CBOs and health departments and encourages input in the
identification of potentially effective “home-grown” interventions.
7. To what extent does the behavioral research portfolio advance the current state of
- The panel stated that DHAP has a measurable impact among emergency rooms and health
departments, but the link between science and activities implemented is not clearly stated,
and priorities are not specified.
- The panel stated that DHAP’s research focuses on HIV-testing strategies and behavioral
interventions, which does not appear guided by research findings. Therefore, linkages
between research and program and how research findings impact incidence and prevalence
are not apparent.
- The panel discussed the impact of conducting population-based research. They stated that
focusing on populations resulted in research being directed at populations regardless of
epidemiological burden. The panel emphasized that research should be based on the science
and not other considerations. Research based on behaviors would be more realistic and
would be easier to disseminate as it is what the practitioners are seeking.
- The panel stated DHAP should consider the literature about what is happening globally to
guide the creation of models for the United States.
- The panel recommended the use of social marketing theory in addition to the use of
traditional behavior change theory when developing programs and interventions.
8. To what extent is the research translation model effective and efficient in moving proven
interventions and strategies into practice?
- The panel recommended that a social-marketing approach should be integrated into more
behavioral research activities. The use of this approach in developing public communication
programs is commendable, but researchers need to shift focus from individual-level
behaviors to more social and structures issues (e.g., from testing to stigmas, access to
service). They also recommended that health departments and CBOs be linked into
campaign and program development early on to assure that local roll-outs and
implementation capacities are most effectively and efficiently deployed. Social marketing also has a central role to play in scaling up intervention dissemination and translation of
research to practice.
- The panel stated that the findings of research can be disseminated faster.
9. What are your recommendations for changes (e.g., enhancements, additions, activities
that can be de-emphasized), future directions, and priorities for the behavioral research
- The panel recommended that CDC should engage in behavioral and social research—
formative (Phase 1) and operations/effectiveness (Phase 4)—that target practices and
behaviors that fuel HIV epidemics across populations and settings; and CDC should work
with CBOs and health departments to determine how best to focus and tailor these
responses in the particular communities. (For example, at the individual level—research on
programs and interventions that address concurrent sexual partnerships, unprotected anal
intercourse, non-disclosure of HIV status. At the societal level—research on CBO and
health department capacity to deliver HIV prevention services [including, but not limited to
EBIs]; scale-up of access to and uptake of male and female condoms, etc.)
- The panel recommended that DHAP review data on epidemiology and varying levels of risk
at the individual level and racial and ethnic disparities to develop and implement program
approaches, interventions, and public health strategies.
- The panel recommended the use of more extensive applied research methodologies to
analyze the implementation of interventions in real-world conditions and the actual cost of
- The panel recommended that research be expanded to include mass communication
(especially agenda setting) and sociological theories and models (scaling social innovations,
diffusion of innovations) to better capture both the complexities of the challenges and also
illuminate new solutions and innovative approaches to craft them.
Top of Page
E. Prevention Programs, Capacity Building, and Program Evaluation
The Program Panel reviewed information on HIV-prevention programs funded at the national,
state, local, and community-based levels, including primary prevention, behavioral interventions
(DEBIs), public health strategies, capacity-building activities and public information campaigns.
The Panel was able to review the types of programs and activities being implemented, monitoring
and evaluation data currently available, and plans for enhancing the prevention programs in the
Topics to be addressed:
- Community planning
- Counseling and testing
- HIV Screening
- Partner services
- Prevention interventions (behavioral, structural, biomedical)
- Communications and social marketing
- Capacity building
- Program evaluation
1. To what extent are the Division’s prevention programs, capacity-building activities, and
program monitoring and evaluation approaches evidence-based, of high scientific quality,
and consistent with the current state of HIV prevention science?
- DHAP lacks a mechanism for programs to explicitly interface with
science and integrate various sources of data (e.g., surveillance,
research agenda, evaluation) to ensure bilateral priority setting
- DHAP should strongly promote evidence-based approaches that work, including syringeaccess
interventions, the safe disposal of syringes, comprehensive sex education, and
condom availability and effectiveness.
- We do not know whether the national collection of HIV intervention are “working,”
addressing populations at greatest risk or reducing health disparities, because evaluation
data are not yet available.
- The Program Evaluation Monitoring System (PEMS) used now does not meet our needs.
We are stuck with legacy elements of PEMS (i.e., the information technology infrastructure
and the inability to accept and analyze data and to generate reports).
- The panel recommends that PEMS be replaced. Until a new data system is implemented,
CDC needs to continue to work with state and local partners on collecting data through
- The committee acknowledges the strengths of DEBIs in the prevention portfolio; however,
DEBIs are not in and of themselves the sum total of an effective, comprehensive prevention
portfolio. There are shortcomings that need to be addressed.
- There needs to be an increase in DEBIs targeted to gay men and gay men of color, so as
to limit adaptation and translation. There needs to be a synthesis of core concepts of
- There needs to be increased attention to and resources for the use of novel technologies
for prevention interventions, especially technologies used by gay and bisexual men to
- CDC should continue to focus on the effectiveness and cost effectiveness of its
- CDC needs to balance the limited population reach of DEBIs with the need to scale up
to reach larger populations. To the extent possible, CDC should link broad-scale
activities with greater reach to more specialized behavioral interventions for higher-risk
- CDC should continue to support the evaluation of “locally grown” interventions through
expertise and resources.
- The prevention communication group should review and apply the science of strategic
message framing and risk communication.
- CDC should be concerned about the effectiveness and cost effectiveness of their programs.
- CDC should provide tools and technical assistance to allow for health departments to
calculate their incidence estimates using the back-calculation methodology and to
implement behavioral surveillance.
2. To what extent are the objectives of the Division’s prevention program, capacity building,
and program monitoring and evaluation portfolios adequately described, clearly linked
to, and consistent with the Division mission, goals, and priorities?
- They are not described. CDC needs to communicate its frameworks and priorities more
succinctly. We need a more comprehensive presentation of DHAP’s portfolio.
- There needs to be a mapping and modeling of interventions, programs, and resources to
more clearly describe how the portfolio will meet the goals. Without such mapping and
modeling, the determination of the mission appropriateness and linkage (goals, etc.) cannot
- DHAP and NCHHSTP should revisit their mission and goals, particularly in the context of
the social determinants of health discussion. There needs to be more specificity, targeting
and prioritization of activities to address these overarching risk determinants.
3. To what extent do the Division’s prevention program, capacity-building, and program
monitoring and evaluation activities meet the needs of CDC’s constituencies (e.g., training
and technical assistance activities sufficiently address growing and evolving needs of
public health workforce; program monitoring and evaluation activities result in sufficient
accountability, least burden to constituents, and best utilization of data to inform
- The panel defined constituencies as CBOs, national organizations, health departments,
policy makers, communities, any grantee, and the general public (re: testing as well as
- Evaluation data needs of local areas are not being met. CDC should start with needs
assessment at the local level then go up rather than in the opposite direction. (See evaluation
comments throughout). To the extent possible, CDC should move beyond process
monitoring (aka PEMS) to increase program capacity for quality assurance and outcome
- The panel recommended an increase in cultural competency of capacity building and
technical assistance by implementing technical assistance programming more flexibly.
Currently there is a need for more culturally matched and locally appropriate technical
- The panel recommended giving CDC funded organizations the opportunity to select from
identified providers of capacity building and technical assistance or allow these providers to
identify their own providers of technical assistance. To implement this now, CDC should
allow CBA providers to use their grant resources to purchase the needed services from
organizations that can provide needed and culturally competent services (if CBA providers
cannot do so).
- Regarding the public health workforce, the panel recommended increasing the focus of the
capacity-building program on workforce retention, public health core competencies, and
capacity. These efforts should include all levels of the public health workforce (community
based, health departments, and CDC).
- CDC should ensure through recommendations and funding that research institutions funded
by CDC collaborate and disseminate information and findings with health departments and
4. To what extent are the Division’s prevention program, capacity-building, and program
monitoring and evaluation activities adequately evaluated, and to what extent are the
interventions and tools within the capacity-building portfolio effectively disseminated?
- While the panel recognizes the importance of this question, it could not respond to this
question due to lack of data.
- DHAP should initiate a comprehensive, ongoing, and proactive review and evaluation of
the portfolio and dissemination.
- DHAP needs to strengthen mechanisms to share lessons learned from evaluation activities
and successful practices.
5. To what extent does DHAP have adequate capacity and sufficient resources devoted to
prevention programs, capacity building, and program monitoring and evaluation,
consistent with the Division mission, goals, and priorities?
- The panel concurs with CDC’s professional judgment and its budget request to Congress;
however, we cannot be entirely sure whether this budget will assure the capacity to meet
- CDC should maintain the programmatic priorities irrespective of budget increase and align
the budget with these priorities.
- The panel recommends an increase in full-time equivalents (FTEs) and funding for
programs and evaluation. (See other evaluation comments throughout the report).
- There should be a dedicated proportion of resources reserved for core functions such as
evaluation. CDC should review other agency models for examples.
- The panel acknowledges the need to halt some activities throughout DHAP, such as legacy
projects that are no longer appropriate.
- CDC should redirect discretionary resources (FTEs and funding) and scientific capacity
toward the prevention programs.
6. To what extent is the mix and balance of prevention program, capacity building, and
program monitoring and evaluation activities appropriate and relevant to the current
- There is a need for a combination of effective approaches in the fight against HIV. DHAP
should develop a comprehensive prevention portfolio, and it should include structural
interventions reflecting a social-determinants-of-health framework and biomedical
- The intervention mix does not adequately meet the needs of gay and bisexual men of all
race/ethnicities or all MSM.
- Surveillance/Data Collection does not currently represent the epidemic with the current
surveillance variables (e.g. MSM versus gay-identified men; non U.S. born individuals).
- The panel recommends updating the social marketing portfolio to represent the epidemic,
the state of program science, and the needs of populations. Not all social marketing
materials should be translated, but many should be developed in native languages.
- Establish a well-defined, well-described portfolio and associate with it an evaluation
framework and data sources that help monitor implementation and measure effectiveness.
7. To what extent are the current nine essential components of comprehensive state and
local HIV prevention programs still appropriate, based upon current surveillance,
research, laboratory, epidemiological, and program evaluation data?
- The panel recommended a redefinition of what are the essential elements of a
Comprehensive HIV Prevention Program, which are defined in the Supplemental Guidance
for HIV Prevention Community Planning, to incorporate activities currently carried out by
non-governmental organizations at the national and community-based level.
8. To what extent does HIV prevention community planning effectively inform
programmatic efforts and meet the needs of constituents, affected populations and other
- The panel recognized the importance of community planning and its ability to increase
accountability, transparency, community participation, community leadership capacity, and
- The panel also acknowledged the high and disproportionate level of resources allocated to
community planning in many jurisdictions and the need to balance the valued process with
the need for local flexibility and appropriate resource allocation to these functions
- CDC should explore new, flexible, and more resource-efficient models for maintaining
community input, accountability, and transparency. The guidance should allow for a variety
of models that assure these elements (input, accountability and transparency).
- CDC needs to use the HIV Prevention Leadership Summit (HPLS) as a means of
reinvigorating the energy around community planning, providing the opportunity for
information and innovation transfer among jurisdictions, and developing community
planning leadership. CDC should also explore regional meeting opportunities.
9. To what extent do the Division’s HIV prevention programs address the goals of reducing
health disparities; supporting program collaboration; and facilitating integration of HIV,
STD, viral hepatitis, and TB services at the client level?
- Establish a prioritized effort to examine the impact of discrimination on health outcomes.
(For example, there is a need to recognize the centrality of gay and bisexual men in the HIV
epidemic who face tremendous health inequalities. There is a need for a lesbian, gay,
bisexual, and transgender [LGBT] focus and an office to examine the general health of
LGBT people; particularly if we move toward a social-determinants-of-health model and
take seriously the impact of homophobia and gender bias, among other factors, on health
- The panel supports the prioritization of focus on blacks, African Americans, Latinos, Asian
and Pacific Islander communities, Native Americans and immigrant populations who face
tremendous health disparities.
- Regarding Program Collaboration/Integration, the panel recommends increasing
collaboration/interface between surveillance, evaluation, and prevention programs to ensure
bidirectional learning and planning.
- The panel recommends an increase in cross-Center and collaboration between health
departments and HHS Program Collaboration and Service Integration (PCSI) and consistent
promotion of collaboration among programs who receive funding from multiple federal
sources (CDC, HRSA, NIH, etc.)
- The panel recommends increased flexibility in cooperative agreements (e.g. 10 percent
flexibility of use allowance across CDC Cooperative Agreements) to allow program and
service integration in order to meet the needs of the local epidemics (e.g. HIV, TB, STD,
- The panel recommends increased resources for intra-agency support and collaboration,
more FTEs across the board, but if the programs collaborated better, then maybe they could
tap into each other’s resources. And surveillance should be part of the process as well.
- There is an opportunity for coordination/collaboration across CDC programs in the Center
to collaborate on social determinants of health. As HIV moves toward biomedical
interventions, the panel expressed hope that we can learn from successes and failures of the
other disease areas (STDs, hepatitis, etc.)
- DHAP should look for opportunities to integrate HIV into the context of other programs
(school health, social services, substance abuse services job corps, etc). CDC should
develop a common language and public health framework across programs.
10. To what extent are effective mechanisms in place to ensure that programmatic needs and
perspectives are informing the Division’s surveillance and research agenda?
- Mechanisms are not in place (or if they are, they are not adequate). CDC needs to assure
program-science interface for the setting of the scientific agenda and to assure translation of
science into interventions.
- CDC should establish a practice/program community review of the research portfolio on an
ongoing basis. The review should examine research priorities and portfolio.
- CDC needs to increase operational, translational, and optimization research to define the
optimal mix of prevention programs.
- See above comments for program collaboration and need for mechanism to assure
bidirectional exchange between research and programs.
11. What are your recommendations for changes (e.g., enhancements, additions, activities
that can be de-emphasized), future directions, and priorities for the prevention program,
capacity building, and program monitoring and evaluation portfolios?
a) Move toward social determinants of health framework.
- Social determinants of health should at least include: poverty, racism, homophobia,
incarceration (and mass imprisonment), homelessness, substance abuse, immigration,
and power inequities.
- This broadened approach should extend throughout the DHAP portfolio and should
become part of CDC’s conversation in the national health reform discussion.
- CDC should seek additional consultation about moving toward this framework.
b) Recognize the centrality of gay and bisexual men in the HIV
- Given that the majority of cases of HIV remain among gay and bisexual men of all races
and ethnicities, DHAP should refocus and emphasize gay, bisexual, MSM, and
transgender people of all race and ethnicities facing tremendous health inequities in
communications, research, and programs.
- DHAP should determine if the disparities framework will be expanded to include MSM
or if a complementary inequities framework is more appropriate to facilitate goal setting
and use of resources.
- DHAP should prioritize gay men and differentiate between gay men and the broader
category of MSM.
c) Strategically allocate DHAP resources, and improve DHAP’s organization and
functioning in order to respond to the epidemic, establish broad partnerships, and
- Given the changes afoot including health reform and the development of a national
AIDS strategy, DHAP should help to create a framework for resource distribution,
which reflects the epidemic, recognizing the need for base funding for performing core
functions. DHAP should utilize the range of data for assessing the need and capacity.
- CDC should identify more useful ways to learn from information about what is
happening in the epidemic (interventions, epidemiology, etc.) at national and local
- CDC should streamline external and internal clearance and procurement processes to
reduce barriers to prevention program implementation (e.g. OMB and internal CDC
- DHAP should increase organizational flexibility to implement innovation and change
and consider organizational restructuring to ensure better program collaboration.
- CDC should develop integrated models that facilitate effective integration among the
centers and their funded programs.
d) Design, fund and implement a more strategic evaluation approach to facilitate
learning across the organization and across a network of providers, to inform
prevention and capacity-building needs, and to address the information needs of
- DHAP should distinguish which data should be collected locally and which nationally.
- CDC should orient evaluation toward both process and outcome.
- CDC should prioritize the establishment of a different and more flexible prevention
evaluation data system.
- There must be close collaboration with state and local partners in the solution for the
interim and long term.
e) Reclaim prevention science, and use it actively to inform policy and program.
- CDC must provide scientifically accurate, culturally appropriate information and tools
to the public (e.g., syringe exchange, condom use, promotion, comprehensive sex
education). We need to evaluate the impact of the Helms Amendment on the provision
of information and programs to our partners, particularly if we are prioritizing sexual
- Because the science of syringe exchange in reducing HIV is broad and compelling,
CDC should request a lifting of the ban on the use of federal funds for syringe
- CDC should assert the efficacy of correct and consistent condom use.
- CDC should fully support (in program and funding), comprehensive sex education as
part of an effective HIV prevention strategy.
- Social marketing and health communications should reach specific populations at
greatest risk for HIV as well as stakeholders.
f) The framework of “essential elements,” while historically focused on health
department cooperative agreements, should now describe the entire CDC HIV
portfolio, not just what is done with and through health departments.
- The “Nine Essential Elements” need to be revisited, updated, and expanded to include
the modes of operation and spheres of influence in which HIV prevention operates.
- CDC should convene a consultation to examine the elements.
Top of Page
Go to Section VI