Skip directly to local search Skip directly to A to Z list Skip directly to navigation Skip directly to site content Skip directly to page options
CDC Home

Unit B: Assessing Program Performance

Immunization Program Operations Manual (IPOM)

Printer friendly version [21 pages]

Assess program performance for program improvement.

 

Overview

Assessing program performance is an essential organization practice that requires use of multiple evidence based data sources. Data sources available to immunization programs include, but are not limited to, vaccine preventable disease (VPD) surveillance systems, vaccination coverage surveys, school vaccination coverage and exemption reporting, Immunization Information Systems (IIS), program evaluation results, administrative data, and vaccine purchase data.

As vaccination services become more widely available, the data sources should also expand beyond traditional health care providers to include complimentary vaccinators, such as pharmacy settings, visiting nurses, work sites, school-located vaccination clinics, and other non-medical settings. Specific program goals and needs guide which data source(s) are used to develop a long-term strategic plan together with a comprehensive annual immunization program plan. Since program goals and needs may change, drastically or subtly, based on factors such as outbreaks or available resources, assessment and evaluation can be a dynamic process that may continually feed back into the planning process. Ideally, this is done on a routine basis, driven by specific program milestones. In actual practice, assessment and evaluation activities are not implemented consistently across programs or integrated into the day-to-day management activities of most programs. However, if implemented routinely, using available data for decision making and incorporating the information into plans can lead to outcome-oriented and accountable actions which can improve how public health activities are planned and managed, especially in resource-limited settings.

The activities and requirements described in this chapter are designed to help immunization programs develop clear plans that will use available data sources, involve partnerships, and include feedback systems that allow continual learning and ongoing improvement to occur. It is expected that assessing program performance will involve a diverse group of public health professionals and partners and will be an ongoing, routine process to inform awardee management and improve effectiveness.

With regard to Required Objective B-3, assessing provider coverage levels and other measures of provider quality, it is important to keep in mind that there are multiple ways to offer ongoing "feedback" about a provider’s coverage levels and other measures (such as missed opportunities) when an IIS is available to generate such data. When that is the case and following a full AFIX visit, the assessment component of AFIX can be generated by the IIS, and the feedback component of AFIX can be shared with providers on a regular basis (some awardees do this quarterly) by means of a simple report that could also include strategies for addressing missed opportunities. Receiving such information regularly serves to remind providers that the IIS is a useful tool and enables them to monitor their progress in improving coverage levels and decreasing missed opportunities.

References are listed after each objective to help immunization programs implement the suggested activities and meet the required objectives.

Awardees must identify activities that will describe HOW the following required objectives will be accomplished. Suggested activities are provided for each objective, but awardees are strongly encouraged to think strategically about the specific activities that will best help them achieve this goal.

Top of Page

Required Awardee Objective: B-1

Annually create, understand, and assess awardee-specific tables of CDC-acquired coverage data from NIS, NIS-Teen, NIS-Influenza, BRFSS, as well as IIS and other program-acquired local and state level data for the purpose of identifying and addressing areas of low or lagging vaccination coverage in children, adolescents and adults.

 

a. Develop annual plan for steps to be taken for improvement.

Suggested Activity:

  1. Review available data sources, and develop a written, five-year immunization program strategic plan designed to improve immunization program performance, focusing on low or lagging vaccination coverage among children, adolescents, adults, and special populations relevant to the awardee jurisdiction.

Suggested Reporting Element:

  • Submit a program strategic plan that covers the five-year funding cycle with a description of the activities to be completed during the budget year of the award cycle.

Suggested Performance Measure:

  • Annual program strategic plan submitted.

Allowable Funding Sources:

317 FA Operations, VFC Operations

 

b. Improve and sustain immunization coverage levels.

Suggested Activities:

  1. Identify the priority, achievable, and affordable activities that the best available evidence indicates will improve and sustain vaccination coverage levels among children, adolescents, and adults in the next year. Activities should focus on low and lagging vaccination coverage levels for specific populations and newer vaccines or vaccines targeted for new populations. For example, specific populations for adults may include groups such as pregnant women, health care personnel, long-term care residents, other high-risk groups based on medical condition (e.g., diabetes, asthma, HIV infection) or behavior/lifestyle (e.g., men who have sex with men, persons seeking care in STD clinics). Include the rationale for choosing the priority, achievable, and affordable activities over other possible activities that may improve vaccination coverage levels.
  2. Conduct, implement, and monitor the outcome of one or more of the prioritized activities.
  3. Monitor and report on legislative changes that may impact vaccination of patients or the participation of providers in vaccination services. Some examples include, but are not limited to:
    • Requirements for vaccination of patients in long-term care and other health care settings and for vaccination of health care personnel.
    • Changes in which vaccines and patient age groups for which pharmacists and other complementary vaccine providers are allowed by law to provide vaccination services.
    • Changes in immunization information system requirements, such as the types of providers and age groups of patients for whom data may be entered and/or accessed.

Suggested Reporting Elements:

  • Submit a prioritized list of activities to be implemented during the five-year funding cycle, with evidence-based justifications that support the choices, which will improve and sustain vaccination coverage levels during the budget year of the award cycle. This can be included in the annual immunization program plan as described in B-1a.
  • Submit an activity implementation plan and proposed outcome-oriented performance measures. This can be included in the annual immunization program plan as described in B-1a.
  • Report legislative changes which impact vaccination of patients or the participation of providers in delivering vaccination services.

Suggested Performance Measures (target to be defined by awardee):

  • Prioritized list of activities and evidence-based justifications submitted.
  • Activity implementation plan and proposed outcome-oriented performance measures submitted.

Allowable Funding Sources:

317 FA Operations, VFC Operations

 

c. Reduce or eliminate coverage disparities by race, ethnicity, and socioeconomic status.

Suggested Activities:

  1. Use the best available data to identify coverage disparities by race, ethnicity, and socioeconomic status among children, adolescents, adults, and special populations relevant to the awardee’s jurisdiction.
  2. Once the coverage disparities are identified, specify the priority, achievable, and affordable activities for each subgroup that the best available evidence indicates will reduce or eliminate the vaccination coverage disparities in the next year. Include the rationale for choosing the priority, achievable, and affordable activities over other possible activities that may reduce or eliminate disparities.
  3. Conduct, implement, and monitor the outcome of one or more of the prioritized activities.

Suggested Reporting Elements:

  • Submit a prioritized list of activities to be implemented during the five-year funding cycle with evidence-based justifications that support choice of activities, which will reduce or eliminate coverage disparities by race, ethnicity, and socioeconomic status during the budget year of the award cycle. This can be included in the annual immunization program strategic plan as described in B-1a.
  • Submit an activity implementation plan and proposed outcome-oriented performance measures. This can be included in the annual immunization program strategic plan as described in B-1a.

Suggested Performance Measures (target to be defined by awardee):

  • Prioritized list of activities with evidence-based justifications submitted. This can be included in the annual immunization program strategic plan as described in B-1a.
  • Activity implementation plan and proposed outcome-oriented performance measures submitted.

Allowable Funding Sources:

317 FA Operations, VFC Operations

 

d. Promote evidence-based strategies.

Suggested Activities:

  1. Specifically identify the priority, achievable, and affordable evidence-based strategies that will be used in the next year to support activities listed in B-1b and B-1c. Describe the best relevant data used to support the decision(s) to use these strategies in the awardee’s jurisdiction, focusing on low and lagging vaccination coverage levels for specific populations (e.g., at-risk adults and newer vaccines, such as HPV). Evidence-based strategies can be taken from "The Guide to Community Preventive Services". If a strategy is proposed that is not part of the "The Guide to Community Preventive Services," please provide the evidence used to support the effectiveness of this strategy.

Suggested Reporting Elements:

  • Submit a prioritized list of evidence-based strategies with justifications that support those strategies, which will improve and sustain immunization coverage levels (B-1b) and reduce or eliminate coverage disparities by race, ethnicity, and socioeconomic status (B-1c) during the budget year of the award cycle. This can be included in the annual immunization program plan as described in B-1a.

Suggested Performance Measures (target to be defined by awardee):

  • Prioritized list of evidence-based strategies with justification submitted.

Allowable Funding Sources:

317 FA Operations, VFC Operations, VFC/AFIX

 

e. Promote vaccination through key partnerships.

Suggested Activities:

  1. Conduct or facilitate presentations, seminars, workshops and in-service trainings on immunization-related topics for public and private health care professionals in collaboration with physician, nurse, hospital, pharmacy, health care system, and public health professional organizations. Training topics and resources should include:
    • Resources for education and training available through CDC.
    • CDC distance learning opportunities, such as the CDC satellite broadcasts, net conferences, onsite courses, DVDs, and online training on vaccine-preventable disease and immunization issues.
    • Collaboration with local provider organizations, such as the American Academy of Pediatrics (AAP), the American Academy of Family Physicians (AAFP), the American College of Physicians (ACP), and the American Pharmacists Association (APhA) to use endorsed peer professionals to conduct practice-based seminars on immunization basics for private providers.
    • Workshops for public and private VFC providers to provide instruction on vaccine storage and handling techniques, VFC program eligibility and screening for children and adolescents, and required VFC vaccine management reports.
  2. Review community assessments, or if unavailable, conduct community assessments to determine and define populations at high risk in the community.
  3. Based on populations identified through various data sources as high risk in the community, identify other governmental organizations or agencies with which to collaborate and partner. Appropriate collaborations may include WIC, STD programs, Indian Health Services, programs serving the elderly or other high-risk populations. Based on assessment, work with these agencies to:
    • Collaborate to link immunization messages in other health programs and events.
    • Ensure that immunization messages are consistent.
    • Link other health programs and events to immunization events, such as National Infant Immunization Week (NIIW) and National Influenza Vaccination Week (NIVW).
    • Link immunization training and education to appropriate staff in related programs serving high-risk populations.
  4. Based on populations identified through various data sources as high risk in the community, determine which non-governmental organizations, such as local business, education, health care, faith-based, and community organizations (interested in promoting the health of the community) with which to collaborate and partner. Through these organizations, as appropriate:
    • Establish or participate in local coalitions and programs.
    • Identify influential community leaders as spokespersons on immunization issues.
    • Collaborate with consumer and advocacy groups to develop and disseminate consumer information about vaccines for targeted high-risk groups.
    • Develop and disseminate immunization information and educational materials for distribution to patients and consumers.
    • Develop and promote health care provider and consumer education programs.
    • Distribute and offer immunization information to immunization providers and to provider organizations through social media, online notices, automatic updates, special mailings, newsletters, communicable disease bulletins, websites and email list services.

Suggested Reporting Elements:

  • Number of educational items and trainings developed and delivered on immunization-related issues.
  • Number of assessments reviewed or conducted to define populations at high risk.
  • Number of partnerships and collaborations developed and/or activities initiated with other governmental organizations and agencies to reach populations at high risk.
  • Number of partnerships and collaborations developed and/or activities initiated with non-governmental entities to promote immunizations.

Suggested Performance Measures (target to be defined by awardee):

  • Prioritized list of evidence-based strategies and partnerships developed based on outcome of needs assessment.

Allowable Funding Sources:

317 FA Operations, VFC Operations

 

f. Support communications to enhance informed vaccine decision-making for consumers, health care providers, and policy makers.

Suggested Activities:

  1. At each health care provider visit, distribute Vaccine Information Statements (VIS) and CDC’s online instructions and communicate their importance and proper use in accordance with the National Childhood Vaccine Injury Act (section 2126 of the Public Health Service Act, 42 U.S.C. Section 300aa-26). See Appendix A.
  2. To ensure that immunization program staff can best communicate immunization best practices to all partners, provide orientation for newly-hired immunization program staff on how to use the IPOM and where to find relevant CDC and other education, training and communication resources for immunization staff, health care providers, consumers, and community partners. See Appendix B.
  3. Utilize the Guide to Community Preventive Services ("Community Guide") as a primary resource for evidence-based recommendations and findings about strategies that work to improve public health. Incorporate findings into all education, training, communication, and community programs reaching health care providers, immunization program staff, consumers, and the community at large. See Appendix C.
  4. Assess provider education needs and identify potential resources to meet those needs. Observation of practice, discussion, surveys, and other methods such as pre- and post-tests may be used. Based on assessments:
    • Disseminate information about the Vaccine Adverse Event Reporting System (VAERS), the surveillance system for reporting adverse events, and the reporting process.
    • Ensure that providers are informed, either directly or through provider organizations, of current and new vaccines covered by the National Childhood Vaccine Injury Act and of the federal requirement for record keeping. Advise providers to subscribe to automatic updates.
    • Disseminate information on legal requirements and procedures for reporting of vaccine-preventable diseases to state/local health departments.
    • Promote CDC’s provider portal with health care providers.
    • Promote CDC’s Immunization Health Care Provider toolkit.
    • Direct providers to CDC’s website for parent information for parents with vaccine concerns.
    • Promote provider-based strategies outlined in the Community Guide to improve immunization practices and coverage.
    • Distribute and promote the Revised Standards for Child and Adolescent Immunization Practices and Revised Standards for Adult Immunization Practices to all public and private immunization providers.
      See Appendix D
      .
  5. Regularly update all immunization clinical staff on appropriate and timely education and training on immunization-related issues by:
    • Ensuring that immunization program staff members view the Epidemiology and Prevention of Vaccine-Preventable Diseases program or the annual update, available as a DVD or web-on-demand product.
  6. Provide policy makers, consumers, and the general public timely education on immunization-related information and changes by:
    • Maintaining accurate and up-to-date websites on vaccine-preventable disease and immunizations.
    • Utilizing a variety of communications channels, including social media and marketing, to provider information.
    • Developing education tools on general and specific immunization information in accordance with the program’s overall strategy based on needs to assure immunization for all ages.
    • Providing updates to consumer groups, legislators, and special interest groups regarding vaccine-preventable diseases, new vaccines, targeted "at-risk" populations, and immunization recommendations and schedules. When appropriate, provide information on evidenced-based immunization interventions, such as laws and regulations and standing orders affecting specific child, adolescent, and adult subpopulations.
    • Disseminating information about the Vaccine Adverse Event Reporting System (VAERS), the surveillance system for reporting adverse events, and the reporting process to all concerned.
  7. Develop communications, messages, campaigns, and programs based on assessed needs of the community and high-risk populations. Based on needs:
    • Utilize CDC resources to promote special events and planned activities such as National Infant Immunization Week (NIIW) and National Influenza Vaccination Week (NIVW).
    • Develop and maintain relationships with public information officers, and work jointly on getting media attention for immunization-related news.
    • Identify and train media spokespeople.
    • Design and implement (when resources are available) qualitative and quantitative audience research and message testing to better understand your target audiences’ knowledge, attitudes, behaviors, and vaccine confidence levels. When research resources are not available, consider using CDC or other published data. Monitor state and local media to assess communication messages and trends in messaging.
    • Develop a system, including the use of social media and marketing, to provide timely updated information as necessary on new vaccines, schedule changes and administration changes.

Suggested Reporting Elements:

  • Percent of providers who are using VIS correctly (as assessed through VFC site visits).
  • Percent of the program’s new immunization staff who complete the orientation within one month of being hired.
  • Number of proven strategies from the Community Guide that are promoted within the community.
  • Percentage of providers whose needs were assessed every other year.
  • Programs developed and promoted matched the assessed needs of health care providers.
  • Percentage of clinical staff reviewing the Epidemiology and Prevention of Vaccine-Preventable Diseases program or update.
  • Number of immunization-related materials and other items developed and delivered to policy makers, consumers, and the general public.
  • Number and type of developed and delivered immunization-related communications programs to consumers and the general public.

Suggested Performance Measures (target to be defined by awardee):

  • Prioritized list of communications needs and activities with data justifications submitted.

Allowable Funding Sources:

317 FA Operations, VFC Operations, VFC/AFIX

Top of Page

Required Awardee Objective: B-2

Report school-enterer coverage annually to CDC using CDC-approved reporting methods or by consulting with CDC staff to identify acceptable alternatives.

 

Required Activities:

  1. Assess vaccination coverage and exemption levels on each vaccine appropriate for kindergarten entry according to state requirements, using a CDC-approved survey methodology, and to the extent possible, assess coverage and exemption levels on each vaccine according to ACIP recommendations.
  2. Track school vaccination and exemption requirements.
  3. Provide feedback to schools, school districts, and state Departments of Education on kindergarten vaccination coverage and exemption rates by school, and work with these partners to improve compliance with state requirements.

Suggested Activities:

  1. Assess vaccination coverage and exemption levels on each vaccine appropriate for middle school entry according to state requirements, using a CDC-approved survey methodology, and to the extent possible, assess coverage and exemption levels on each vaccine according to ACIP recommendations.
  2. Provide feedback to schools, school districts, and state Departments of Education on middle school vaccination coverage and exemption rates by school, and work with these partners to improve compliance with state requirements and ACIP recommendations.

Reporting Requirements:

  • Annually submit to CDC the annual school assessment report:
    • Kindergarten assessment methods and vaccination coverage and exemption data, with coverage and exemptions reported for public and private school separately.
    • Changes in school vaccination and exemption requirements annually.
    • Summary of feedback provided to schools, school districts, and state Departments of Education on kindergarten vaccination coverage and exemption rates by school, and work completed with these partners to improve compliance with state requirements.

Required Performance Measures:

  • Annual School Assessment report submitted by April 30 with required information included on kindergarten coverage and exemptions; changes in school vaccination and exemption requirements; summary of feedback to schools, school districts, and state Departments of Education on kindergarten vaccination coverage and exemption rates by school; and work completed with these partners to improve compliance with state requirements.

Suggested Performance Measures (target to be defined by awardee):

  • Percentage of kindergarten and middle school children who have received each recommended vaccine, measured separately for public and private schools.
  • Percentage of kindergarten and middle school children entering school who were exempt from one or more recommended vaccines, measured separately for public and private schools.
  • Percentage of kindergarten and middle school children entering school who were exempt from each recommended vaccine, measured separately for public and private schools.

Allowable funding sources:

317 FA Operations

Top of Page

Required Awardee Objective: B-3

Work with VFC providers on quality improvement processes to increase coverage levels and decrease missed opportunities using AFIX components, as appropriate, and move toward use of IIS as primary source of data for provider coverage level assessment by the end of the project period.

 

Suggested Activities:

  1. Conduct AFIX site visits to at least 25% of VFC enrolled providers.
  2. Update and maintain AFIX program standards using CDC’s AFIX Policies and Procedures Manual as a reference guide.
  3. Ensure that AFIX visits and all provider contacts are in compliance with CDC’s program standards and requirements provided in the AFIX Policies and Procedures Manual.
  4. Initiate educational campaigns with VFC providers to inform them about the change in AFIX assessment protocol.
  5. Ensure that all AFIX staff that conduct AFIX visits have "refresher" training annually.
  6. Ensure that AFIX Coordinators participate in AFIX quarterly calls and CDC supported trainings for AFIX and AFIX/IIS.
  7. Develop plans to implement the use of AFIX reporting tools provided for conducting and documenting site visit information (PAPA online tool, CoCASA version 9, and the AFIX Site Visit Questionnaire). The AFIX timeline provided by CDC to be used as a reference document when developing those plans.
    See Appendix E.

Required Performance Measures:

  • Annual submission of the AFIX Annual Report.
  • Annual submission of the AFIX policies and procedures manual.

Suggested Performance Measures (target to be defined by awardees):

  • Percentage of eligible providers receiving an AFIX visit. ("Eligible" providers include all VFC-enrolled sites or a sub-group of VFC-enrolled sites. The criteria to select the sub-group will be defined by the awardee and may include components such as size of practice, nature of practice, etc.)
  • Percentage of providers who received AFIX follow-up within six months of the AFIX visit.
  • Percentage of assessments conducted using IIS data.
  • Percentage of provider sites that implemented follow-up action items based on feedback.
  • Immunization coverage level results.

Allowable funding sources:

317 FA Operations, VFC Operations, VFC/AFIX

Top of Page

Required Awardee Objective: B-4

Assure adherence to CDC’s Manual for the Surveillance of Vaccine-preventable Diseases, including Vaccine Adverse Events Reporting.

 

a. Conduct vaccine-preventable disease surveillance and case investigation.

Required Activities:

  1. Implement and maintain surveillance systems to investigate and document cases and outbreaks of vaccine-preventable diseases (VPDs). Investigate cases that are reportable in the jurisdiction, implement appropriate public health activities, and ensure that appropriate clinical specimens are tested and relevant epidemiologic information is collected for Congenital Rubella Syndrome (CRS), diphtheria, Haemophilus influenzae, measles, meningococcal disease, mumps, pertussis, polio, invasive pneumococcal disease, rubella, tetanus, pediatric (< 18 years of age) influenza deaths, hepatitis A, hepatitis B, and varicella.

Suggested Activities:

  1. Monitor percent of case reports with complete information submitted to CDC within one month of diagnosis.
  2. Develop administrative policies and procedures to assure the systematic, institutionalized reporting of cases and suspected cases of VPDs by providers, health care institutions, schools, child care centers, universities, and laboratories.
  3. Obtain the authority for health department staff to review medical records of persons who are cases or suspected cases of all VPDs.
  4. Ensure availability of written, up-to-date guidelines for case investigation, outbreak investigation, and outbreak control of all VPDs. In program areas where an immunization information system (IIS) is operational, incorporate the use of the IIS into case investigation guidelines when documenting the vaccination status of cases and controls.
  5. Ensure that health department staff responsible for VPD surveillance and response is trained to perform VPD case and suspect case investigations, outbreak investigations, and outbreak control.
  6. Initiate VPD case investigations and outbreak investigations promptly and implement outbreak control measures and other necessary public health interventions in a timely manner.
  7. Ensure that up-to-date laboratory guidelines are developed and distributed for each VPD, detailing the appropriate clinical specimens for testing, the recommended laboratory tests and laboratories, appropriate specimen handling, and expected timelines for laboratory results. Ensure that laboratory capacity is available for testing clinical specimens for VPDs.
  8. Assure that laboratories report all results which indicate need for public health action, as described in CDC’s "Manual for Surveillance of Vaccine-preventable Diseases." Ensure that clinics, schools, child care facilities, hospitals, and other VPD reporting sites routinely submit surveillance reports to health departments.
  9. Conduct enhanced, active surveillance in communities where a VPD is prevalent, or where there is increased incidence.
  10. Pursue unreported cases of VPDs by searching laboratory, hospital and/or death certificate data, especially for VPDs which are rare or uncommon.
  11. Investigate outbreaks occurring in schools, child care and institutional facilities, and offer control efforts either through provision of vaccine in public clinics or by referrals to primary health care providers. In outbreak settings, obtain clinical specimens from at least one case for laboratory confirmation.
  12. Analyze, review, and interpret surveillance data regularly and outbreak data as needed.
  13. Disseminate surveillance morbidity and mortality data regularly to surveillance partners, network participants, providers, policy makers, and the public.

Reporting Requirements:

  • Annual Progress Report (Disease Surveillance)

Required Performance Measures:

  • Submission of Annual Progress Report (Disease Surveillance)

 

b. Submit case and/or death notifications to CDC.

Required Activities:

  1. For routine surveillance, collaborate with appropriate staff to submit timely and complete electronic case and/or death notifications to CDC for cases that are reportable in the jurisdiction. Notify CDC about cases immediately by phone and electronically transmit complete case reports and supplemental surveillance information to CDC via the National Notifiable Diseases Surveillance System (NNDSS) within one month of diagnosis for CRS, diphtheria, measles, polio, rubella, and pediatric (<18 years of age) influenza deaths. Collect and electronically transmit complete case reports and supplemental surveillance information to CDC via NNDSS within one month of diagnosis for Haemophilus influenzae, meningococcal disease, mumps, pertussis, invasive pneumococcal disease, tetanus, hepatitis A, hepatitis B, and varicella.

Suggested Activities:

  1. Monitor percent of case reports with complete information submitted electronically to CDC within one month of diagnosis.
  2. Ensure that appropriate procedures exist and are followed for accurately entering, analyzing, and reporting surveillance, investigation, and outbreak data in a timely fashion.
  3. Ensure that appropriate procedures and informatics functionalities exist to electronically transmit case notifications to CDC in an accurate, complete, and timely fashion.
  4. Analyze, review, and interpret surveillance data regularly and outbreak data as needed.
  5. Disseminate surveillance morbidity and mortality data regularly to surveillance partners, network participants, providers, policy makers, and the public.

Reporting Requirements:

  • Case notifications must be provided to CDC through NNDSS.
  • Annual Progress Report (Disease Surveillance)

Required Performance Measures:

  • Case notifications provided to CDC through NNDSS.
  • Submission of Annual Progress Report (Disease Surveillance)

 

c. Assess timeliness and completeness of case/death investigation, reporting and notification.

Required Activities:

  1. Evaluate timeliness and completeness of each case/death investigation, reporting and notification for cases of VPDs that are reportable in the jurisdiction. Monitor the quality of VPD surveillance by reviewing surveillance data and surveillance indicators to identify problems and strategies to resolve the problems.

Suggested Activities:

  1. Assess the proportion of Haemophilus influenzae invasive disease cases among children <5 years of age with complete vaccination history, and implement activities to ensure completeness of data.
  2. Assess the proportion of Haemophilus influenzae isolates from cases <5 years of age that were serotyped, and implement activities to ensure appropriate case investigation and completeness of data.
  3. Assess the proportion of measles cases with complete vaccination history, and implement activities to ensure completeness of data.
  4. Assess the proportion of measles cases or chains of transmission that have an imported source, and implement activities to ensure appropriate case investigation and completeness of data.
  5. Assess the proportion of meningococcal cases with complete vaccination history, and implement activities to ensure completeness of data.
  6. Assess the proportion of meningococcal cases with known serogroup, and implement activities to ensure completeness of data.
  7. Assess the proportion of mumps cases for which appropriate clinical specimens were obtained and submitted to the laboratory, and implement activities to ensure appropriate case investigation and completeness of data.
  8. Assess the proportion of mumps cases with complete vaccination history, and implement activities to ensure completeness of data.
  9. Assess the proportion of pertussis cases from which clinical specimens are obtained, and implement activities to ensure appropriate case investigation and completeness of data.
  10. Assess the proportion of probable and confirmed pertussis cases meeting the clinical case definition that are laboratory confirmed, and implement activities to ensure appropriate case investigation and completeness of data.
  11. Assess the proportion of cases confirmed by isolation of B. pertussis by culture, and implement activities to ensure appropriate case investigation and completeness of data.
  12. Assess the proportion of probable and confirmed pertussis cases with a complete vaccination history, and implement activities to ensure completeness of data.
  13. Assess the proportion of pneumococcal invasive disease cases among children <5 years of age with complete vaccination history, and implement activities to ensure completeness of data.
  14. Assess the proportion of pneumococcal isolates from cases of invasive disease <5 years of age that are serotyped and tested for antibiotic resistance, and implement activities to ensure appropriate case investigation and completeness of data.
  15. Assess the proportion of confirmed rubella cases among women of child-bearing age with known pregnancy status, and implement activities to ensure completeness of data.
  16. Assess the proportion of confirmed rubella cases that are laboratory confirmed, and implement activities to ensure appropriate case investigation and completeness of data.
  17. Assess the percentage of varicella cases with complete information for age, vaccination history, and severity of disease, and implement activities to ensure completeness of data.
  18. Monitor the quality of surveillance by applying surveillance indicator concepts to additional VPDs.
  19. Monitor the quality of additional aspects of VPD surveillance by applying other surveillance indicators, as described in CDC’s "Manual for Surveillance of Vaccine-Preventable Diseases."

Reporting Requirements:

  • Annual Report (Disease Surveillance)

Suggested Performance Measures (target to be defined by awardee):

  • Percent of case reports with complete information submitted to CDC within one month of diagnosis.
  • Percent of case reports with complete information submitted electronically to CDC within one month of diagnosis.
  • Proportion of Haemophilus influenzae invasive disease cases among children <5 years of age with complete vaccination history.
  • Proportion of Haemophilus influenzae isolates from cases <5 years of age that were serotyped.
  • Proportion of measles cases with complete vaccination history.
  • Proportion of measles cases or chains of transmission that have an imported source.
  • Proportion of meningococcal cases with complete vaccination history.
  • Proportion of meningococcal cases with known serogroup.
  • Proportion of mumps cases for which appropriate clinical specimens were obtained and submitted to the laboratory.
  • Proportion of mumps cases with complete vaccination history.
  • Proportion of pertussis cases from which clinical specimens are obtained.
  • Proportion of probable and confirmed pertussis cases meeting the clinical case definition that are laboratory confirmed.
  • Proportion of cases confirmed by isolation of B. pertussis by culture.
  • Proportion of probable and confirmed pertussis cases with a complete vaccination history.
  • Proportion of pneumococcal invasive disease cases among children <5 years of age with complete vaccination history.
  • Proportion of pneumococcal isolates from cases of invasive disease <5 years of age that are serotyped and tested for antibiotic resistance.
  • Proportion of confirmed rubella cases among women of child-bearing age with known pregnancy status.
  • Proportion of confirmed rubella cases that are laboratory confirmed.
  • Percentage of varicella cases with complete information for age, vaccination history, and severity of disease.

Allowable Funding Sources:

317 FA Operations

 

d. Designate staff to coordinate VAERS and Vaccine Safety activities.

Required Activities:

  1. Designate a VAERS Coordinator* with overall responsibility for VAERS-related activities:
    • Comply with activities related to VAERS reporting and monitoring as stated in the online Manual for the Surveillance of Vaccine-Preventable Diseases, 5th Edition, 2011. VAERS reports may be submitted directly to VAERS from providers. If this is done, no further action is needed from the VAERS Coordinator under this section (4.1). However, if VAERS reports from the state are submitted through the VAERS Coordinator, then required activities for the VAERS Coordinator include:
      • Registers with Epi-X at epiXhelp@cdc.gov to receive quarterly report summaries of the VAERS reports submitted.
      • Reviews each report for completeness (especially the critical boxes: 3, 4, 7, 8, 10, 11, and 13), obtains any other necessary information, and clarifies any questions about the report.
      • Assigns an identifying immunization project number using the two-letter state postal abbreviation, two- or four-digit representation for year, and the state numbering sequence. For example, the 57th report received in Arizona in 2011 begins with AZ, followed by 11, followed by 057, and should look like this: AZ11057. This number is entered into box 24 of the VAERS report.
      • Sends the original report with the identifying number to VAERS and keeps a copy. As with local reporting, the cases should be forwarded rapidly to VAERS and not sent in a batch. CDC encourages VAERS reports to be submitted through the VAERS online encrypted and secure electronic submission format. Any further correspondence about a report should include the six-digit VAERS ID number, which is assigned by the VAERS system. Reports are entered into the VAERS database under this number. It is also helpful to have the patient’s name and date of birth, if available, to help identify the specific report. VAERS maintains the confidentiality of patients’ personal identifying information, consistent with the requirements of the National Childhood Vaccine Injury Act (NCVIA).
      • Completes the quarterly update report that is sent by VAERS via Epi-X to each VAERS Coordinator. (Although these follow-up requests are sent quarterly, the case reports are scanned upon receipt at VAERS and available to CDC and FDA for review in near real-time upon request.) This report contains a list of all initial reports received during the quarter, by VAERS ID number and VAERS Coordinator project number (called "imm proj. report no." in Box 24 of the VAERS form), and serves as an acknowledgement of those reports. Specific missing or incomplete information for these reports is noted and completed in the appropriate boxes. The quarterly update report also lists reports for which VAERS requests recovery status at 60 days post-vaccination and at one year post-vaccination. The VAERS Coordinator submits to VAERS any requested missing information, as well as follow-up recovery status information for each listed report at 60 days and one year post-vaccination. The VAERS Coordinator may update any other pertinent information about these individuals, such as vaccination information or date of birth. Responses to quarterly report questions can be submitted to VAERS by mail, fax, auto upload tool or email. If sending by email, please do not include patient name, date of birth, or other identifying information, and instead include only VAERS ID numbers.
      • Updates VAERS with any personnel, fax, phone, or address changes. This is done by means of a quarterly e-mail request from VAERS to the state health department.
    • Work with local health departments, public health clinics, providers and other vaccinators to ensure compliance with national and state guidance and policies on reporting vaccine adverse events to VAERS.
      • Establish/maintain protocols and systems to accept and submit reports of vaccine adverse events in accordance with state policies. There is no federal requirement that VAERS reports be submitted through the state VAERS Coordinator (i.e., reports can be submitted directly to VAERS by the preliminary/initial reporter). CDC prefers that health care providers outside the state and local public health system directly report to VAERS to best ensure accuracy of information in the report and facilitate follow-up. However, states may wish to maintain protocols and systems to support this capability.
      • Post information and/or links on the state health department website to federal and state guidance and policies on vaccine adverse event reporting to VAERS and to procedures and resources for reporting to VAERS.
      • Facilitate reporting of adverse events that are legally required to be reported. The National Childhood Vaccine Injury Act (NCVIA) requires health care providers to report any adverse event listed by the manufacturer as a contraindication to further doses of the vaccine; or any adverse even listed in the "VAERS Table of Reportable Events Following Vaccination that occurs within the specific time period after vaccination[5 pages]".
      • Encourage submission of VAERS reports for clinically significant adverse events that are not listed in the Table of Reportable Events.
  2. Designate a Vaccine Safety Coordinator* to collaborate with CDC’s Immunization Safety Office (ISO) on public health preparedness and response issues.

    The Vaccine Safety Coordinator should:
    • Serve as CDC’s main point-of-contact for vaccine safety in the awardee’s jurisdiction.
    • Coordinate with the awardee’s VAERS Coordinator as needed.
    • Alert CDC to vaccine safety concerns in awardee’s jurisdiction and respond to vaccine safety emergencies.
      • Report vaccine safety emergencies and events of concern requiring vaccine safety responses to the CDC State Vaccine Safety Coordinator at the Immunization Safety Office (404-639-8256) or the CDC Emergency Operations Center (770-488-7100).
      • Collaborate with CDC and other partners (e.g., FDA, local health departments, health care facilities, providers) to respond to and investigate reports of serious adverse events in accordance with state health department policy.
    • Become familiar with Vaccine Adverse Event Reporting System (VAERS) and ISO’s vaccine safety monitoring activities.
    • Communicate with ISO staff to facilitate adequate exchange of information, prompt response to vaccine safety emergencies, and optimal risk communication.
    • Participate in meetings, such as conference calls, with CDC ISO staff for updates and information exchange, training, and data interpretation as needed. ISO convenes update meetings.
    • Provide information and updates as needed to other public health officials (e.g., State Epidemiologists and Immunization Program staff).
    • Serve as the state point-of-contact for health care providers and local health departments for vaccine safety-related issues.
    • Provide outreach and education to partners within awardee’s jurisdiction involved in vaccine safety monitoring (e.g., local health departments and the general public).
    • Establish collaborations with other state and local health officials involved in the vaccine program in jurisdiction.
    • Identify and respond to vaccine safety issues of concern in respective jurisdiction.
    • Provide timely communication and communication/information products to public health officials, providers, the general public, and other stakeholders. CDC is available to assist as needed.
    • Develop and implement protocols for responding to and investigating reports of serious adverse events following vaccination.
    • Document the number and types of responses for historical purposes.

Reporting Requirements:

  • Report vaccine adverse events to VAERS as appropriate.
  • Report vaccine safety emergencies and events of significant public health concern to the CDC State Vaccine Safety Coordinator at the ISO (404-639-8256) or to the CDC Emergency Operations Center (770-488-7100). Determination of vaccine safety emergencies and events of significant public health concern is at the discretion of the Vaccine Safety Coordinator and the state/project area department of health.

Suggested Performance Measures (target to be defined by awardee):

  • Name of person (VAERS Coordinator) with overall responsibility for VAERS-related activities.
  • Document capability to receive quarterly VAERS reports from CDC via Epi-X if VAERS reports from the state are submitted via the VAERS Coordinator.
  • Name of person (Vaccine Safety Coordinator) to serve as point-of-contact for CDC to allow a direct line of communication to prepare for and respond to vaccine safety issues and emergencies.

Allowable Funding Sources:

317 FA Operations

Note: * CDC prefers that awardees combine the VAERS Coordinator and Vaccine Safety Coordinator into one position. However, awardees have the option to designate two or more staff to fill these roles.

Top of Page

Required Awardee Objective: B-5

Engage in ongoing evaluation of program activities based on CDC’s guidance.

 

Required Activities:

  1. Review the Program Evaluation presentation and related materials shared during the 2013 VFC in Action trainings conducted by CDC; these items are available within the VFC Documents library in SAMS. These evaluation trainings are based upon CDC’s Program Evaluation Framework which includes the following six steps:
    • Engage stakeholders.
    • Describe the program.
    • Focus the evaluation design.
    • Gather credible evidence.
    • Justify conclusions.
    • Ensure use and lessons learned.
  2. Conduct a Vaccine Storage and Handling program evaluation using CDC’s guidance and Framework for Program Evaluation.

Reporting Requirements:

  • Submit by email, for CDC review, an Evaluation Plan using CDC’s Word template.
    • Submit the updated (based on CDC feedback) Evaluation Plan via PAPA.
  • Submit a final report via PAPA.

Required Performance Measures:

  • Submission of required documents.

Please note - Beginning in 2016, awardees will be required to conduct an evaluation for one of the three following CDC-identified priority areas using CDC’s guidance and Framework for Program Evaluation:

  • Adult Vaccination
  • HPV Vaccination
  • Vaccine Financing and Billing

However, CDC recognized that for some awardees, other program areas may be more relevant to evaluate. Therefore, awardees do have the option to choose another area to evaluate. Awardees choosing an area other than the three listed above must include a strong justification for why another program area is more critical for evaluation.

Allowable funding sources*:

317 FA Operations, VFC Operations, VFC/AFIX

*allowable funding source is dependent on the evaluation topic - please check with CDC project officer.

Top of Page

 

Images and logos on this website which are trademarked/copyrighted or used with permission of the trademark/copyright or logo holder are not in the public domain. These images and logos have been licensed for or used with permission in the materials provided on this website. The materials in the form presented on this website may be used without seeking further permission. Any other use of trademarked/copyrighted images or logos requires permission from the trademark/copyright holder...more

External Web Site Policy This graphic notice means that you are leaving an HHS Web site. For more information, please see the Exit Notification and Disclaimer policy.

 
Contact Us:
  • Centers for Disease Control and Prevention
    1600 Clifton Rd
    Atlanta, GA 30333
  • 800-CDC-INFO
    (800-232-4636)
    TTY: (888) 232-6348
    Contact CDC-INFO
USA.gov: The U.S. Government's Official Web PortalDepartment of Health and Human Services
Centers for Disease Control and Prevention   1600 Clifton Rd. Atlanta, GA 30333, USA
800-CDC-INFO (800-232-4636) TTY: (888) 232-6348 - Contact CDC–INFO
A-Z Index
  1. A
  2. B
  3. C
  4. D
  5. E
  6. F
  7. G
  8. H
  9. I
  10. J
  11. K
  12. L
  13. M
  14. N
  15. O
  16. P
  17. Q
  18. R
  19. S
  20. T
  21. U
  22. V
  23. W
  24. X
  25. Y
  26. Z
  27. #