On This Page
Once a company has conducted assessment and planning for adult immunization programs, and developed the specific tasks of implementation for these programs, it is time to develop the evaluation plan. This evaluation plan should be in place before any program implementation has begun.
Metrics for worker productivity, health care costs, heath outcomes, and organizational change allow measurement of the beginning (baseline), middle (process), and results (outcome) of workplace health programs. It is not necessary to use all these metrics for evaluating programs. Some information may be difficult or costly to collect, or may not fit the operational structure of a company. These lists are only suggested approaches that may be useful in designing an evaluation plan.
These measures are designed for employee group assessment. They are not intended for examining an individual’s progress over time, which would raise concerns of employee confidentiality. For employer purposes, individual-level measures should be collected anonymously and only reported (typically by a third party administrator) in the aggregate, because the company’s major concerns are overall changes in productivity, health care costs, and employee satisfaction.
In general, data from the previous 12 months will provide sufficient baseline information and can be used in establishing the program goals and objectives in the planning phase, and in assessing progress toward goals in the evaluation phase. Ongoing measurements every 6 to 12 months after programs begin are usually appropriate measurement intervals, but measurement timing should be adapted to the expectations of the specific program.
Vaccine-preventable diseases are infectious diseases that can be prevented by immunization (vaccination). Traditionally, vaccines have been associated with protecting young children, but far too many adults become ill, are disabled, and die each year from diseases that could easily have been prevented by vaccines. Vaccines not only prevent disease in the people who receive them but often create “herd immunity,” meaning that even unvaccinated individuals are at lower risk of disease if most of their community is immunized. Everyone from young adults to older adults can benefit from immunizations.
Among vaccine-preventable diseases in adults, influenza has the greatest impact in the U.S. population.
- An average of 36,000 deaths and over 200,000 hospitalizations associated with influenza occur each year in the United States1-2
- The combination of influenza and pneumonia was the eighth leading cause of death among all persons in the United States in 2005, accounting for 63,000 deaths3
- The overall national economic burden of influenza-attributable illness for adults, age 18 years and above is $83.3 billion. Direct medical costs for influenza in adults totaled $8.7 billion including $4.5 billion for adult hospitalizations resulting from influenza-attributable illness4
- Influenza is also responsible for substantial indirect costs ($6.2 billion annually), mainly from lost productivity. Each year, among adults age 18 to 64 years, 17 million workdays are lost to influenza-related illness4
Worker productivity measures for adult immunization5-8
Healthier employees are less likely to call in sick. Companies can sometime assess sick day use to determine whether health programs are increasing worker productivity.
- Determine the number of sick days per employee due to
vaccine-preventable diseases such as influenza in the past 24 months (or
over two winters since one winter is considered a flu season). These data
can be found in time and attendance systems
- If employees are required to present a physician’s note upon return to work after sick leave, these notes can also be reviewed
- This measure may be less useful if there has been a large increase or decrease in numbers of employees over the past 24 months
- Measurement over 2 seasons is beneficial since there may be variation in the severity and spread of influenza in a given season
- Many people call a number of illnesses the "flu" when many of the illnesses are not influenza. When measuring the baseline number of sick days due to influenza be aware of the fact that "flu" is often not influenza
- Conduct baseline surveys of employee reports of influenza and/or pneumonia episodes in the past 24 months (or 2 flu seasons)
- Determine the costs of worker absenteeism related to vaccine-preventable
diseases such as influenza including costs of replacement workers, costs in
training replacement workers, and loss and delay in productivity
- Determine the absenteeism costs of employees before and after a vaccine-preventable disease program such as an influenza education program or vaccination campaign
- Determine time employees spend during working hours participating in vaccine-preventable disease-related worksite programs
- Re-assess the number of sick days per employee due to vaccine-preventable diseases such as influenza at the first follow-up evaluation. This evaluation should occur at least one year after baseline measures
- Periodic repeats of baseline measures
- Assess changes in the average number of sick days per employee in repeated annual follow-up evaluations
- Assess changes in the time employees spend during working hours participating in vaccine-preventable disease-related worksite programs
- Assess changes in costs from baseline (annually)
- Compare absenteeism costs of employees before and after a vaccine-preventable disease program such as an influenza education program or vaccination campaign
Health care costs measures for adult immunization5-8
In contrast with the worker productivity costs described above, health care costs are measures of the direct medical expenses of providing employee health care and preventive health programs.
- Determine costs and use for health care, such as disability, health care costs, and immunizations for vaccine-preventable diseases such as influenza related illness and disability
- Determine the health care use and costs of employees before and after a vaccine-preventable disease program such as an influenza education program or vaccination campaign
- Periodic repeats of baseline measurements
- Assess changes in health care use and costs from baseline
- Compare health care use and costs of employees before and after a vaccine-preventable disease program such as an influenza education program or vaccination campaign
Health outcomes measures for adult immunization5-8
The effectiveness of adult immunization programs depends on the intensity of program effort and the use of multiple interventions. A rule of thumb is that the more programs implemented together as a package or campaign, the more successful the interventions will be.
- Determine current rates of employee immunization from employee health survey or health risk appraisalAn assessment tool used to evaluate an individual’s health. An HRA could include a health survey or questionnaire (see Employee Health Survey); physical examination, or laboratory tests resulting in a profile of individual health risks often with accompanying advice or strategies to reduce the risks.
- Determine levels of diagnostic and treatment procedures related to vaccine-preventable diseases such as influenza from health care and pharmaceutical claims data
- Determine employee knowledge, attitudes, and beliefs about adult
- Evaluate employees’ current knowledge of the health benefits of adult immunizations
- Measure employee’s knowledge of current adult immunization guidelines
- Assess employee awareness of existing workplace adult immunization programs, policies and benefits; and programs available in the community (e.g., influenza immunization programs available at pharmacies)
- Periodic repeats of baseline measures
- Assess changes in levels of employee immunizations, especially influenza immunizations
- Assess changes in employee knowledge, attitudes, and beliefs about adult
- Measure changes in employee knowledge of current adult immunization health benefits and guidelines
- Assess changes in employee awareness of existing workplace adult immunization programs, policies, and benefits; and community programs
Organizational change measures for adult immunization5-8
Improvement in adult immunization requires ongoing support from employers. New programs can be added over time and evaluated periodically for their effectiveness. For best results, recognition of the benefits of immunization programs should become an inherent part of organizational change and corporate culture.
Measuring organization change is an assessment of company-initiated programs and policies that affect most employees regardless of their health status. These efforts need to be integrated for greatest effectiveness and will require time for full implementation. Regular measures of employee attitudes and program development are key in determining whether new programs are effective or require further adaptation to prevent continuing investment in ineffective efforts.
- Determine workplace barriers to employees’ ability to obtain immunizations
- Assess current workplace adult immunization programs
- List current adult immunization options for employees through
worksite and identify number of employees (i.e., participation) using
each option. Examples:
- Number of immunization programs (e.g., education seminars, on-site influenza immunization clinics) and participation in these programs
- Availability of educational materials on adult immunization
- Number of communications/media campaigns regarding adult immunization
- Number of immunization related policies
- Number of environmental strategies to reduce influenza incidence such as handwashing stations
- Number of partnerships with community resources for adult immunization programs such as pharmacies or local health departments
- Determine costs of current company adult immunization programs such
- Staffing, equipment, and space
- Employee time to participate in worksite adult immunization programs such as influenza during work hours (e.g., education or on-site immunization clinic)
- Conduct survey of employee satisfaction with current workplace supported adult immunization programs
- List current adult immunization options for employees through worksite and identify number of employees (i.e., participation) using each option. Examples:
- Reassess barriers to employee engagement in adult immunization programs
- Document steps taken and progress toward implementing each intervention
- List numeric goals in each form of intervention within a designated
time period (e.g., desired increases in annual employee immunization
rates or decreases in seasonal influenza rates)
- Employee reach (e.g., number of educational pamphlets distributed)
- Employee participation (e.g., number of employees immunized)
- Describe timeline for implementation of each planned intervention (e.g., length of time and timing of tasks to develop, initiate, and conduct a mass campaign)
- Create a baseline budget for new interventions including classes, or organizing an on-site clinic, instructors, classroom space, materials, etc
- Identify opportunities for new partnerships with community groups who provide adult immunization programs (e.g., pharmacies, local health department, or local hospital)
- List numeric goals in each form of intervention within a designated time period (e.g., desired increases in annual employee immunization rates or decreases in seasonal influenza rates)
- Reassess employee satisfaction regarding workplace supported adult immunization programs
- Measure reductions in the number and type of employee barriers to receiving immunizations
- Assess changes in workplace adult immunization programs including
progress in achieving goals and in implementation of each intervention
(e.g., length of time and timing of tasks to develop, initiate, and conduct
a mass campaign)
- Measure changes in the number of adult immunization options for
employees through the worksite and changes in employee participation
using each option before and after the program or campaign. Examples:
- Number of new programs developed and offered to employees and participation in those programs
- Number of new educational materials developed and made available to employees
- Number of new workplace communications/media campaigns including posters, brochures, and signs encouraging frequent handwashing
- Number of new workplace policies regarding adult immunization programs developed and implemented compared to baseline
- Number of new environmental strategies (e.g., handwashing stations) developed and implemented compared to baseline
- Number of new partnerships with community groups who provide adult immunization programs (e.g., pharmacies, local health departments or local hospitals)
- Assess changes in program costs from baseline
- Increases in staffing or equipment needs due to new program offerings
- Changes in employee participation time during work hours (e.g., education or on-site immunization clinic)
- Assess changes in survey responses for employee satisfaction following implementation of workplace supported adult immunization programs and compare with baseline
- Measure changes in the number of adult immunization options for employees through the worksite and changes in employee participation using each option before and after the program or campaign. Examples:
Depending on goal success, evaluate the need to adjust workplace programs.
The assessment tools described in the assessment module include specific questions related to adult immunization.
- Q11; Q12; Q13; Q20d; Q21a,b; Q22; Q23; Q24a,b,d,h,i,l; Q26; Q27; Optional Questions A, B, C, I, J, OO
- Q29; Q30; Q31; Q32; Q33; Optional Questions V, W, Y, Z
- Q39; Q40; Q41; Optional Questions CC, JJ, LL
- The Centers for Disease Control and Prevention's (CDC) Vaccines and Immunizations Statistics and Surveillance website provides a number of reports, articles, and data on immunization coverage, and vaccination surveys. The CDC Flu Activity and Surveillance website provides information on the annual number of influenza cases on a state or national level
- National Business Group on Health’s Issue Brief – Vaccinating Against the Flu: A Business Case [PDF - 160KB] describes a number of strategies that employers can take to protect employees from the flu as well as the benefits and economic burden to the workplace
- Health Risk Appraisals at the Worksite: Basics for HRA Decision Making [PDF - 2.3MB] is a guide developed by the National Business Coalition on Health in collaboration with the Centers for Disease Control and Prevention (CDC) in the selection and use of health risk appraisals in the workplace available for employers
- The CDC Healthy Communities Program developed the Community Health Assessment aNd Group Evaluation (CHANGE) assessment tool to provide communities with a picture of the policy, systems, and environmental change strategies currently in place throughout the community, where gaps exists and facilitate action planning for making improvements. The CHANGE tool address five community sectors including worksites and health indicators related to physical activity, nutrition, tobacco use, chronic disease management, and leadership