Evaluation Capacity-Building Plan 2021-2025

Executive Summary

Over the past 15 years, the National Institute for Occupational Safety and Health (NIOSH) has taken steps to integrate program evaluation into its culture. Namely, it has developed, implemented, and continually refined the process by which it evaluates the relevance and impact of its research programs. Having completed 13 of these reviews, Institute evaluators identified topic areas that NIOSH must continue to cultivate to ensure the relevance and impact of its research in the future.

This NIOSH Evaluation Capacity-Building (ECB) Plan outlines specific needs within those areas and how NIOSH will address them over the five-year period of fiscal years 2021 through 2025.

This document was designed as a learning agenda, as required of federal departments by the Foundations of Evidence-Based Policy Making Act. Therefore, the Plan includes a learning sheet for each topic area, including key learning questions and activities, as well as steps for addressing each key question. Table 1 outlines the key learning questions and activities for all topic areas. Because NIOSH will transition to a learning agenda after completing these steps, formatting the Capacity-Building Plan in this way will create a clear a path toward the next significant evaluation step.

As part of the learning process through implementation of the ECB Plan and external factors such as public health emergencies and legislation, the Institute will update the Plan at the end of each fiscal year.

Continue reading the plan using the buttons below

Key Learning Questions

Key Learning Questions

Key Learning Questions

Key Learning Activities

Key Learning Activities

Key Learning Activities

How can NIOSH incorporate the collection and documentation of intermediate outcomes (IOs) into its processes?

Key Learning Questions

How can NIOSH incorporate the collection and documentation of intermediate outcomes (IOs) into its processes?

  1. Assess motivations and barriers to collecting IOs.
  2. Develop educational and awareness raising materials around IOs.
  3. Raise awareness and educate NIOSH staff about IOs.
Key Learning Activities
  1. Assess motivations and barriers to collecting IOs.
  2. Develop educational and awareness raising materials around IOs.
  3. Raise awareness and educate NIOSH staff about IOs.

How can NIOSH improve its program evaluation process to obtain useful, feasible recommendations?
How can NIOSH better utilize evaluation findings and recommendations?

Key Learning Questions

How can NIOSH improve its program evaluation process to obtain useful, feasible recommendations?
How can NIOSH better utilize evaluation findings and recommendations?

  1. Understand why review panels might provide recommendations that are beyond the capacity of NIOSH programs to implement.
  2. Refine the current process for interactions between NIOSH and review panels to increase utilization of panel findings and recommendations.
  3. Identify barriers and motivators at the researcher, program, and NIOSH levels to implementing feasible
    review panel recommendations.
  4. Increase the inclusion of panel review recommendations in funding opportunities and decision-making at all levels of the Institute.
Key Learning Activities
  1. Understand why review panels might provide recommendations that are beyond the capacity of NIOSH programs to implement.
  2. Refine the current process for interactions between NIOSH and review panels to increase utilization of panel findings and recommendations.
  3. Identify barriers and motivators at the researcher, program, and NIOSH levels to implementing feasible
    review panel recommendations.
  4. Increase the inclusion of panel review recommendations in funding opportunities and decision-making at all levels of the Institute.

What is the long-term strategy for conducting external program reviews at NIOSH?

Key Learning Questions

What is the long-term strategy for conducting external program reviews at NIOSH?

  1. Develop a better understanding of program review strategies utilized by other federal agencies and research organizations.
  2. Develop a statement of purpose and objectives that guide external program reviews.
  3. Develop a long-term strategy for conducting external reviews of NIOSH programs.
Key Learning Activities
  1. Develop a better understanding of program review strategies utilized by other federal agencies and research organizations.
  2. Develop a statement of purpose and objectives that guide external program reviews.
  3. Develop a long-term strategy for conducting external reviews of NIOSH programs.

How can NIOSH use implementation (formerly translation) research to enhance successful adoption, implementation, and sustainment of NIOSH solutions?

Key Learning Questions

How can NIOSH use implementation (formerly translation) research to enhance successful adoption, implementation, and sustainment of NIOSH solutions?

  1. Refine definition and develop supporting materials for implementation (formerly translation) research.
  2. Develop implementation research materials to build capacity within NIOSH.
Key Learning Activities
  1. Refine definition and develop supporting materials for implementation (formerly translation) research.
  2. Develop implementation research materials to build capacity within NIOSH.