Evaluation Capacity-Building Plan 2021-2025

Executive Summary

Over the past 15 years, the National Institute for Occupational Safety and Health (NIOSH) has taken steps to integrate program evaluation into its culture. Namely, it has developed, implemented, and continually refined the process by which it evaluates the relevance and impact of its research programs. Having completed 13 of these reviews, Institute evaluators identified five topic areas that NIOSH must continue to cultivate to ensure the relevance and impact of its research in the future.

This NIOSH Evaluation Capacity-Building Plan outlines specific needs within those areas and how NIOSH will address them over the next five years.

This document was designed as a learning agenda, as required of federal departments by the Foundations of Evidence-Based Policy Making Act. Therefore, the Plan includes a learning sheet for each topic area, including key learning questions and activities, as well as steps for addressing each key question. Table 1 outlines the key learning questions and activities for all five topic areas. Because NIOSH will transition to a learning agenda after completing these steps, formatting the Capacity-Building Plan in this way will create a clear a path toward the next significant evaluation step.

Key Learning Questions

Key Learning Activities

Key Learning Questions

How can NIOSH incorporate the collection and documentation of intermediate outcomes (IOs) into its processes?

Key Learning Activities

  1. Assess motivations and barriers to collecting IOs. (Year 1)
  2. Develop further guidance to encourage collection and documentation of IOs. (Year 2)
  3. Raise awareness and educate NIOSH staff about what IOs are, why they are important, and how to document them. (Year 3)

Key Learning Questions

How can NIOSH improve its program evaluation process to obtain useful, feasible recommendations?

How can NIOSH better utilize evaluation findings and recommendations?

Key Learning Activities

  1. Understand why review panels might provide recommendations that are beyond the capacity of NIOSH programs to implement. (Year 1)
  2. Refine the current process for interactions between NIOSH and review panels to increase utilization of panel findings and recommendations. (Year 5)
  3. Identify barriers and motivators at the researcher, program, and NIOSH levels to implementing feasible
    review panel recommendations. (Year 1)
  4. Increase the inclusion of panel review recommendations in funding opportunities and decision-making at all levels of the Institute. (Year 5)

Key Learning Questions

What are the most effective approaches and strategies for communicating NIOSH impact with target audiences?

Key Learning Activities

  1. Complete development and user assessment of impact sheets. (Year 2)
  2. Research ways to effectively assess communication approaches and strategies. (Year 3)
  3. Tap into any ongoing or recently completed audience analysis activities. (Year 3)
  4. Develop an internal educational campaign to increase understanding of audience awareness and communication channels and formats. (Year 4)

Key Learning Questions

What is the long-term strategy for conducting external program reviews at NIOSH?

Key Learning Activities

  1. Develop a better understanding of program review strategies utilized by other federal agencies and research organizations. (Year 1)
  2. Develop a statement of purpose and objectives that guide external program reviews. (Year 2)
  3. Develop a long-term strategy for conducting external reviews of NIOSH programs. (Year 3)

Key Learning Questions

How can NIOSH use translation research to enhance successful adoption, implementation, and sustainment of NIOSH solutions?

Key Learning Activities

  1. Develop a shared understanding of translation research across the Institute. (Year 1)
  2. Build a core group of translation research subject matter experts within NIOSH. (Year 4)
  3. Build translation research into projects by adding translation expertise to the effort. (Year 5)

Continue reading the plan using the buttons below

Page last reviewed: December 17, 2020