Topic Area 5: Long-term strategy for external program review

Although the Institute has made incredible progress in the area of program evaluation, the evaluation process lacks any long-term strategy. For example, NIOSH waits to assess program readiness for evaluation and to select programs for review until it is about to start conducting the reviews. The next logical step in NIOSH’s progression is cementing program evaluations into its culture. Leveraging what NIOSH has learned from its evaluation experiences, and the enthusiasm surrounding the Foundations of Evidence-Based Policy Making Act, the Institute will develop a long-term external program review strategy. This strategy will support program efficiency, relevance, and effectiveness while also striving for a balance between the investment made in program reviews and the benefit gained.

In order to begin developing such a strategy, it is prudent to look toward other federal organizations to learn from their program evaluation experiences. NIOSH will reach out to colleagues at other federal organizations to learn more about their evaluation processes and will examine the literature for ideas. Additionally, it will be valuable to speak to the leaders of the five NIOSH programs that recently underwent review to gain their perspective on important considerations for developing a long-term strategy.

Since the external program reviews will continue to be the most resource-intensive type of evaluation that NIOSH conducts, drafting a purpose statement and objectives for this type of review should be completed next. These will likely influence decisions regarding timing and program selection elsewhere in the strategy. On the basis of the findings from the previous information-gathering step and the Institute’s vision for this type of program review moving forward, the NIOSH Evaluation Forum will draft the purpose statement and objectives by outlining the overall and specific aims of the review, such as its scope.

These aims will subsequently be reviewed by NIOSH program leaders, the NIOSH Leadership Team, and the NIOSH Office of Policy, Planning, and Evaluation (OPPE). Once those items have been finalized, NIOSH will create a small workgroup for the express purpose of developing a long-term strategy for external program reviews. It will consider factors such as the investments/benefits of such reviews, the characteristics of programs that would undergo external impact reviews, the cycle for conducting the reviews, and the preparation required for the reviews. Ideally, the strategy would provide programs with clear expectations for reviews and plenty of time to prepare over a much longer period of time (to improve the connection between planning and evaluation). In addition, it would avoid evaluation of the same research multiple times. The NIOSH Evaluation Forum, program leaders, Leadership Team, OPPE, and Board of Scientific Counselors will review the final draft of the long-term strategy.

Continue reading the plan using the buttons below