Evaluation of NIOSH Programs

NIOSH reinitiated its independent, rigorous program review process in 2017, using the framework developed for the review of eight NIOSH research programs by the National Academies in 2005-2008. These new evaluation studies have the same goal of assessing program relevance and impact through a peer review process. However, based on the lessons learned from the previous eight reviews and advances in evaluation science, NIOSH evaluators are applying a theory-based approach to the evaluations: a modified version of Contribution Analysis.1 The emphasis of this approach is on demonstrating a program’s contribution to an outcome rather than establishing causal attribution.

NIOSH is utilizing a scientific, professional, and engineering support contractor to assemble an independent panel to review each program. NIOSH’s contractor recruits a panel chair for each NIOSH program review, who in turn recruits the remainder of the independent panel of experts. The panel is composed of experts in the content area under review, translation science, and program evaluation. Each panel receives an evidence package that contains information about the program’s activities, products developed, and examples of how others have used those products to improve occupational safety and health over the past 10 years. The evidence packages focus on specific health and safety areas and are not comprehensive of all program activities. Program representatives also provide additional detail to panels during an in-person meeting through presentations and discussion.

Using the information from the evidence package and in-person meeting, panelists provide individual scores for relevance and impact, rationale for those scores, and recommendations. The panel chair averages the scores and synthesizes the rationale and recommendations into a final report. Program leaders write a response, with particular attention to if and how they will actively pursue each panel recommendation at the current time. The evidence packages, review panel reports, and program responses developed to date, are available below.

  1. Downes A, Novicki E, Howard J [2018]. Using the contribution analysis approach to evaluate science impact. Am J Eval April: [Epub ahead of print], https://doi.org/10.1177/1098214018767046external icon
Page last reviewed: July 27, 2020