Evaluation of NIOSH Programs
NIOSH uses program reviews to evaluate the relevance and impact of its research programs. Most recently, NIOSH contracted independent panels of experts to conduct its reviews. Prior to that, reviews were conducted by the National Academies of Science, Engineering, and Medicine (National Academies).
Independent Panel Reviews
In 2017-2019, NIOSH conducted reviews of five programs by independent panels of experts. Each panel reviewed an evidence package and heard presentations from program staff describing the program’s activities and accomplishments over the past 10 years. Panelists provided individual scores for relevance and impact, using a scale of one to five. They also made recommendations for future activities. The panel chair averaged the scores and compiled the recommendations into a final report. All programs received a score of 8 out of 10 or better.
NIOSH used a modified version of contribution analysis as the evaluation framework. This approach emphasizes demonstrating a program’s contribution to an outcome rather than establishing causal attribution. Learn more about this process as described as a case study in the American Journal of Evaluation
The five programs reviewed were:
- Emergency Preparedness and Response
- Exposure Assessment
- Healthcare and Social Assistance
National Academies Reviews
From 2005-2008, the National Academies conducted evaluation studies of eight NIOSH programs:
- Agriculture, Forestry, and Fishing Research Program
- Construction Research Program
- Health Hazard Evaluation Program
- Hearing Loss Research Program
- Mining Research Program
- Personal Protective Technology Research Program
- Respiratory Disease Research Program
- Traumatic Injuries Research Program
To ensure consistency across reviews, NIOSH charged the National Academies in 2005 to develop a review framework. The framework document included standards and methods used in all eight reviews.
NIOSH’s advisory committees reviewed and scored programs progress implementing select recommendations in 2014. See NIOSH Docket 278 for more detail.