Evaluation of NIOSH Programs

NIOSH uses program reviews to evaluate the relevance and impact of its research programs. Most recently, NIOSH contracted independent panels of experts to conduct its reviews. Prior to that, reviews were conducted by the National Academies of Science, Engineering, and Medicine (National Academies).

Independent Panel Reviews

In 2017-2019, NIOSH conducted reviews of five programs by independent panels of experts. Each panel reviewed an evidence package and heard presentations from program staff describing the program’s activities and accomplishments over the past 10 years. Panelists provided individual scores for relevance and impact, using a scale of one to five. They also made recommendations for future activities. The panel chair averaged the scores and compiled the recommendations into a final report. All programs received a score of 8 out of 10 or better.

NIOSH used a modified version of contribution analysis as the evaluation framework. This approach emphasizes demonstrating a program’s contribution to an outcome rather than establishing causal attribution. Learn more about this process as described as a case study in the American Journal of Evaluation

The five programs reviewed were:

• Construction
  • Evidence package – Focuses on silica, musculoskeletal disorders, noise exposure and hearing loss, highway work zones, and fall prevention.
  • Expert panel report
• Emergency Preparedness and Response
  • Evidence package – Focuses on Emergency Responder Health Monitoring and Surveillance (ERHMS); anthrax preparedness and response capabilities; and other emergency preparedness activities and responses.
  • Expert panel report
• Exposure Assessment
  • Evidence package – Focuses on methods development and direct-reading methods and sensors.
  • Expert panel report
• Healthcare and Social Assistance
  • Evidence package – Focuses on safe and healthy workplaces, musculoskeletal disorders, hazardous drugs and other chemicals, and infectious disease transmission.
  • Expert panel report
• Mining
  • Evidence package – Focuses on disaster preparedness and response, ground control, and respirable hazards.
  • Expert panel report

National Academies Reviews

From 2005-2008, the National Academies conducted evaluation studies of eight NIOSH programs:

• Agriculture, Forestry, and Fishing Research Program
• Construction Research Program
• Health Hazard Evaluation Program
• Hearing Loss Research Program
• Mining Research Program
• Personal Protective Technology Research Program
• Respiratory Disease Research Program
• Traumatic Injuries Research Program

To ensure consistency across reviews, NIOSH charged the National Academies in 2005 to develop a review framework. The framework document included standards and methods used in all eight reviews.

NIOSH’s advisory committees reviewed and scored programs progress implementing select recommendations in 2014. See NIOSH Docket 278 for more detail.