Review & Evaluation: National Academies Evaluation
NOTE: Text in this summary was drawn from the National Academies Final Report: Respiratory Diseases Research at NIOSH: Reviews of Research Programs of the National Institute for Occupational Safety and Health (a link to the full report appears below).
The NIOSH RDRP has made essential contributions to preventing occupational respiratory disease. The National research Council has rated the program a 5 out of 5 for relevance, and a 4 out of 5 for impact. To further increase its effectiveness, the program should adequately fund and engage in high priority occupational respiratory disease surveillance, continue to improve its work in support of its strategic goals, and develop sampling and analytic methods that include exposure assessment scientists as an integral part of the program’s activities.
Respiratory diseases caused by exposures to dangerous materials in the workplace have a major adverse effect on worker health, with the NIOSH estimating that deaths from work-related respiratory diseases and cancers account for about 70% of all occupational disease deaths. Respiratory diseases caused by exposures in the workplace include a broad spectrum of adverse health effects, including airway diseases such as asthma, interstitial lung diseases such as silicosis, respiratory infectious diseases such as tuberculosis, and respiratory cancers. These health effects can arise in a wide range of occupational settings and range from mild, reversible conditions to progressive fatal disorders, and can be linked to short-term or long-term exposures. These effects have tremendous implications for worker health and, by extension, the national economy.
The mission of the NIOSH RDRP is to provide leadership for the prevention of work-related respiratory diseases, using a scientific approach to gather and synthesize information, create knowledge, provide recommendations, and deliver products and services to those who can effect prevention.The program has five strategic goals: (1) prevent and reduce work-related airway diseases; (2) prevent and reduce work-related interstitial lung diseases; (3) prevent and reduce work-related infectious respiratory diseases; (4) prevent and reduce work-related respiratory malignancies; and (5) prevent respiratory and other diseases potentially resulting from occupational exposures to nanomaterials.
This report evaluates the NIOSH RDRP with respect to the relevance of its work to improve occupational safety and health, and the impact of the program’s research in reducing work-related hazardous exposures, illnesses, and injuries. It is one of several independent reviews of NIOSH research programs by the National Academies. All of the reviews were based on the same framework, which was created by a parent committee established by the National Academies. That framework document established a scoring system from 1 to 5 for impact and relevance, with 5 being the highest:
Rating of Relevance
- Score of 5: Research is in highest-priority subject areas and highly relevant to improvements in workplace protection; research results in, and NIOSH is engaged in, transfer activities at a significant level (highest rating).
- Score of 4: Research is in high-priority subject area and adequately connected to improvements in workplace protection; research results in, and NIOSH is engaged in, transfer activities.
- Score of 3: Research focuses on lesser priorities and is loosely or only indirectly connected to workplace protection; NIOSH is not significantly involved in transfer activities.
- Score of 2: Research program is not well integrated or well focused on priorities and is not clearly connected to workplace protection and inadequately connected to transfer activities.
- Score of 1: Research in the research program is an ad hoc collection of projects, is not integrated into a program, and is not likely to improve workplace safety or health.
Rating of Impact
- Score of 5: Research program has made a major contribution to worker health and safety on the basis of end outcomes or well-accepted intermediate outcomes.
- Score of 4: Research program has made a moderate contribution on the basis of end outcomes or well-accepted intermediate outcomes; research program generated important new knowledge and is engaged in transfer activities, but well-accepted intermediate outcomes or end outcomes have not been documented.
- Score of 3: Research program activities or outputs are going on and are likely to produce improvements in worker health and safety (with explanation of why not rated higher).
- Score of 2: Research program activities or outputs are going on and may result in new knowledge or technology, but only limited application is expected.
- Score of 1: Research activities and outputs are NOT likely to have any application.
- Score of NA: Impact cannot be assessed; program not mature enough.
Assessment of Research Relevance and Impact
The report assigned the RDRP a score of 5 for relevance because the program’s activities are in the highest-priority subject areas and are highly relevant to improvements in workplace protection. The RDRP is successful in transferring research findings, technologies, and information into workplace practices. This is particularly true for activities related to interstitial lung diseases, airways and infectious diseases, and respiratory cancers. Activities related to some subprograms do not reach this highest level of relevance, but these activities are in important research areas and also make some connections to improvements in workplace protection. Another extremely relevant program focuses on exposure to nanomaterials in the workplace. This program addresses an important need to understand and control engineered nanoparticles and other types of ultrafine particles such as those from welding, diesel engines, and fires.
The report assigned the RDRP a score of 4 for impact because while the program has made major contributions to worker health and safety in many of its programs, some have had a smaller impact or the impact is not clearly observed. For example, the documented decrease in the prevalence of latex sensitization, early decreases (pre-2000) in coal workers’ pneumoconiosis, and decreases in silicosis-related deaths in the mining industry are the result of NIOSH intervention and recommendations, and have had a high impact on protecting worker health. In other cases—for example, spread of infectious diseases such as SARS—it is nearly impossible to disentangle the specific contributions NIOSH has made from those of other contributors in limiting the onset or spread of respiratory diseases and resulting decrease in risk or death to exposed workers. However, the program has played a major role in developing respiratory protection systems, guidance documents, and educational materials which have helped to contain what could have been a larger disaster. After much deliberation on how to weigh the assessments of different subprograms, the committee reached a consensus that the program was clearly better than that called for in a score of 4, but not in sum what the committee would rate a 5. If the committee had been given the option of providing non-integer scores, the score for program impact would have been between 4 and 5.
Emerging Issues and Future Research Opportunities
The report assesses progress by the RDRP in targeting new research relevant to future improvements in workplace protection from occupational respiratory disease and sought to identify significant emerging research areas important to the mission of NIOSH. One general issue that emerged from this analysis is inadequate surveillance for diseases and exposures to toxicants. Surveillance activities are critical to fulfilling the objectives of all the major program goals and assessing the performance of current and future RDRP activities. The report emphasized this as a continuing and emerging issue even though it recognized that the lack of financial and personnel resources and not the lack of awareness or expertise are the major causes of this weakness. The report strongly supports collaborations with industry in disease surveillance and exposure monitoring activities in the workplace, particularly in the face of emerging evidence of a link between exposure and outcome. The report also supports a range of ongoing RDRP activities that proactively address the incidence of future diseases and emerging diseases. These activities include the development and improvement of methods to detect respiratory effects earlier and more accurately, continued research on the molecular mechanisms of workplace-related respiratory diseases, characterization of those agents responsible for respiratory effects, and evaluations of genetic variability that affects worker susceptibility. By reducing and eliminating worker exposure to toxicants and pathogenic organisms, NIOSH research on improving respirators is another essential means to address the threat of respiratory diseases among workers.
Recommendations for Program Improvement
The report makes the following recommendations to help ensure that the RDRP continues to maintain its progress toward protecting workers from respiratory disease. These recommendations are important, despite the high scores given for relevance and impact, which reflect the guidance for ranking established by the framework committee and the committee’s recognition of the financial constraints the RDRP has operated under. The recommendations are prospective and are meant to help support NIOSH in identifying opportunities to improve the relevance and impact of its research portfolio.
Strategic Goal 1: Prevent and Reduce Work-Related Airway Diseases
Improve detection of work-related asthma, work-related fixed obstructive airway disease, and relevant exposures. The RDRP should systematically evaluate whether its work-related asthma programs are being compromised under NIOSH’s new priority-setting approach. Because workplace exposures contribute highly to the likelihood of developing asthma, work in preventing and detecting asthma has the potential to greatly improve occupational safety and health among the U.S. workforce. In addition, RDRP should continue to support population-based studies of associations between occupational exposures and chronic obstructive airway disease.
Strategic Goal 2: Prevent and Reduce Work-Related Interstitial Lung Diseases
Continue and expand efforts to prevent coal workers’ pneumoconiosis (CWP), silicosis, fiber-induced interstitial lung disease, and chronic beryllium disease. The activities related to interstitial lung diseases form a critical core of the RDRP and have provided well-documented improvements in occupational health. It is important that the RDRP continue to expand its activities in these areas so it can build on its earlier successes and respond to new challenges.
Strategic Goal 3: Prevent and Reduce Work-Related Infectious Respiratory Diseases
Continue and expand efforts to protect workers from occupational exposures and to define mechanisms that make workers susceptible to respiratory infections. Enhance surveillance for outbreaks of known and emerging occupational respiratory infections. Develop an overarching structure for the infectious disease program component and coordinate with other federal agencies to adopt technologies for the detection of bioterrorism agents for the protection of workers. The RDRP’s efforts on infectious diseases appropriately concentrate on preventing infection through the use of respirators and understanding why certain people are susceptible to these infections. In addition, more robust surveillance for disease outbreaks in occupational settings is needed.
Strategic Goal 4: Prevent and Reduce Work-Related Respiratory Malignancies
Develop a comprehensive plan for addressing respiratory malignancies in the workplace while assuring the integration of this plan with NIOSH and other federal agency efforts. Refocus research on diagnostic tools to research on biomarkers of exposure or early detection of risk specific to occupational cohorts. The program has had strong impacts in reducing and preventing respiratory cancers from exposures to hexavalent chromium, silica, and diesel exhaust, and continues to address challenging problems related to occupational risk from lung cancer. RDRP should ensure that its research in these areas are well integrated into an overall program of occupational cancer research and not arbitrarily separated by these efforts.
Strategic Goal 5: Prevent Respiratory and Other Diseases Potentially Resulting from Occupational Exposures to Nanomaterials
NIOSH should continue to play a leading role in informing and guiding national and international efforts to address potential occupational hazards and risks associated with the use of manufactured nanomaterials. The report generally supports the RDRP’s research efforts on nanomaterial toxicity, exposure, and dose-response. However, the committee is concerned that there may not be enough long-term data for risk assessments, and therefore the RDRP should consider other approaches for dealing with the potential health impacts of these new materials carefully.
NIOSH should provide appropriate resources for and engage in high-priority occupational disease surveillance. The effectiveness of past NIOSH surveillance activities for coal-dust-related diseases highlight the importance of improved surveillance for other respiratory disorders. Produce a programmatic approach to the development of sampling and analytic methods that include exposure assessment scientists as an integral part of RDRP activities. Exposure assessment is a core component of occupational respiratory disease research and prevention activities. The RDRP does not have specific, programmatic methods for exposure assessment, however. The RDRP is encouraged to explore research strategies in its emergency response efforts. The RDRP has made important contributions to the research and surveillance of respiratory disease in emergency responders to recent disasters, including the World Trade Center and anthrax terror attacks, and Hurricanes Katrina and Rita. Much more can be learned about the relationship between exposure and disease response and ultimately about protecting emergency responders, however. In addition, the RDRP is encouraged to continue to develop cooperative work with other agencies that conduct research in infection and terrorism. The RDRP should prioritize all research proposals under consideration for funding according to the RDRP strategic plan, which needs to be updated periodically. An emerging challenge is how research priorities for respiratory diseases that cut across sectors will be treated. The RDRP needs systems to govern the awarding of grants, contracts, and cooperative agreements and to integrate this external research into the NIOSH program to avoid duplication and inappropriate expenditure on low-priority research.
|Mark Utell (Chair)||University of Rochester, New York|
|John Balmes||University of California, San Francisco|
|Paul Blanc||University of California, San Francisco|
|Elisabeth Chamberlin||Massey Coal Services|
|Rogene Henderson||Lovelace Respiratory Research Institute|
|David Manninno||University of Kentucky, Lexington|
|James Merchant||University of Iowa, Iowa City|
|Jacqueline Nowell||United Food And Commercial Workers International|
|Charles Poole||University of North Carolina, Chapel Hill|
|Richard Schlesinger||Pace University|
|Noah Seixas||University of Washington, Seattle|
|Ira Trager||University of California, Berkeley|
|David Wegman||University of Massachusetts, Lowell|
More information is available at the National Academies Evaluation of NIOSH Research Programs page.
Evidence Package for the National Academies' Review 2006-2007
Materials submitted from the program to its NA evaluation committee. Includes emergency response and disaster preparedness, respirator policy, and sampling and analytical methods activities.
- Page last reviewed: December 21, 2012
- Page last updated: December 21, 2012
- Content source:
- National Institute for Occupational Safety and Health Respiratory Health Division