NIOSHTIC-2 Publications Search

Development of an approach for routinely evaluating current NIOSH notifications: report on trial #2 (Halowax).

Needleman C
Cincinnati, OH: U.S. Department of Health and Human Services, Public Health Service, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health, Purchase Order, 1996 Sep; :1-71
This report is part of a two-tiered project designed to help the National Institute for Occupational Safety and Health (NIOSH) evaluate its programs related to high-risk worker notification. Task I of the project, which examines the long-term impact of a notification/ screening program carried out by NIOSH in the early 1980s, is being reported separately. Task II, the focus of this report, deals with the agency's current notifications. Task II aims not at research per se, but rather at developing a practical tool for routinely monitoring the notification process. NIOSH presently provides the research subjects in worker health studies with information about study results as a matter of routine policy. The purpose of Task II is to develop and test procedures and instruments for evaluating these current NIOSH notifications, focusing in particular on those that involve the sending of individual letters to study subjects. The agency's present method of monitoring its current notifications relies on "bounceback" cards -- brief evaluation survey forms printed on postage-paid postcards included with the notification materials. This approach is low-cost and convenient, but like all subject-initiated feedback, it unavoidably suffers from selection bias and may fail to reflect the full range of responses. In addition, it gives no insight into the reactions of other stakeholder groups (for example, employers and union officials) involved in the notification. The picture emerging from subject-initiated bounceback cards is a very partial one, likely to omit exactly the information that would help most for quality control and program improvement. Task II aims to develop and test a practical way for agency staff to cast a somewhat wider net in getting feedback related to current notifications. The goal is to strengthen the evaluation approach, while remaining realistic about the amount of staff time and effort actually available to carry out evaluations. The evaluation approach needs to be modest in terms of the time and resources required; sharply focused on issues of programmatic concern to the agency; and suited to use by the agency's own staff on a routine basis. As explained in an earlier report, Task II has been planned as a multi-stage project with three separate data-gathering efforts, referred to in this report as "trials." Each trial centers on a different notification effort, selected by the NIOSH Project Officer from among the letter notifications recently carried out by the agency. Trial # 1 dealt with a notification related to talc exposure at Gouverneur Talc Company (GTC) in New York. Preceded by a period of planning and consultation with agency notification staff, this effort represented an exploratory pilot of the proposed evaluation methodology. It allowed the contractor to try out and refine working drafts of the instruments to be used in telephone interviews; to test how well NIOSH's contact information lends itself to doing random telephone surveys; to gather interview data from 25 randomly selected workers in the notified cohort, with attention to any problems that arise in data collection logistics or worker cooperation; to contact representatives of selected stakeholder groups so as to discover the best procedures and any issues or problems associated with getting their input; and to experiment with different formats for reporting the results succinctly. Results of Trial # 1 (described in a report dated May 1996) have been discussed with the NIOSH Project Officer and other agency staff. Trial #2, the subject of this report, reflects some changes suggested by the experience of Trial # 1. Trial # 2 serves as an opportunity for the contractor to test these modified procedures; to work out any further rough spots in the methodology for gathering the evaluation data; and to develop clear guidelines for analyzing and reporting the evaluation findings. As before, the results will be discussed by the contractor and agency personnel. If the results of Trial #2 show a need for further changes in procedures, these will be made in preparation for Trial #3. The plan is then for Trial #3 to test the evaluation methodology as used by agency staff, with only indirect support from the outside contractor. The forms and procedures will be used by Trial #3's notification officer exactly as they would be applied in routine practice, with the outside contractor's role limited to monitoring this process and documenting any problems or concerns that arise. Based on this experience, possibly reflecting further changes, a final version of all forms and procedures will then be recommended to the agency as a standard tool for routine evaluation of all letter notifications in the future.
Risk-factors; Information-processing; Sociological-factors; Psychological-factors; Psychological-processes
Carolyn Needleman, Ph.D, Bryn Mawr College, Bryn Mawr, PA 19010
Publication Date
Document Type
Purchase Order Report
Fiscal Year
NIOSH Division
Source Name
National Institute for Occupational Safety and Health
Performing Organization
Carolyn Needleman
Page last reviewed: March 18, 2022
Content source: National Institute for Occupational Safety and Health Education and Information Division