Introduction
The overarching goal of intervention effectiveness research is to demonstrate the impact of interventions to prevent work-related injury and illness. To develop optimal strategies for improving occupational safety and health (OSH) intervention effectiveness research however, effectiveness studies must be considered within the context of the broader intervention research field as a whole. Research studies that inform intervention development as well as studies that evaluate the implementation of interventions are essential complements to effectiveness studies.

Ideally, interventions should progress cyclically through each of three phases. That is, rather than starting with the effectiveness phase, it’s very important to carefully evaluate the development of an intervention, as well as the process by which it can be successfully implemented. Findings from these phases provide the information necessary to determine the appropriateness, and even the feasibility, of conducting a more rigorous effectiveness study. Positive results from a rigorous effectiveness study could then be used to justify labeling an intervention as a “best practice” and then developing and implementing appropriate policies or national guidelines.

I’d like to focus on the 3 phases of the intervention research process: developmental, implementation, and effectiveness (Goldenhar LM, et al., 2001). And then discuss the tasks that need to be carried out within those 3 phases.

Developmental research phase
Intervention development is the process of deciding on an intervention strategy and developing, or adapting it to fit the context of interest. Previously developed interventions might be selected, new interventions might be developed from scratch, or some combination of these two approaches might be used.

Some questions asked during this phase of the process include:
1) What changes are needed to enhance the health of the population?
2) What are the best ways to bring about these changes?
3) What is the extent of buy in from stakeholders?
4) Are there any useful OSH guidelines or principles?

With respect to #4, existing principles and practice guidelines from OSH and other fields can guide intervention development or selection.

For example the hierarchy of controls might be an appropriate framework to use as a starting point when attempting to reduce occupationally-related dermal exposure. As is well-known, this hierarchy prioritizes intervention types in order of being most to least effective at reducing injury and disease. There is some variation in how people categorize the interventions or controls, particularly the administrative ones, but the basic idea is the same – the less involved the worker has to be for protecting herself using specific behaviors, the better.

Eliminating the hazard at the source using an engineering control or substituting hazardous materials with safer alternatives is the most preferred method for controlling skin exposures. This first requires that engineers and/or chemists work in the lab to develop and test methods until they work in a controlled environment (this is called efficacy testing). Then, still within the developmental phase, the methods need to be pilot tested in the field for usefulness and acceptability by workers.

For an example in the dermal exposure area, workers were highly exposed to metal working fluids that were being continuously pumped over the object they were working on. To reduce exposure, a solenoid valve was introduced that diverted the flow of fluid from the working piece back to the reservoir. This not only greatly reducing the extent the worker came in contact with the fluid, it also had the added benefit of reducing the loss of metal working fluid.

If the first line-of-defense is not available or possible, the next best choice is to identify or develop an intervention that eliminates or reduces the path of exposure between the hazardous material and the worker. This might involve developing an automated procedure that replaces work done by hand or developing tools for workers that allow them to do the work without contacting the hazardous material. Interventions such as job-rotation and education and training are included in this category as well. Compared to the first types of interventions, these often require a greater degree of worker and management involvement.

For example, workers using isocyanate-containing paints had to manually clean their spray gun after each use by dipping it in a can of solvent which not only caused both skin and inhalation exposure, it was also time-consuming and limited productivity. The intervention developed was an automated sealed spray gun cleaner and the use of a second spray gun so that one was being cleaned while the other was being used. Because the cleaning unit was enclosed, automated, and the unit was ventilated, worker exposure was dramatically reduced.

Keep in mind however that just because an intervention has been put in place, it doesn’t mean that workers will know how to use it, or they might even be resistant to changing the way they’ve always worked. Thus, education and training is often necessary and implementation evaluation is critical.

The least preferred method of reducing worker exposure is to control it at the exposure target. Examples for reducing dermatitis include wearing gloves and other types of personal protective equipment and/or applying barrier creams and moisturizers. Clearly, these are the least desirable types of interventions because they require the greatest degree of continual and constant commitment on the part of workers and/or the management. For those of you who have ever worked with hazardous materials, you know that putting on gloves and then trying to work with them on can be a real pain. You lose your tactile sense, and for women and small men in particular, they often do not fit – which, of course can even be dangerous.

Implementation research phase
Implementation studies are designed to systematically document how well an intervention is carried out in the field. Implementation analysis should be integral to effectiveness studies, providing the essential characterization of intervention quality, comprehensiveness, and intensity, as well as the frequency and duration of intervention delivery or participation achieved.

Important implementation questions best answered in comparison to intervention plans and expectations include:
Which and how many potential recipients were reached by the intervention?
Did the content and method of delivery meet expectations?
What was the timing of implementation?
Were competing or conflicting activities taking place during the intervention?

Both quantitative and qualitative data collection methods are useful for fully answering these types of implementation questions.

When linked to developmental and effectiveness studies, implementation studies serve several purposes. First, they provide feedback for ways to enhance the intervention as well as improve its acceptance. Second, implementation research findings are helpful for interpreting effectiveness study findings. For example, in an intervention study designed to reduce dermatitis among workers, quantitative outcome data may not show a reduction; however, qualitative implementation data might show that the reason no effect was shown was because managers needed a greater level of understanding of how to minimize employee risk and more training on how to share this knowledge with the workers.

Findings from an implementation research study can also be used to guide the dissemination and replication of an effective intervention to other settings. By providing clear, systematic information about the implementation process, others are able to efficiently adapt the intervention to new settings, avoiding identified and documented pitfalls.

Effectiveness research phase
Finally, effectiveness studies probably need the least amount of explanation. These types of studies tell us the extent to which an intervention worked, or did not work, under real world, rather than lab, conditions and are sometimes referred to as impact, outcome, or summative evaluation studies. Such studies answer questions such as:
(1) To what extent did the intervention reduce occupational injuries, illnesses, disability or fatalities?
(2) To what extent did the intervention reduce worker exposure to hazardous conditions?
(3) What is the effect of the intervention on the social and economic consequences of work injury and illness
(4) How have workers’ knowledge, attitude, or behaviors changed due to the intervention?

Of course, the need to determine if your intervention or program did what it was supposed to do is self-evident. Obtaining this type of information allows us to put to best use the limited resources available for OSH interventions.

Integrating developmental, implementation and effectiveness research
As I mentioned earlier, in an ideal world, developmental, implementation, and effectiveness studies would be interdependent, being conducted in a three-phase sequence to fully characterize the basis, implementation experience, and effectiveness of a given intervention. The findings of implementation and effectiveness studies would then feed back to inform further development of the intervention, thus establishing a cycle of continuous improvement. Although this type of comprehensive intervention research effort is not typically seen in OSH research, there are beginning to be some models for doing it.

Five intervention research tasks
Finally, in our framework, we propose a basic set of common tasks that should be carried out in most intervention research studies. Rather than a prescriptive list, it is intended to provide a general checklist for optimizing intervention research quality and impact. Five groups of tasks are enumerated around the center of the intervention research cycle (see figure).

Gather background information - A sound developmental study would include a detailed needs assessment. Questions would be asked to help characterize the problem and its history, the range of intervention alternatives, and the socio-political work, and evaluation settings, such as labor-management relations or any different evaluation agendas among stakeholders. Similarly, background information on the intervention itself should also be gathered before conducting implementation or effectiveness studies. Questions would include: What is the evaluation history of this intervention? Have implementation or effectiveness studies been conducted on similar interventions in the past? If so, what insights from previous studies can be applied? What specific aspects of implementation or effectiveness should be evaluated? Why? What are the perspectives of various stakeholders on such questions? What principals guide the intervention?

Develop partnerships - For intervention research to achieve maximal impact, the perspectives of all stakeholders should be considered and addressed, to the extent feasible. Stakeholder involvement in the research process maximizes the chances that evaluation findings will be applied to improving intervention efforts. Another aspect of partnerships pertains to the research team. Although perhaps not easily accomplished, the concept of using multidisciplinary research teams to plan and carry out the intervention project makes sense. Depending on the nature of the project, these teams could include workers, employers, and scientists from various disciplines. Participatory action research methods integrate stakeholder perspectives by actively involving intervention subjects as participants in the research process. This is the most promising partnership of all.

Thus, in partnership you will decide:
What is the outcome objective?
What do you expect will happen or change as a result of the intervention? and
How will the objectives be met?

Choose methods or designs - Findings from the first two steps help guide the selection of study methods and designs to answer the chosen evaluation questions in the most efficient and economical way. For example, using the information gathered in steps 1 and 2, a developmental lab-based study might be designed and conducted to evaluate the efficacy of a barrier cream compared to chemical-protective gloves to reduce contact dermatitis. For an implementation study, one might choose the qualitative focus group method to determine from workers if pilot interventions had been successfully implemented. For an effectiveness study, a population-based time series design might be appropriate for evaluating the impact of an OSHA enforcement initiative, whereas a small-scale, controlled experiment might be feasible for evaluating a promising worker education intervention at a couple of worksites.

Randomized, controlled trials are the gold standard for determining cause and effect between interventions and outcomes. In the OSH context, however, such studies are sometimes not feasible due to practical, ethical, legal, or other constraints. Other design options, for example, quasi-experimental studies and case studies are available to use when randomized, controlled experiments are not possible. Causal inference can be greatly enhanced by combining qualitative and quantitative approaches.

If you are interested in learning more about these options and conducting these types of studies, consult the Guide (Robson, et al., 2001).

Conduct/complete the research - The first three tasks are primarily planning activities. The fourth task would be to carry out what has been planned. Clearly, in the intervention implementation and effectiveness phases, research teams must anticipate running into and managing the typical difficulties of field research. These can include, but are not limited to, changes in the intervention mid-stream, change in workplace personnel or priorities or in participants’ decisions to participate, ethical issues that arise when an intervention appears very effective, etc.

Some questions to ask to help ensure the research can be conducted and completed include:
1. What resources are available for data collection, analysis, and reporting?
2. Are there already existing data that can be used? If so, how will they be accessed/obtained?
3. If not, who will collect new data, how will it be collected, and who will oversee and manage the process?
4. Will data be gathered on all participants or only a sample? If a sample, how will the sample be selected and what is the sample size?
5. How often will data be collected? How often reported?
6. How will be data be processed and analyzed? By whom?

Report or disseminate findings - This task is typically seen as the essential closing of the intervention research loop. There are always at least two key groups to whom you should report your findings: intervention participants and other OSH intervention researchers. For both groups, direct and timely reporting in a format and language that is understood is optimal. Various types of media should be used or even whole media campaigns can be developed to disseminate findings to interested parties including other workers, different employers, interventionists, labor unions, employer associations, and sponsors/funders of interventions and evaluation studies.

Reporting intervention research findings in the published literature or through other information dissemination mechanisms is also necessary so that the broad OSH community can learn and benefit from the findings. This crucial step, which applies equally to negative as well as positive findings, is what makes it possible for the intervention research cycle to progress through contributions made from investigators working in diverse contexts on the full array of OSH intervention research questions.

Acknowledgement
This presentation is derived in part from the work of the National Occupational Research Agenda (NORA) Intervention Effectiveness Research Team. For more information about the team, visit the NIOSH website http://www.cdc.gov/niosh and follow the links to the NORA team.

References
Goldenhar LM, LaMontange AD, Katz T, Heaney C, Landsbergis P (2001) The Intervention Research Process in Occupational Safety and Health: An overview from the National Occupational Research Agenda Intervention Effectiveness Research Team. Journal of Occupational and Environmental Medicine, 43(7):616-622.

Robson, LS, Shannon, HS, Goldenhar, LM, Hale, AR (2001) Guide to Evaluating the Effectiveness of Strategies For Preventing Work Injuries: How to Show Whether a Safety Intervention Really Works., NIOSH publication #2001-119. (To obtain a copy, contact NIOSH by phone 1-800-356-4674 within the US or 513-533-8471 outside the US, Fax: 513-533-8573, E-mail: pubstaft@cdc.gov or download using the publications links on the website http://www.cdc.gov/niosh.)