Formal evaluation of a tobacco-free campus initiative is an ongoing activity that is necessary to assess whether the initiative has achieved its objective and what tangible outcomes it has had. The evaluation can also identify ways to improve the initiative. Management input can be as valuable in this phase as it is in your initial assessment and planning phases.
Conducting the Evaluation
Begin planning early to ensure that the data you want will be collected during all stages of the initiative. Develop a plan for evaluating the implementation and impact of the policy initiative, including the impact of the expanded cessation services. Remember to keep it fairly simple to assure that the evaluation will be conducted.
Below are some sample questions that will help you develop your evaluation design:
- What information does management want? How does management define success?
- Are employees aware of the new policy and available tobacco cessation services? Are employees who use these services satisfied with the support received?
- Are employees satisfied with how the policy is being enforced?
- What information is needed to determine whether you have achieved the project goals and objectives?
- How will this information be gathered? Are data collection instruments needed (e.g., employee survey)?
- Who needs to be involved? Who will collect the data? Who will analyze the data? Who will be responsible for tracking and compiling employee questions and comments?
- When should key tasks be completed?
- How long will the information be kept and how will the security of the data be ensured?
- How will evaluation data be reported?
Consider conducting a pre- and post-implementation employee survey to assess awareness and knowledge of the policy and available tobacco use cessation services. Before collecting employee input, obtain guidance from appropriate agency experts to help determine what approvals are needed. For example, federal agencies are subject to Office of Management and Budget (OMB) regulations and will experience a long lead time if OMB approval is needed. Similarly, if your assessment is part of a research project, check with the appropriate institutional review board (IRB) to determine whether IRB approval is needed. Each agency or company should work with its own internal policy and legal staff to develop appropriate guidelines and procedures for gathering employee input.
To measure the extent and effectiveness of TFC policy enforcement, track the number of employees and visitors who are reported using tobacco on campus on a monthly basis. This can be determined by the number of "health tickets" distributed each month, if you choose to use that communication strategy.
In addition, consider evaluating quit rates after six months among those who have received tobacco use cessation services through the employee health services. For example, track how many employees have
- Called or come in for assistance
- Visited the TFC Web site
- Tried to quit
- Received medications
- Received a second round of medications
- Succeeded in quitting for at least three months
Report evaluation results to the planning team and top management.
After you have completed the initial evaluation of your TFC initiative, consider conducting periodic follow-up assessments to determine whether the initiative is continuing to be implemented as planned and to identify necessary modifications.
Other Helpful Information
- HWI’s Evaluation Information
- Introduction to Program Evaluation for Comprehensive Tobacco Control Programs
- Key Outcome Indicators for Evaluating Comprehensive Tobacco Control Programs
- Office of Management and Budget Regulations