Effective TB Interviewing for Contact Investigation: Facilitator Led Training Guide 2006
Course evaluation is the assessment of a training program based on
Participants’ written and verbal comments and ratings; and
Participants’ improvement in skills and knowledge both on a short- and long-term basis.
Evaluation should measure whether the course objectives have been accomplished. There are several methods for completion of the course evaluation process for both immediate and long-term feedback. The performance of objectives is one measure for course evaluation (outcome evaluation); another is to determine how the course progressed (process evaluation). This will assist in planning future training.
The best way to assess the course structure and participant satisfaction is through immediate evaluation. This can be done through a simple written form that participants should be required to complete at the end of the course. The purpose of the immediate written feedback is to determine
The strengths and weaknesses of the course
The participants’ self-reported satisfaction with their increase in learning and the skills built
How the course’s format helped or deterred from learning
Other types of training that may be required
The effectiveness of teaching style of the facilitators and trainers.
A sample evaluation tool for the course is shown in Figure 2 (page 23). A skill-building evaluation is included in Figure 3 (page 24). This can also be incorporated into a written participant evaluation form for immediate evaluation.
Figure 2. Sample Written Course Evaluation Form
Please respond to the following statements by circling the appropriate number for your response (from the rating scale below).
1 - Strongly agree 2 - Agree 3 - Disagree 4 - Strongly disagree
|1. The objectives were clearly stated at the beginning of the course.||1||2||3||4|
|2. The course objectives were satisfactorily met.||1||2||3||4|
|3. The trainer(s) was (were) knowledgeable about the subject matter.|
[LIST TRAINERS NAMES INDIVIDUALLY]
|4. The trainer(s) exhibited effective training skills during the course.|
[LIST TRAINERS NAMES INDIVIDUALLY]
|5. The trainer(s) presentation was interactive.|
[LIST TRAINERS NAMES INDIVIDUALLY]
|6. I was given enough opportunities to ask questions and express concerns.||1||2||3||4|
|7. I was given enough time to practice skills in eliciting names and other|
necessary information for locating contacts.
|8. This course was long enough to build my skills.||1||2||3||4|
|9. The course activities promoted skill-building.||1||2||3||4|
|10. The learning environment was comfortable.||1||2||3||4|
|11. I would recommend this course to others.||1||2||3||4|
|12. My TB interviewing skills were greatly enhanced by taking this course.||1||2||3||4|
|If any of the above ratings are 3 or 4, list which activities could have been improved and how.|
|What did you find most beneficial about this course?|
|What would you have changed about this course?|
|Did the training meet your expectations?|
The written course evaluation form should be given to the participants at the end of the course during the designated evaluation completion time.
Figure 3. Post-Course Self-Skills Evaluation Form
This course was designed to build TB interviewing skills. Indicate the degree to which this course did or did not provide you with sufficient information and skills to perform these tasks listed. Please respond to the following statements by circling the appropriate number for your response (from the rating scale below) in the spaces to the right of each statement.
1-Strongly Agree 2-Agree 3-Disagree 4-Strongly disagree
During an interview I will be able to:
|Build trust and rapport with patient||1||2||3||4|
|Use open-ended and close-ended questions||1||2||3||4|
|Communicate at the patient’s level of comprehension||1||2||3||4|
|Solicit an index patient’s feedback||1||2||3||4|
|Motivate and encourage active participation of the index patient||1||2||3||4|
|Display nonjudgmental behavior||1||2||3||4|
|Assess the need for a proxy||1||2||3||4|
|Develop flexibility in the interview process||1||2||3||4|
|Recognize the need to stop and reschedule a stalled interview||1||2||3||4|
|Identify and address patient concerns||1||2||3||4|
|Recognize and address verbal problem indicators||1||2||3||4|
|Recognize and address nonverbal problem indicators||1||2||3||4|
|Maintain control of the interview||1||2||3||4|
|Identify and resolve communication barriers||1||2||3||4|
|Formulate an infectious period||1||2||3||4|
|Distinguish between a close and casual contact||1||2||3||4|
|Develop rationale for contact investigation plan||1||2||3||4|
The more important, yet more challenging, type of evaluation is impact evaluation. This is a specific long-term evaluation of interviewing performance changes over time. Impact evaluation measures the way in which the participants’ interviewing skills have changed as a result of the course. This is challenging because you will have to analyze whether any changes were due to the course itself or to other external factors. Therefore, the best way to measure change is by conducting an evaluation process prior to the course and then again after the course.
Figure 4 (pages 26-27) shows a sample interview evaluation form in two parts. This form should be used by an experienced interviewer or observer to assess a course participant’s interviewing skills. In order to assess the impact of the training, an evaluation should be done prior to the course. The same observer should assess the same participant after the course as well. Use of the same observer and interviewer pair will avoid inter-observer variability. One observer can also assess multiple interviewers. Participant evaluations should be done as close as possible to the end of the course, or at least within 3 months of the completion of the course. Another interviewer evaluation should be considered within 6 months of the training, in order to gauge the lasting impacts of the course.
Figure 4. Sample Interview Evaluation Form
|Sample TB Patient Interview Evaluation Form||Excellent||Satisfactory||Needs Improvement|
|1. Reviews index patent’s medical record|
|2.Obtains/reviews locating information for the index patient|
|3. Establishes preliminary infectious period|
|4. Develops strategy for interview process|
|5. Arranges interview appointment time and place|
|6. Arranges and ensures privacy|
|7. Introduces self|
|8. Explains purpose of interview|
|9. Emphasizes confidentiality|
|Information & Education Exchange|
|10. Collects/confirms the index patient’s personal information|
|11. Determines the index patient’s level of disease comprehension|
|12. Provides appropriate TB education|
|13. Reviews symptom history|
|14. Discusses basis of diagnosis|
|15. Discusses appropriate disease intervention behaviors|
|16. Defines close and casual exposure|
|17. Verbalizes a sense of urgency|
|18. Identifies household, workplace/school, other congregate-setting, and social and recreational contacts|
|19. Pursues detailed contact information|
|20. Persists to identify all close contacts|
|21. Explains contact referral process|
|22. Invites index patient’s questions|
|23. Reviews and reinforces adherence to treatment plan|
|24. Establishes a date for re-interview|
|26. Demonstrates professionalism|
|27. Establishes trust and rapport|
|28. Listens actively|
|29. Uses open- and closed-ended questions appropriately|
|30. Communicates at the index patient’s level of comprehension|
|31. Provides factually correct information|
|32. Solicits the index patient’s feedback|
|33. Provides encouragement|
|34. Uses appropriate nonverbal communication|
|35. Motivates and encourages active participation of the index patient|
|36. Displays nonjudgmental behaviors|
|37. Assesses the need for identifying an appropriate proxy|
|38. Displays flexibility in the interview process|
|39. Recognizes the need to stop and reschedule a stalled interview|
|40. Identifies and addresses index patient’s concerns|
|41. Recognizes and addresses verbal problem indicators|
|42. Recognizes and addresses nonverbal problem indicators|
|43. Maintains control of interview|
|44. Identifies and begins resolution of barriers|
|45. Refines the infectious period|
|46. Distinguishes between close and casual contact|
|47. Develops rationale for contact investigation plan as verbalized to patient|
Finally, you and any cofacilitators should evaluate the training program’s effectiveness. This can be done not only from formal written and external observation, but also from your own reactions. Here are questions you may ask yourself about the quality of your training. Responses to these questions can be used to improve the next interviewing course offering:
How did participants react to the course? (Process evaluation)
Watch the participants during the training. If they are uncomfortable or tense, try to determine the source of the problem and how to make people more comfortable. It helps to identify those who were uncomfortable in terms of experience levels and the format of the activity.
What did the participants learn from the workshop? (Outcome evaluation)
Review the course learning objectives and assess whether you think these have been met.
Ask yourself before each activity:
What makes this activity appropriate now and what will participants learn?
What changes may be necessary to make certain activities appropriate at the time they are conducted?
Ask yourself after each activity:
What have I learned from these activities?
What have the participants learned from these activities?
Short-term and long-term evaluations can assist in improving future courses and can suggest refresher training that can be done on a periodic basis.