Effective TB Interviewing for Contact Investigation: Facilitator Led Training Guide 2006

Return to Table of Contents

9. Evaluation

Course evaluation is the assessment of a training program based on

• Participants’ written and verbal comments and ratings; and

• Participants’ improvement in skills and knowledge both on a short- and long-term basis.

• Evaluation should measure whether the course objectives have been accomplished. There are several methods for completion of the course evaluation process for both immediate and long-term feedback. The performance of objectives is one measure for course evaluation (outcome evaluation); another is to determine how the course progressed (process evaluation). This will assist in planning future training.

Immediate Evaluation

The best way to assess the course structure and participant satisfaction is through immediate evaluation. This can be done through a simple written form that participants should be required to complete at the end of the course. The purpose of the immediate written feedback is to determine

• The strengths and weaknesses of the course

• The participants’ self-reported satisfaction with their increase in learning and the skills built

• How the course’s format helped or deterred from learning

• Other types of training that may be required

• The effectiveness of teaching style of the facilitators and trainers.

A sample evaluation tool for the course is shown in Figure 2 (page 23). A skill-building evaluation is included in Figure 3 (page 24). This can also be incorporated into a written participant evaluation form for immediate evaluation.

Figure 2. Sample Written Course Evaluation Form

Please respond to the following statements by circling the appropriate number for your response (from the rating scale below).

1 - Strongly agree 2 - Agree 3 - Disagree 4 - Strongly disagree

1. The objectives were clearly stated at the beginning of the course.	1	2	3	4
2. The course objectives were satisfactorily met.	1	2	3	4
3. The trainer(s) was (were) knowledgeable about the subject matter. [LIST TRAINERS NAMES INDIVIDUALLY]	1	2	3	4
4. The trainer(s) exhibited effective training skills during the course. [LIST TRAINERS NAMES INDIVIDUALLY]	1	2	3	4
5. The trainer(s) presentation was interactive. [LIST TRAINERS NAMES INDIVIDUALLY]	1	2	3	4
6. I was given enough opportunities to ask questions and express concerns.	1	2	3	4
7. I was given enough time to practice skills in eliciting names and other necessary information for locating contacts.	1	2	3	4
8. This course was long enough to build my skills.	1	2	3	4
9. The course activities promoted skill-building.	1	2	3	4
10. The learning environment was comfortable.	1	2	3	4
11. I would recommend this course to others.	1	2	3	4
12. My TB interviewing skills were greatly enhanced by taking this course.	1	2	3	4
If any of the above ratings are 3 or 4, list which activities could have been improved and how.
What did you find most beneficial about this course?
What would you have changed about this course?
Did the training meet your expectations?
Other comments:

The written course evaluation form should be given to the participants at the end of the course during the designated evaluation completion time. 

Figure 3. Post-Course Self-Skills Evaluation Form

This course was designed to build TB interviewing skills. Indicate the degree to which this course did or did not provide you with sufficient information and skills to perform these tasks listed. Please respond to the following statements by circling the appropriate number for your response (from the rating scale below) in the spaces to the right of each statement.

1-Strongly Agree 2-Agree 3-Disagree 4-Strongly disagree

During an interview I will be able to:

Build trust and rapport with patient	1	2	3	4
Listen actively	1	2	3	4
Use open-ended and close-ended questions	1	2	3	4
Communicate at the patient’s level of comprehension	1	2	3	4
Solicit an index patient’s feedback	1	2	3	4
Motivate and encourage active participation of the index patient	1	2	3	4
Display nonjudgmental behavior	1	2	3	4
Assess the need for a proxy	1	2	3	4
Develop flexibility in the interview process	1	2	3	4
Recognize the need to stop and reschedule a stalled interview	1	2	3	4
Identify and address patient concerns	1	2	3	4
Recognize and address verbal problem indicators	1	2	3	4
Recognize and address nonverbal problem indicators	1	2	3	4
Maintain control of the interview	1	2	3	4
Identify and resolve communication barriers	1	2	3	4
Formulate an infectious period	1	2	3	4
Distinguish between a close and casual contact	1	2	3	4
Develop rationale for contact investigation plan	1	2	3	4

Change in Performance and Impact Evaluation

The more important, yet more challenging, type of evaluation is impact evaluation. This is a specific long-term evaluation of interviewing performance changes over time. Impact evaluation measures the way in which the participants’ interviewing skills have changed as a result of the course. This is challenging because you will have to analyze whether any changes were due to the course itself or to other external factors. Therefore, the best way to measure change is by conducting an evaluation process prior to the course and then again after the course.

Figure 4 (pages 26-27) shows a sample interview evaluation form in two parts. This form should be used by an experienced interviewer or observer to assess a course participant’s interviewing skills. In order to assess the impact of the training, an evaluation should be done prior to the course. The same observer should assess the same participant after the course as well. Use of the same observer and interviewer pair will avoid inter-observer variability. One observer can also assess multiple interviewers. Participant evaluations should be done as close as possible to the end of the course, or at least within 3 months of the completion of the course. Another interviewer evaluation should be considered within 6 months of the training, in order to gauge the lasting impacts of the course.

Figure 4.  Sample Interview Evaluation Form

Sample TB Patient Interview Evaluation Form	Excellent	Satisfactory	Needs Improvement
Pre-Interview Activity
1. Reviews index patent’s medical record
2.Obtains/reviews locating information for the index patient
3. Establishes preliminary infectious period
4. Develops strategy for interview process
5. Arranges interview appointment time and place
6. Arranges and ensures privacy
Introduction
7. Introduces self
8. Explains purpose of interview
9. Emphasizes confidentiality
Information & Education Exchange
10. Collects/confirms the index patient’s personal information
11. Determines the index patient’s level of disease comprehension
12. Provides appropriate TB education
13. Reviews symptom history
14. Discusses basis of diagnosis
15. Discusses appropriate disease intervention behaviors
Contact Identification
16. Defines close and casual exposure
17. Verbalizes a sense of urgency
18. Identifies household, workplace/school, other congregate-setting, and social and recreational contacts
19. Pursues detailed contact information
20. Persists to identify all close contacts
21. Explains contact referral process
Conclusion
22. Invites index patient’s questions
23. Reviews and reinforces adherence to treatment plan
24. Establishes a date for re-interview
Communication
26. Demonstrates professionalism
27. Establishes trust and rapport
28. Listens actively
29. Uses open- and closed-ended questions appropriately
30. Communicates at the index patient’s level of comprehension
31. Provides factually correct information
32. Solicits the index patient’s feedback
33. Provides encouragement
34. Uses appropriate nonverbal communication
35. Motivates and encourages active participation of the index patient
36. Displays nonjudgmental behaviors
Problem Solving
37. Assesses the need for identifying an appropriate proxy
38. Displays flexibility in the interview process
39. Recognizes the need to stop and reschedule a stalled interview
40. Identifies and addresses index patient’s concerns
41. Recognizes and addresses verbal problem indicators
42. Recognizes and addresses nonverbal problem indicators
43. Maintains control of interview
44. Identifies and begins resolution of barriers
Analysis
45. Refines the infectious period
46. Distinguishes between close and casual contact
47. Develops rationale for contact investigation plan as verbalized to patient

Comments

Facilitator Evaluation

Finally, you and any cofacilitators should evaluate the training program’s effectiveness. This can be done not only from formal written and external observation, but also from your own reactions. Here are questions you may ask yourself about the quality of your training. Responses to these questions can be used to improve the next interviewing course offering:

How did participants react to the course? (Process evaluation)

Watch the participants during the training. If they are uncomfortable or tense, try to determine the source of the problem and how to make people more comfortable. It helps to identify those who were uncomfortable in terms of experience levels and the format of the activity.

What did the participants learn from the workshop? (Outcome evaluation)

Review the course learning objectives and assess whether you think these have been met.

Ask yourself before each activity:

• What makes this activity appropriate now and what will participants learn?

• What changes may be necessary to make certain activities appropriate at the time they are conducted?

Ask yourself after each activity:

• What have I learned from these activities?

• What have the participants learned from these activities?

Short-term and long-term evaluations can assist in improving future courses and can suggest refresher training that can be done on a periodic basis.