Professional Development 201: From Basic to Dynamic

6: After the Webinar

Evaluation Planning

After your webinar, you should conduct an evaluation. Evaluation is one of the six professional development practices and critical for improving future efforts.

Carefully consider how evaluation data will be used and by whom when planning the evaluation of a webinar. This will help you focus the evaluation on the items for which you (and other relevant stakeholders) plan to take action based on participant responses. Identify the topics where feedback is needed the most. This could include facilitator effectiveness, content, technical components, handouts, timing, and interactivity.

Develop a plan before the webinar for securely and efficiently collecting, storing, analyzing, and reporting evaluation data.

There may be options, such as polling features, for collecting evaluation data in the webinar platform. Otherwise, an outside mechanism may be needed for collecting data. This could include linking to survey software or asking participants to respond to a set of emailed questions. Consider whether is it important for your participants to be able to respond to the evaluation anonymously. Regardless of the data collection method used, build time into the agenda, prior to the closers, for participants to complete the evaluation form.

PD101 discussed how recent studies have shown that traditional pre- and post-questionnaires can misrepresent the impact of training. A better approach is the retrospective pre-post method in which participants complete one questionnaire at the end of the training. In this questionnaire, they are asked to retrospectively assess their knowledge or skill level prior to the event and then assess it now, afterward. This reduces participant bias by presenting the same frame of reference to both pre- and post-questions.

Here are some general guidelines for designing a good retrospective pre-post questionnaire.

• Use concise instructions that clearly guide the respondent in what to do.
• Remind the respondent to provide two answers for each question: one for past assessment of knowledge, skills, and attitudes and one for a current assessment.
• Place the pre-training question first on the form, and
• Format the questionnaire in a way that helps respondents understand what they are being asked to do. For example, use shading, font styles, and question layouts that call attention to the pre-post ratings.

If using an evaluation form, be sure to include options for both qualitative and quantitative responses.

Qualitative responses can be both direct and open-ended responses. These give participants the chance to not only give honest and direct feedback in their own words, but detailed comments of information you may not have addressed in other questions.

Quantitative responses can be closed-ended questions such as yes or no answers, checklists, and rating scales.

It is also possible to use questions that combine quantitative and qualitative responses.

Here are some guidelines:

• Make response options as clear as possible.
• Standardize response categories and scales to make them uniformly consistent.
• Avoid biased words or phrases and jargon.
• Avoid asking about multiple items in a single question (e.g., 'rate the sound and visual quality of the webinar').
• Involve those who will be responsible for data analysis early in the process to ensure a smooth transition from data collection to data analysis.

Remember:

• The easier it is for participants to respond to the evaluation, the higher the response rate will be.
• The more organized and automated the data collection process is upfront, the easier it will be to analyze data later.
• Ask questions that will inform future webinars or professional development events.

Download the Data Collection Framework for more information on developing evaluation questions. Also, visit https://www.cdc.gov/healthyyouth/evaluation/ for information on analyzing qualitative data.