Start an Evaluation
CDC’s Evaluation Framework is a concise, thorough approach that can be easily understood and applied. The framework, which has been adapted for use with CoPs, emphasizes six logical steps and can be used as a starting point for CoPs evaluation. 25
Each of the six steps is described in detail below.
- Engage StakeholdersThe evaluation cycle begins by engaging stakeholders (i.e., the persons or organizations with an investment in what will be learned from an evaluation and what will be done with the knowledge). Public health work, including CoPs, involves partnerships; therefore, any assessment of a public health program requires considering the value systems of the partners. Stakeholders should be engaged in a way that ensures their perspectives can be heard and understood. When stakeholders are not engaged in this way, evaluation findings might be ignored, criticized, or resisted because they do not address the stakeholders’ questions or values. After becoming involved, stakeholders help to execute the remaining steps.
- Describe the CommunityCommunity descriptions convey the mission and objectives of the CoP being evaluated. Descriptions should be sufficiently detailed to ensure understanding of the CoP’s goals and strategies. The description should discuss the CoP’s capacity to effect change, its stage of development, and how it fits into the larger public health community. CoP descriptions set the frame of reference for all subsequent decisions in an evaluation. The description enables comparisons with similar CoPs and facilitates attempts to connect community components to their effects. Moreover, stakeholders may have differing ideas regarding CoP goals and purpose. Evaluations done without agreement on the community definition are likely to be of limited use. Sometimes, negotiating with stakeholders to formulate a clear and logical description will bring benefits before data are available to evaluate CoP effectiveness.
- Focus the Evaluation DesignThe direction and process of the evaluation must be focused to assess the issues of greatest concern to stakeholders, while using time and resources as efficiently as possible. Not all design options are equally well-suited to meeting the information needs of stakeholders. After data collection begins, changing procedures might be difficult or impossible, even if better methods become obvious. A thorough plan anticipates intended uses and creates an evaluation strategy with the greatest chance of being useful, feasible, ethical, and accurate.
- Gather Credible EvidencePersons involved in an evaluation should strive to collect information that will convey a well-rounded picture of the CoP and be seen as credible by the evaluation’s intended audience. Information (i.e., evidence) should be perceived by stakeholders as believable and relevant for answering their questions. Such decisions depend on the evaluation questions being posed and the motives for asking them. Credible evidence strengthens evaluation conclusions and the recommendations that follow.
- Justify ConclusionsEvaluation conclusions are justified when linked to the evidence gathered and judged against agreed-upon values or standards set by the stakeholders. Stakeholders must agree that conclusions are justified before they will use the evaluation results with confidence.
- Ensure Use and Share Lessons LearnedLessons learned during the evaluation should automatically translate into informed decision-making and appropriate action. To make sure the evaluation findings are used correctly, deliberate effort is needed to ensure the evaluation processes and findings are disseminated and interpreted appropriately. Preparing for use involves strategic thinking and continued vigilance of the changing environment, both of which begin in the earliest stages of stakeholder engagement and continue throughout the evaluation.
- Utility: Does the evaluation serve the information needs of intended users?
- Feasibility: Is the evaluation realistic, prudent, diplomatic, and frugal?
- Propriety: Has the evaluation been conducted legally, ethically, and with due regard for the welfare of those involved in the evaluation, as well as those affected by its results?
- Accuracy:Will the evaluation reveal and convey technically adequate information about the features that determine worth or merit of the CoP?
Moving Along with the Evaluation
Once you understand the basic concepts of evaluation, several additional activities can be completed to target the evaluation to the needs of your community.
- First, assess the state of the field your CoP addresses. A SWOT analysis (defined below) can help you define what currently exists, what is needed, and where the CoP can target its activities.
- Use the information collected in the analysis to develop your CoP’s goals and SMART objectives (defined below). Specific questions related to the work conducted within the CoP can then be added to the Evaluation Interview Template.
|SMART Objectives Template[DOC – 187KB]||When it comes time to define the goals and objectives of your CoP, this template will walk you through the process of developing specific, measurable, achievable, realistic, and time-based objectives.||Leaders of new and existing CoPs|
|Evaluation Interview Template[DOC – 189KB]||This template helps you start-off your evaluation process by aligning it with the standard evaluation for CoPs. The template contains sample evaluation objectives and sample questions. Each CoP can modify the template by modifying or adding additional questions that are specific to its goals and objects.||Leaders of new and existing Communities of Practice (CoPs)|