Program Evaluation Framework Checklist for Step 2
Describe the Program
‹View Table of Contents
A logic model is a graphic depiction (road map) that presents the shared relationships among the resources, activities, outputs, and outcomes/impacts for your program. It depicts the relationship between your program’s activities and its intended effects, in an implicit ‘if-then’ relationship among the program elements — if I do this activity, then I expect this outcome. Among other things, a logic model helps clarify the boundary between ‘what’ the program is doing and ‘so what’—the changes that are intended to result from strong implementation of the “what.”
A logic model can focus on any level of an enterprise or program: the entire organization, one of its component departments or programs, or just specific parts of that department or a program. Of course, the boundary between “what” and “so what” will vary accordingly.
Logic models differ widely in format and level of detail. Here are some key terms used in logic models, although not all are employed in any given model:
- Inputs: The resources needed to implement the activities
- Activities: What the program and its staff do with those resources
- Outputs: Tangible products, capacities, or deliverables that result from the activities
- Outcomes: Changes that occur in other people or conditions because of the activities and outputs
- Impacts: [Sometimes] The most distal/long-term outcomes
- Moderators: Contextual factors that are out of control of the program but may help or hinder achievement of the outcomes
Let’s get started. Here are the key steps to developing a useful logic model:
Gather information available on the program, including but not limited to:
- Mission and vision
- Goals and objectives
- Current program descriptions such as websites, program descriptions, fact sheets
- Strategic plans
- Business, communication, and marketing plans
- Existing/previous logic models
- Existing performance measures and/or program reviews
Review the information and extract from it to create a two-column table including:
- Column 1: Activities: What the program and its staff do.
- Column 2: Outcomes: Who or what beyond the program and its staff needs to change and how. In generating outcomes, it helps to Identify the target audiences for program activities and the action they must take in order for the activities to be successful.
- Within the list in column 2, identify the most distal outcome: What is the big public health problem you aim to address with your program?
Logic models are the most common, but not the only, name applied to a visual depiction of a program. Here are some names of others approaches that either replicate or closely resemble logic models in their format and intent. There are occasions where one approach/format is a better fit than another, but often any of these will work equally well:
- Program Roadmaps
- Theory of Change
- Theory of Cause
- Theory of Action
- Concept(ual) Maps
- Outcome Maps
- Logical Frameworks (LogFrames)
Clarify the activities and outcomes with stakeholders* to ensure:
- Appropriate classification; no activities are actually outcomes and no outcomes listed are actually activities
- No major redundancy in list of activities or list of outcomes
- No major missing activities or outcomes
Decide whether the activities should be ordered sequentially. If so:
- Think about the “logical” relationship among the activities—which may or may not be the same as how they unfold over time— and determine if some activities need to occur before others can be implemented
- Order the activities within the columns into earlier or later activities to reflect the sequential relationships
Check in with your stakeholders
- To ensure the activities and outcomes reflect their understanding of the program to ensure:
- There are no major missing activities or outcomes
- The logical progression of activities
- The logical progression of the outcomes
- To (re)affirm the intended uses of the logic model (i.e., assess implementation, assess effectiveness, performance measurement, strategic planning)
The intended uses of the logic model, will determine which, if any, of the elaborations below would make the logic model more useful.
If depicting the program logic in a roadmap format is desirable, then:
- Write each of the existing activities and outcomes on a sticky note, or equivalent
- Move the notes around to allow for drawing of lines to depict logical relationships
- Draw in lines remembering that lines may go from:
- One or more activities to a subsequent activity
- One or more activities to an outcome
- One or more proximal outcomes to a more distal outcome
If outputs are desired because stakeholders would like clarification of the direct result of the activities, then using the logic model table or (better) the roadmap:
- Identify the activities for which outputs are desired
- Identify the link between those activities and their successor activities or outcomes
- Thinking about that logical link, what are the key attributes of the activity that must be present for it to produce its successor activity or outcome
- Place the outputs in the appropriate place in the logic model table or roadmap
If inputs are desired because stakeholders would like clarification of necessary resources to implement the program, then:
- Identify the key inputs without which the program cannot be implemented. Think about broad categories such as staff, equipment, data, funds, and partnerships.
- Place the inputs into a column to the left of the activities in the logic model.
- If it is important to see the link between each input and the activity it affects, then draw arrows from each input to the related activity
If moderators are desired because—in the view of stakeholders and users—clarification of potential facilitators or barriers in the larger environment is necessary:
- Identify the key moderators, thinking of broad categories such as political, economic, social, and technological
- Identify what links in the program logic will be facilitated or impeded by the presence or absence of sufficient levels of the moderator. Remember moderators can facilitate or impede the ability of one activity/output to generate a successor activity/output, one activity/output to generate an outcome, a proximal outcome to generate a more distal outcome
- Be especially conscious of key moderators without which the program cannot be implemented
- Place the moderators into the appropriate place in the logic model table or roadmap.
- If using a roadmap, decide whether to leave the moderators in one block at the bottom of the logic model or draw lines from each moderator to the logical link it will facilitate or hinder
- Review and affirm or further refine with stakeholders, especially those who will use the logic model
Review and affirm the elaborations of the logic model with stakeholders to ensure it accurately represents the program and the relationships among the components
Create a narrative to go with the logic model. A one-page logic model will not be able to capture all the nuances of the program. The narrative will help explain the components of the logic model and how they work together to accomplish the outcomes. The narrative should include the following:
- An expanded description of the activities, outcomes, and other components of the logic model
- Any key linkages between activities, between activities and outcomes, and between different outcomes
- Attribution v. contribution to outcomes, etc.
- Stakeholder expectations for what will be accomplished, etc.
*Stakeholders are people or organizations that are invested in the program, are interested in the results of the evaluation, and/or have a stake in what will be done with the results of the evaluation. This definition is found in Checklist for Step 1: Engage Stakeholders.
Checklists for Each Step
- Step 1 Checklist: Engage Stakeholders
- ›Step 2 Checklist: Describe the Program
- Step 3 Checklist: Focus the Evaluation
E-mail: cdceval@cdc.gov