February 2018 Meeting Summary

Executive Summary

Adapting Clinical Guidelines for the Digital Age is a multi-stakeholder initiative bringing together experts in creation, informatics, communications, implementation, and evaluation of clinical guidelines to holistically approach improving the uptake of guidelines. Over 200 participants (who attended in person and virtually) helped map out implementable ideas for ways to more easily, quickly, accurately, and consistently apply guidelines in patient care. A post-meeting implementation will test the ideas from the multi-stakeholder group, tracking metrics along the way to demonstrate how well the new process has improved from the status quo in at least one guideline.

Figure 1. Pictorial summary of the linear current state of how guidelines are translated to patient care and the non-linear proposed future state

Figure 1. Pictorial summary of the linear current state of how guidelines are translated to patient care and the non-linear proposed future state, where all perspectives are represented up front and throughout the process of developing and implementing the guideline.


Kaizen, a Japanese word that means “change for the better” or continuous improvement, is a long-term approach that systematically seeks to achieve incremental changes to improve efficiency and quality. Kaizen focuses on large scoped processes to remove waste and maximize value to the customer. During this week-long event, participants mapped out the current process, described an ideal state, identified waste in the current process, and created a future state with an implementation plan and metrics. Kaizen’s techniques are employed with the ultimate customer in mind: the patient.


As a federal agency, the Centers for Disease Control and Prevention (CDC) shares an important role in public health. Producing guidelines that are easily translated, disseminated, communicated, and implemented is critical to improving health care and outcomes. The goal is to package the evolving science in a way that allows for clinicians to easily use and implement evidence-based recommendations. As there is a great responsibility to get the science right, there is an even greater responsibility to help clinicians and patients to use this science. This involves understanding how a guideline can be created in a way that facilitates its use.

The Adapting Clinical Guidelines for the Digital Age initiative is finding ways to help clinicians and patients make the best decisions when they need it. Guidelines are important, but scores of intrusive alerts in the clinical workflow in an attempt to help use the guidelines in practice can result in “alert fatigue” for most clinicians and run the risk of being ignored. The goal is to help clinicians and patients adhere to the recommendations in guidelines based on evolving science. CDC, clinical organizations, and patients will all benefit from these improvements.

To meet this overarching goal, the meeting focused on creating actionable and implementable guidelines that can be easily found and used in a timely manner, with a foundation of evaluation and feedback loops throughout the process. There was an emphasis on clinical decision support (CDS) tools and health IT standards that are currently available to augment patient care and clinical knowledge such as CDS hooks, SMART (Substitutable Medical Applications and Reusable Technologies), Fast Healthcare Interoperability Resources (FHIR), Clinical Quality Language (CQL), and Application Program Interfaces (API).

Adapting Clinical Guidelines for the Digital Age Meeting Summary
Knowledge Level Description Example
 Figure 2. Four levels of knowledge for CDS artifacts
L1 Narrative guideline Published guideline for a specific disease that is written in the format of a peer-reviewed journal article
L2 Semi-structured Flow diagram, decision tree, or other similar format that describes recommendations for implementation
L3 Structured Standards-compliant specification encoding logic with data model(s), terminology/code sets, value sets that is ready to be implemented
L4 Executable CDS that is implemented and used in a local execution environment (e.g., CDS that is live in an EHR production system)

Adapted from: Boxwala, AA, et al.. A multi-layered framework for disseminating knowledge for computer-based decision support. J Am Med Inform Assoc 2011(18) i132-i139.

The Kaizen participants were divided into subgroups, called value streams, according to areas of expertise/background and the need for various perspectives around a particular area of the process. The following value stream groups were created:

  • Guidelines Creation/Summarizing the Evidence Base
  • Informatics Framework for Guideline Translation
  • Translation and Implementation Support
  • Dissemination Tools and Communication Methods
  • Evaluation Framework 

A summary of key ideas and challenges is captured below, including the scope for each group, team members, critical challenges, and next steps. These groups will take the ideas mapped during the meeting and test them by piloting the future state process in each value stream on actual guidelines and real-world scenarios.

General Summary

The following were general themes that could be found across value stream groups or as part of the full group discussions:

  • Need a core team with all related expertise (scientific, informatics, communications, implementation, and evaluation) up front and throughout the process in order to help reduce downstream issues, resource over-utilization, re-work (creating unnecessary cost and lag time), inaccuracies, and inconsistencies
  • Should disseminate guidelines in more structured and actionable formats (L2 as a start, preferably L3)
  • Need to standardize aspects of implementation at the local level that can be the same across organizations (currently, implementation is highly variable across organizations)
  • Need more standardized, centralized modalities (such as repositories) to disseminate guidelines and related artifacts with streamlined communications planned at the outset of guideline development; program-specific websites are not efficient, which ultimately makes them less effective
  • Must consider challenges in effectively implementing and following clinical practice guidelines in daily practice, especially for patients with multiple co-morbidities which compel the need to follow multiple guidelines
  • Plan evaluation at the outset of creating new or updated guidelines and embed evaluation throughout the entire process, monitoring the effectiveness of guidelines and creating feedback loops back to the guideline authors to help evolve the science as the evidence changes
  • Need to remove siloes and variability as much as possible across CDC’s programs, and eventually across all guideline developing organizations, especially within the context of each of the value streams (guideline creation, informatics, dissemination & communication, evaluation)

Participants identified the following challenges that may prevent success in making the desired improvements in the process:

  • Significant variation and lack of standardization across most processes necessitates a larger effort to get to a standardized process
  • Evolving from a linear to a non-linear process creates uncertainty on how to best integrate each of the different value streams into one cohesive process
  • Funding and resources will be a challenge throughout the process
  • Culture change across the industry (and within each institution) may be the most challenging aspect of making these changes industry-wide
  • Competing priorities may impede the ability for this work to be adequately tested and vetted in order to make it an industry standard approach

Summary of Each Value Stream Group

Guidelines Creation/Summarizing the Evidence Base
Purpose: Integrate digitized translation of guidelines with scientific guideline development

Guidelines Creation/Summarizing the Evidence Base
Purpose: Integrate digitized translation of guidelines with scientific guideline development
Scope Start: Identifying the clinical need Scope End: Prepare for an update
In Scope: Selection of work group, what the guidelines will cover, what method is used to evaluate the evidence, identify how to get it implemented Out of Scope: Writing the narrative guideline (NOTE: This would happen concurrently but is not being addressed as part of adapting the guideline for CDS)

Team Members:
Vilma Carande-Kulis, Donald Casey Jr., Joanne Cono, Kathryn Curtis, Sarah Demeke, Laura Fochtmann, Priya Jakhmola, Briana Lucido, Dyann Koffman, Dana Meaney-Delman, Ryan Mullins, Mary Nix, Frank Opelka, Amrita Tailor, Sanjeev Tandon, Per Olav Vandvik, Monique Yohanan.

”Ah-Ha” Moments:

  • Expertise from downstream stakeholders needs to be incorporated at the beginning and throughout the guideline development and implementation process in order to reduce waste and create more effective guidelines
  • Realization that there are variations in the procedures used for guideline creation (e.g., not everyone uses the process that was crafted by the Institute of Medicine)
  • It is important to first ask if there is a need for the guideline and what the benefit will be to the patients/public
  • Consistent and constant monitoring and evaluation needs to occur throughout the process and feedback loops created to inform the need for updates to the guidelines
  • Current tools created to help with the use of guidelines are used inconsistently, sometimes with inaccurate implementations


  • Lack of funding and resources
  • Potential lack of leadership support across CDC and other organizations to create a standardized process
  • Competing priorities with other work

Ideal State:

  • Streamlined, repeatable, and “digitized” digestible guidelines are the standard
  • Artificial Intelligence (AI) machine learning replaces all development processes with automation

Future State:

  • Engage all stakeholders in the initial development of guidelines, such as those with expertise in evaluation, informatics, implementation, and communication
  • Digitize the guidelines by involving informatics at the beginning, with everyone speaking the same language and producing a standardized output that is digestible for the next level of implementation
  • Create a standard way to prioritize and classify guidelines
  • Pilot on actual guidelines to test the new standardized process

Informatics Framework for Guideline Translation
Purpose: Determine how to get from a narrative guideline to a structure, implementable format

Scope Start: Narrative guideline (e.g., PDF) Scope End: Computable CDS artifact
 In Scope: Translation process (standards related to translation); versioning updates; formalisms/formats – API, web services, CQL  Out of Scope: Uptake of guidelines and usability.

Team Members: Paula Braun, Monmi Buragohain, Stephen Downs, Floyd Eisenberg, Margaret Filios, Nedra Garrett, Christina Grasso, Kristen Hagemann, Andrew Hamilton, Stanley Huff, Charles Jaffe, Robert McCready, Bryn Rhodes, Larie Smoyer, Angeline Ti, Sridevi Wilmore, Julia Skapik

”Ah-Ha” Moments:

  • People serving in the various roles needed throughout the process (including guideline developers, informaticians, terminologists, and implementers) should represent those perspectives from the beginning and throughout the process in order for the guideline to be translated using standard frameworks
  • Most of the needed components of the overall process are currently happening, but they are occurring in siloes and further downstream in the process where they are less effectively and efficiently utilized
  • Informatics frameworks could add more value if standardized and identified as part of the guideline creation process – need to determine how much specification and standardization can be done early in the guideline creation process
  • Specifying the guidelines is as complex at the global level as it is at the local level
    • NOTE: “Global” refers to things that are applied to items that are meant to be distributed broadly, such as the guidelines from guideline developing organizations (e.g., CDC, medical societies) or CDS tools specified according to currently accepted standards. “Local” refers to the more specific use of the previously mentioned items, such as guidelines that have been incorporated into a clinical workflow or CDS tools that have been mapped or integrated with a specific organization’s EHR or other devices.
  • Processes for testing and feedback are insufficient in the current state
  • There is a lot of unnecessary redundancy in the current state that could be prevented if certain steps were completed earlier in the overall process
  • Between the future state and current state, the scope looks similar, but the scope got longer in the future state – more process steps in the future state indicate a greater level of standardization occurring, with the intent that downstream value streams such as Translation and Implementation will benefit


  • Attempting to apply informatics in a vacuum, without the perspective of guideline authors as well as implementers
  • Lack of a longitudinal core team that includes all relevant perspectives to review the content as it is developed
  • Failure to pilots the new process on actual guidelines
  • Failure to integrate value streams

Ideal State:

  • Primary stakeholders and a core team involved from the beginning and throughout the process
  • Evaluation done throughout the entire process
  • Feedback to and from clinicians
  • Using the same interoperable infrastructure across guideline developing organizations serves as the foundation for creating standardized digitized guidelines

Future State:

  • Translate a guideline from a narrative document (L1) to a semi-structured L2 artifact, and then developing a structured L3 artifact will help clinical organizations with local translation and implementation (executable L4)
    • Suggestion that at least one CDC pilot develop a SMART on FHIR app as the L3 format supporting the guideline
  • Make decisions earlier in the process to save time downstream, reduce variability, and cost.
    • Once something is built, we make it available.
    • This allows us to get away from more of a waterfall type build to agile development.

Dissemination Tools & Communication Methods
Purpose: Determine the tools most helpful to disseminate digitized guidelines (e.g., CDS tools) and ways to communicate their availability for implementation in practice

Scope Start: When guidelines are cleared, published Scope End: Share communications with identified stakeholders through identified channels
In Scope: Creating communication plans and dissemination. Can look at different forms of communication Out of Scope: Clearance, creating sample data

Team Members: Barry Blumenfeld, Mark Braunstein, Shandy Dearth, Genet Demisashi, Lisa Fatheree, Edwin Lomotan, Titilope Oduyebo, Karen Schoelles, Eileen Storey, Helen Talley-McRae, Hilary Wall, Debra Willis.

”Ah-Ha” Moments:

  • Need a communications and dissemination strategy up front, especially when considering potential updates to guidelines
  • There is tremendous variation within CDC and beyond, with no standard dissemination and communication process across organizations
  • Need to develop a communication plan up front
  • Anticipate that the dissemination strategy may change as the idea of what a guideline is may evolve


  • Challenging to map out a process that is high-level enough to translate across different groups yet detailed enough to be implemented

Ideal State:

  • Automation for updates to the guidelines or to the CDS tools, centered on creating awareness
  • Robust repositories housing all kinds of guidelines and their supporting tools for easier access for implementers
  • Guidelines published with use cases, delineating how the developers expect them to be implemented

Future State:

  • Automate processes, build the software and the tools based on standard communications needs, which would create a communication plan with all the correct elements that are related to each specific guideline
  • Join the guideline creation process at the beginning and develop user stories to understand what tools and methods of communication are necessary to successfully disseminate a guideline
  • Guidelines existing in structured form within a repository could function as both a dissemination point and a platform for user stories
  • Incorporate evaluation and constant feedback loops throughout whole process to allow rapid cycle improvement

Translation and Implementation Support
Purpose: Implement digitized guidelines (e.g., CDS tools) at the local level and standardize (to the degree possible) local implementation steps that all organizations undergo 

Scope Start: Guideline package (whether to disseminate or not)  Scope End: Incorporating guidelines in practice
 In Scope: Piloting translation, creating the tools, understanding end user adoption and CDCs role in implementation support.  Out of Scope: Broad implementation and localized implementation into delivery of services, iterative improvement possibly.

Team Members: Suzanne Beavers, Michelle Dardis, James Doyle, Jon Duke, Randall Elder, Genevieve Luensman, Susan McBride, Blackford Middleton, Nivedita Mohanty, Asim Mujahid, Patrick O’Conner, Sharon Sebastian, Julia Skapik, Dawn Smith, DuWayne Willett, Catherine Nguyen, Barry Blumenfeld

“Ah-Ha” Moments:

  • The process varies depending on the size and resource of each implementing organization
  • The disease specific approach of a guideline does not take into account that a patient may have many diseases—this approach does not help clinicians deal with co-morbidities
  • Translation is being replicated in thousands of organizations, with high costs, diminished accuracy, and inconsistencies in applying the same guideline between organizations
  • Need more standardization to streamline processes and implementation across organizations
  • Evaluation has become a necessary component at every step of the process
  • Future state is much smaller in scope as a result of more of the translation work being incorporated upstream with guideline creation. Localized translation and best practices for CDS implementation at clinical sites is the focus for this value stream


  • Motivation and culture – for the industry and for individual organizations
  • Inadequate workforce
  • Funding
  • Insufficient L3 Package

Ideal State

  • Guideline incorporation is fast, easy, affordable, adaptive, direct, and patient centered

Future State:

  • Take incoming L3 packages and develop how to get them to L4
  • Create a way to measure incoming L3 artifacts to ensure all necessary information needed for local implementation is included
  • Testing in both local and global environments
  • Pilot sites to be time boxed and evaluated with reporting at the end of the pilot

Evaluation Framework
Purpose: Incorporate appropriate evaluation steps throughout each value stream as well as for the overall process in order to gauge how successfully the process is working

Scope Start: At the beginning of the overall process Scope End: Across all the value groups and overall process
In Scope: All groups in the process, evaluating each value stream within the overall evaluation approach. Out of Scope: At guideline incorporation and CDS implementation in a clinical site

Team Members: Valeria Carlson, Timothy Carney, Jennifer Clark, Clay Cooksey, Gema Dumitru, Angela Glotstein, Richard Gregg, Melanie Gwynn, Nancy Habarta, Tara Jatlaoui, Christine Liow, Ira Lubin, Steven Luxenberg, Heather Minnick, David Murphy, Lourdes Guevara, Catherine Staes, Khadija Turay, Maura Whiteman.

”Ah-Ha” Moments:

  • Evaluation should apply to different types of stakeholders, such as guideline developers, vendors, and healthcare professionals
  • Evaluating and tracking metrics on the current state process is limited and therefore difficult to identify waste – conversely, there is a lot of opportunity to broaden evaluation within all the value streams, including evaluating the overall process
  • Evaluation should be integrated within each value stream

Ideal State

  • Include evaluation from the beginning and throughout the guideline process
  • Collect evaluation metrics and data in real time
  • Access feedback from end users including providers, implementers, and patients informing continuous improvements to the guidelines and to the guideline process

Future State:

  • Evaluation planning at the beginning and throughout the process, including checkpoints along the way that determine whether the product (e.g., CDS tool) or process is ready to move to the next step
  • Incorporate feedback loops and evaluate outcomes
  • “Bake in” evaluation to all process components, identifying data that are needed and/or missing in order to evaluate appropriately
  • Create evaluation for the overall process and outcomes in addition to the integrated evaluation throughout each value stream

Senior Leadership Final Remarks – Chesley Richards, Executive Sponsor

Together, we have made enormous progress on a very important topic with mission-oriented focus and skill to impact ease, speed, accuracy, and consistency in adopting guidelines, which ultimately impacts people’s lives. CDC is committed to making the changes planned during this event. In this regard, it is paramount that, in particular, government agencies come together and connect better than we have in the past. The processes and strategies touched on at this Kaizen event present an opportunity to mitigate this disconnect as well as connect with all stakeholders who should be involved. It is clear based on the perspectives of all the participants, the path forward includes:

  • Developing guidelines not only with the scientists but also with experts in informatics, communications, implementation, and evaluation being part of the guideline process from the beginning
  • Creating test beds to trial the guidelines for implementation in patient care
  • Considering more centralized dissemination: CDS that can be located and accessed centrally could significantly help those interested in implementing CDS tools find and execute the tools that can help improve the adoption of guidelines
  • Harnessing evidence generated from applying guidelines in practice is critical – feedback loops that allow data to move and be analyzed in real-time can identify gaps or issues that can be used to improve the guidelines

With the right level of commitment and each stakeholder considering his or her respective roles, these impactful changes are possible.

Anyone interested in joining one of the groups, applying the future state process to their guideline as a “pilot”, or serving as a clinical testing site, please contact Maria Michaels maria.michaels@cdc.gov.

Page last reviewed: February 15, 2019
Content source: