Evaluation of Genomics Applications in Practice and Prevention: Implementation and Evaluation of a Model Approach
Linda A. Bradley and Ieasha T. Edwards
What Is the EGAPP™ Project?
The Evaluation of Genomic Applications in Practice and Prevention (EGAPP™) project is a 3-year model project developed by the Centers for Disease Control and Prevention’s (CDC’s) Office of Genomics and Disease Prevention (OGDP). The goal of this initiative is to support the first phases of a coordinated process for evaluating genetic tests and other genomic applications that are in transition from research to clinical and public health practice in the United States. The EGAPP™ Project aims to draw on existing recommendations for action in the United States (1,2) as well as from knowledge gained from previous CDC initiatives, including the recently completed ACCE Project (3,4). This project will also integrate knowledge from existing processes for evaluation and appraisal (e.g., Agency for Healthcare Research and Quality/U.S. Preventive Services Task Force, and the CDC Task Force on Community Preventive Services) .
The ACCE Project
Conducted by the Foundation for Blood Research under a cooperative agreement with CDC’s Office of Genomics and Disease Prevention (OGDP), the ACCE project proposed and tested a model process for collecting, evaluating, interpreting, and reporting data about DNA and related testing for disorders with a genetic component. An important aspect of this process was the identification of gaps in knowledge.
ACCE takes its name from the four previously defined components of evaluation: Analytic validity; Clinical validity; Clinical utility; and Ethical, legal, and social implications (1,2). The evaluation process is based on an analytic framework of more than 40 targeted questions that establish the specific clinical disorder being evaluated, the test(s) to be used, and the setting in which the testing will be conducted (e.g., primary iron overload in adults using HFE testing in the setting of population screening).
For more information on the ACCE Project, see Chapter 5, ACCE Reviews of Genetic Tests: BRCA1, BRCA2 and CFTR.
Why Is Evaluation of Emerging Genomic Applications a Public Health Issue?
The success of the Human Genome Project has led to increasingly rapid translation of genomic information into clinical applications. Genetic tests have been developed for approximately 1,100 diseases, and more than 800 disorders are currently available for clinical testing (5). Although most genetic testing is used for diagnosing rare genetic disorders, a growing number of genetic tests have population-based applications, including carrier identification, predictive testing for inherited risk for common diseases, and pharmacogenetic testing for variation in drug response. These tests and other anticipated applications of genomic technologies for use in screening and prevention have the potential for broad public health impact.
Consumers, health professionals and government advisory groups have raised issues about the current status of genetic-testing implementation and oversight, including the need to develop evidence to establish efficacy and cost-effectiveness before tests are commercialized (1,2,6-8). In addition, as consumers’ interest in and demand for new genomic technologies continues to rise, the need for timely and reliable information becomes increasingly crucial. This information will enable health care providers, payers (insurers), consumers, and policy makers to decide which tests are safe and effective and to ensure that they are used appropriately. Expert panels, professional organizations, and clinical experts (e.g., Task Force on Genetic Testing, Secretary’s Advisory Committee on Genetic Testing) have produced recommendations on the development of genetic tests (1,2,6-8). However, a coordinated approach has not yet been developed for effective translation of genomic implications into clinical practice and health policy and for post market monitoring.
EGAPP™ Working Group
The EGAPP™ Project has established an independent, non-federal Working Group composed of experts from fields including health care, epidemiology, genomics, public health, laboratory practice, and evidence-based medicine. Key roles of the working group include:
- Consider input from stakeholders and experts, develop criteria, and prioritize and select topics for evidence-based review.
- Establish methods and process for evidence reviews.
- Oversee expert and peer review of commissioned evidence reports.
- Develop conclusions or recommendations based on the evidence.
- Provide advice about other project activities, such as pilot data collection studies and project evaluation.
A key objective is to identify, engage, and continuously involve a wide range of stakeholders in the project. Early participation of stakeholders is needed to:
- Suggest priority topics for review.
- Provide input on the content and format of information that is needed and useful from the stakeholders’ different perspectives.
- Contribute technical expertise.
In later stages of the project, stakeholder groups will have an important role in developing informational messages targeting specific audiences, based on the evidence reviews and the Working Group’s conclusions/recommendations. For this model project, the primary target audiences are health care providers and consumers, and the key secondary audiences are policy makers and health care purchasers and payers. The stakeholder groups will also provide important feedback on the value and impact of the evidence reports and informational messages developed.
Other EGAPP™ Activities
In addition to establishing the EGAPP™ Working Group and involving stakeholders, project activities will include:
- Supporting evidence-based reviews performed by expert groups on topics selected by the Working Group.
- Supporting pilot data collection studies to provide needed data on issues such as utilization, access, performance in practice, or the resolution of specific identified gaps in knowledge.
- Implementing a comprehensive plan for evaluating the EGAPP™ Project.
- Considering mechanisms to sustain an ongoing systematic process for evaluation of genomic applications.
This project will focus on tests that have the potential for broad application and public health impact (e.g., population screening, tests for guiding clinical intervention). The large number of genetic tests used for diagnosing rare, single-gene disorders are less likely to be reviewed; however, the methods and standards developed for EGAPP™ reviews may be of interest to groups focused on improving access to quality genetic tests for rare disorders. For more information on this topic, see Chapter 8, Enhancing Genetic Testing for Rare Diseases: Improving Availability, Access, and Quality.
Expert Meeting on Evidence-Based Review of Genomic Applications
On January 24-25, 2005, CDC hosted an invitational conference that brought together 21 experts in the areas of evidence-based medicine, health care, genomics, health technology assessment, epidemiology, ethics, and health economics from the United States, Canada, and the United Kingdom. Participants also represented various professional settings, including public health, academia, government agencies, the U.S. Preventive Services Task Force and the Community Preventive Services Task Force, clinical and laboratory practice, industry, and regulation. The participants considered both existing and potential approaches and methodologies for systematic evaluation of genetic tests and other genomic applications as well as reviewed lessons learned by existing programs and projects that conduct systematic reviews.
Why Might EGAPP™ Project Activities Interest State and Local Public Health Professionals?
The EGAPP™ Project plans to engage state and local public health professionals and involve them in the evaluation process. For example, these important stakeholders can help to identify emerging genetic tests and technologies for which accurate and objective data are most acutely needed in order to inform appropriate use and policy development. The stakeholders may also contribute to other proposed activities, such as design and dissemination of information summaries to target audiences, pilot studies to collect data on test utilization and performance in practice, and efforts to develop public-private partnerships.
How Will Success Be Measured?
Comprehensive evaluation of the process, products, and impact is fundamental to this model project. Outcomes of interest include the ability to establish and support a transparent and publicly accountable process for systematic evaluation, engage stakeholders, and develop and maintain effective partnerships and collaborations. This project will also seek feedback on the quality and usefulness of products, such as evidence reports and summaries; working group conclusions/recommendations; targeted informational messages; and post-market data on quality, acceptability, or utilization of tests. Impact will be assessed through feedback from the stakeholders on project awareness, use and value of information, success of dissemination, and measurable changes in practice, policy, or reimbursement/coverage decisions.
- National Institutes of Health. Final Report of the National Institutes of Health - Department of Energy Task Force on Genetic Testing [online] 1997 [cited 2005 Mar 23]. Available from URL: http://www.genome.gov/10001733.
- Enhancing the Oversight of Genetic Tests: Recommendations of the SACGT [online] 2001 [cited 2005 Mar 23]. Available from URL: http://oba.od.nih.gov/SACGHS/sacghs_documents.html#GHSDOC_009.
- Haddow JE, Palomaki GE. ACCE: a model process for evaluating data on emerging genetic tests. In: Khoury MJ, Little J, Burke W, editors. Human Genome Epidemiology. New York (NY): Oxford University Press, Inc.; 2003:217-33.
- ACCE model system for collecting, analyzing and disseminating information on genetic tests [online] 2005 [cited 2005 Mar 23]. Available from URL: http://www.cdc.gov/genomics/gtesting/ACCE/fbr.htm.
- GeneTests Web site [cited 2005 Mar 23]. Available from URL: http://www.genetests.org/ .
- Secretary’s Advisory Committee on Genetics, Health and Society (SACGHS). Informational Briefings [online] 2003-04 [cited 2005 Mar 23]. Available from URL: http://oba.od.nih.gov/SACGHS/sacghs_home.html.
- Burke W, Atkins D, Gwinn M, Guttmacher A, Haddow J, Lau J, et al. Genetic test evaluation information needs of clinicians, policy makers and the public. Am J Epidemiol 2002; 156:311-18.
- Burke W. Genetic testing. N Engl J Med 2002; 347:1867-75.