Draft Genetic Test Review

---

Cystic Fibrosis
Clinical Utility

(347KB)

---

CLINICAL UTILITY

Question 26: What is the natural history of the disorder?
Question 27: What is the impact of a positive (or negative) test on patient care?
Question 28: If applicable, are diagnostic tests available?
Question 29: Is there an effective remedy or acceptable action, or other measurable benefit?
Question 30: Is there general access to that remedy or action?
Question 31: Is the test being offered to a socially vulnerable population?
Question 32: What quality assurance measures are in place?
Question 33: What are the results of pilot trials?
Question 34: What health risks can be identified for follow-up testing and/or intervention?
Question 35: What are the financial costs associated with testing?
Question 36: What are the economic benefits associated with actions resulting from testing?
Question 37: What facilities/personnel are available or easily put in place?
Question 38: What educational materials have been developed and validated, and which of these are available?
Question 39: Are there informed consent requirements?
Question 40: What methods exist for long term monitoring?
Question 41: What guidelines have been developed for evaluating program performance?

---

CLINICAL UTILITY

Question 38: What educational materials have been developed and validated and which of these are available?

Summary The components of educational materials for both providers and patients have been published Approximately 5-10 minutes of provider time was spent in direct educational contact with the patients Four examples of patient educational material have been reviewed and found to satisfy most, but not all, content criteria rate either ‘adequate' or ‘superior' using objective readability and presentation standards

Introduction
Prenatal screening program need educational materials appropriate for two separate but overlapping audiences: providers and patients. Because screening will be implemented at primary care settings, where expertise in genetics and genetic testing may be limited it is important that providers have access to information about how to offer and interpret prenatal cystic fibrosis testing. In addition, they should be able to answer most of the patient questions that will arise. There also needs to be access to expert genetic resources in the event that difficulties arise. Equally important, there need to be high quality, validated educational materials for patients to help them make informed decisions.

Provider education
The Secretary's Advisory Committee on Genetic Testing (SACGT) has recently suggested a genetic test template to inform and educate health professionals (Federal Register, 2000). The seven elements are summarized in Table 4-25. The first six of these are likely to be similar in any laboratory offering prenatal cystic fibrosis testing, except for minor differences in the laboratory procedures. The seventh component, billing and reimbursement, is laboratory-specific. Answers to some of the elements are not yet readily available (e.g., analytic versus clinical laboratory performance).

Table 4-25. Components of a Genetic Test Information Sheet for Health Care Providers

Component	Description
Purpose of the test	The purpose and setting for testing (e.g. predictive, newborn)
Clinical condition	Description including prevalence, manifestations and prognosis
Definition of test	Specific laboratory measurements
Analytical validity	How accurately the test measures the intended analyte
Clinical validity	Accuracy with which measurement predicts clinical condition
Clinical utility	Contribution of the test results to improved outcome
Cost of testing	Costs, billing information and reimbursement policies (e.g. CPT code)

Developing educational materials for health care providers
At least 13 centers have developed educational materials of this type as part of pilot studies of prenatal cystic fibrosis screening (Question 35). It would be helpful if these materials were to be reviewed before any new material is created. For providers, educational material should cover at least the first six components of the SACGT genetic test template. Reviewers should be drawn from professional organizations and work with professional educators involved in adult education and continuing education for health care professionals. If existing materials do not meet the SACGT criteria, laboratories could create and make available their own provider education materials. Any such newly created materials need to be formally validated. Methods of validation include: 1) the use of focus groups, and 2) field-testing the educational materials with health care providers and their staffs, followed by evaluative surveys. Such surveys should be designed and carried out by professional evaluators. The American College of Obstetricians and Gynecologists has recently produced a Clinical and Laboratory Guideline on this subject for its members (ACOG Guidelines, 2001). The American College of Medical Genetics has also published Laboratory Standards and Guidelines on this topic and much of that information would be useful in provider educational materials (Grody et al., 2001).

 

Gap in Knowledge: Content of Provider Educational Material. Missing in Table 4-25 are other laboratory-specific instructions to the health care provider. Examples of additional provider education topics are listed in Table 4-26. Many laboratories include this type of information as part of a ‘Resource Guide', and it is often covered again in face-to-face encounters between laboratory outreach personnel and primary care providers (e.g., rounds, seminars and office visits). Lack of knowledge about, or failure to follow, these instructions is a major cause of pre-analytic errors identified in studies of prenatal screening for cystic fibrosis (Dequeker et al., 2000).

 

Table 4-26. Other Laboratory-specific Information that Ought to be Included in Provider Education

Component	Description
Sample collection	Description of sample type, volume, and labeling requirements
Shipping instructions	Packaging and addressing
Time requirements	Type of shipper, turn-around-time
Requistion slip	Description, ordering and justification of fields
Consent	Who is responsible fo collecting and maintaining consent
Reporting	How reports are transmitted and interpreted. Model used.
Confidentiality	How it will be maintained
Certification	Certifications/licenses for specific genetic testing
Further testing	What further testing might be suggested/required
Urgent Action	What results require immediate physician contact

The teachable moment
For both providers and patients, effective provision of information involves two critical elements: timing and types of materials. Effective timing makes use of the “teachable moment”, that point in a learning experience when the learner is more receptive to accepting and using new information, accepting new attitudes, or learning new skills (Woods et al., 1996). Fr the health care provider, the teachable moment comes at the time when an issue arises with a patient. For that reason, written material that the provider can use at the time of the patient visit are far more effective than didactic sessions that may take place days, weeks, or even longer before the issue actually arises with the patient. For the patient, that moment occurs when required information is provided precisely at the time it is needed to help that patient make a decision (Leist et al., 1990). It is, therefore, important that appropriate educational/informational materials be readily available to providers and patients at the time when the materials will be most useful.

Patient education
Educating pregnant women about prenatal screening for cystic fibrosis has been described as providing relatively complex genetic information to large populations in order that couples make informed choices. In reality, the aim of patient educational is more limited. Several key points need to be clearly stated so that those involved feel that they can make an informed decision about whether or not they want to have their pregnancy screened. In addition, it is important to ensure that ample time is available for questions and answers between the provider and patient, without placing a burden upon the office. Table 4-27 summarizes data from published pilot studies showing the type of patient educational materials and average time needed for transmitting the information. All studies utilized printed pamphlets and most also reported verbal interactions with the health provider/staff. Two studies also included videotapes. Most providers found that an average of 5-10 minutes was spent in direct educational contact (e.g., time spent reading the pamphlet or watching the video were not included).

Table 4-27. Summary of Patient Educational Materials Used in Prenatal Cystic Fibrosis Pilot Studies

Screening Center	Pamphlet/ Brochure	Videotape	Verbal Explanation/ Questions	Estimated Minutes Spent
Edinburgh, Scotland	√		√	NR
Copenhagen, Denmark	√			NR
Manchester, England	√		√	10
Oxford, England	√			NR
East Berlin, Germany	√		√	NR
Maine, USA	√		√	10
Aberdeen, Scotland	√		√	NR
Rochester, NY	√		√	5
N California, USA	√	√	√	10
Leeds, England	√		√	11
Milan, Italy	√		√	10-15
Los Angeles, USA	√	√	√	30
New York, USA	√		√	3-51

NR – Not Reported
¹Group counseling session for 12 couples lasted 40 minutes

Evaluation of patient and provider educational materials
Several formal evaluations of existing written patient and provider educational materials for genetic testing have been reported (Loeben et al., 1998). The materials were evaluated for content, using 10 criteria for minimal content drawn from published recommendations by policy-making bodies (Table 4). The content overlaps with that presented in Table 1. The authors analyzed 115 pamphlets (20 of these were for cystic fibrosis testing), most of which (65 percent) were developed by the organizations offering testing, or in collaboration with other groups (27 percent). The remainders were from disease-specific support groups or the March of Dimes. As shown in Table 4-28, most of the materials failed to address many of the criteria listed. The authors concluded that the patient pamphlets were more complete and that provider materials seemed more aimed at identifying patients for testing than for providing information to evaluate the tests. It was not possible to obtain separate information for the 20 cystic fibrosis pamphlets from the published report. Little, if any, information was available about reading level or patient response to the material.

Table 4-28. Criteria for Content Analysis of Genetic Test Educational Materials

Criterion	Description	Specified Pamphlet Adresses Criteria (%)
		Provider	Patients
Intended purpose	Screening, diagnostic or predictive testing	28	36
Test performance	Sensitivity, specificity and predictive values	38	54
Risks, limits, benefits	Medical or social benefits or risks	7	13
Rights of patients	Voluntary nature, confidentiality, consent	3	27
Candidates for testing	Any medical or family history criteria	52	77
Description of condition	Symptoms, characteristic and incidence	45	100
Genetic counseling	Counseling is available or necessary	52	67
Interpretation of test	Risk of disease for positive and negative results	21	36
Treatment options	Treatment, prevention or medical management	7	62
Cost	Cost to the patient	52	31

Availability of patient education material
Patient educational materials are already available from several sources, including some that have been published as part of reports of prenatal cystic fibrosis pilot studies (Mennie et al., 1992; Loader et al., 1996). Other patient educational materials are available on-line. In addition to these, two patient educational pamphlets have recently been produced by the American College of Obstetricians and Gynecologists and the American College of Medical Genetics. These pamphlets are available in English only and are now available for purchase.

Objective methods for evaluating patient educational materials
Cystic fibrosis screening programs should be aware that it is not ordinarily necessary to develop patient education materials from scratch. Several evaluation tools have been developed that will allow assessment of existing materials and provide suggestions for improving them, if necessary. In evaluating materials, two critical elements need to be considered: content and patient comprehension. Among the published guidelines for evaluation are Loeben's criteria for content analysis of educational materials (Loeben et al., 1998), listed in Table 4-28 above; and Suitability Assessment of Materials (SAM) for evaluation of materials for patient comprehension (Doak et al., 1995). Because the average literacy level of the population in the United States is between the 6^th and 8^th grades, one of the most important considerations in evaluating materials is reading level. Simple and clear language is necessary so that the message is understood. SAM criteria address this, along with other factors, including purpose, scope, inclusion of summary, behavior-oriented content, reading grade level, use of active voice and commonly used words, cover graphic that shows purpose, simple line drawings, relevance of illustrations, lists, explanation of tables, captions used for graphics, material layout factors, typography, number of items under subheadings, and cultural appropriateness. The SAM evaluation takes 45 minutes and provides a numerical measure of a document's suitability. There are many published tools for estimating the ‘readability' of written materials including the widely used and accepted Fry formula (Fry, 1977). There are also several computerized formulae, such as the Fog Index, the Flesch Reading Ease (Tefki, 1987) or the Flesch Grade Level. Although most of the educational materials used in the prenatal cystic fibrosis pilot studies were originally written in English, alternative languages and cultural approaches should be considered, depending on the population being served. It is always important to involve consumers in the planning and review of patient educational materials (Myers et al., 1994).

Table 4-29 gives summary ratings of four available sets of patient materials. The ratings for the materials are based on the Loeben, et al. (1998) and SAM guidelines. A SAM “superior” rating indicates that the materials satisfy 70-100 percent of the SAM criteria, an “adequate” rating satisfies 40-69 percent, and a “not suitable” rating satisfies 0-30 percent of the criteria.

Table 4-29. Summary Ratings for Four Available Sets of Patient Educational Materials

Material	Content Criteria	Sam Rating
Cystic Fibrosis Carrier Testing: The Decision is Yours (ACOG, 2001)	Meets most (1)	Adequate (5,7)
Cystic Fibrosis Testing: What Happens if Both My Partner and I Are Carriers (ACOG, 2001)	Meets most (2,3,4)	Adequate (6,7)
Cystic Fibrosis: It's Your Choice (Loader et al., 1991)	Meets most (2)	Superior
Information About Cystic Fibrosis (CF) and CF Carrier Testing (Clayton et al., 1995)	Meets most (1,2,4)	Superior

Notes

1. does not address test performance
2. does not address confidentiality or consent
3. does not address availability of counseling
4. does not address cost to the patient
5. literacy demand is somewhat high
6. literacy level is above grade 9
7. few or no illustrations, graphics not relevant or not explained

Readability scores are based on the Fry Readability Graph (Fry, 1977).

In another published review that concentrated on 28 provider and/or patient pamphlets containing educational materials directed at cystic fibrosis testing, the authors concentrated on three content areas: description of cystic fibrosis, life expectancy and reproductive options (Loeben et al.; 1998). Each sentence in the pamphlet was classified as expressing “Optimism/Hope”, “Neutral” or “Pessimism/Caution”. The results were stratified based on geography (United States and the United Kingdom), the type of laboratory (non-commercial and commercial), and whether the setting was prenatal or other. There was a wide variation in the number of sentences describing cystic fibrosis (median 6.5, range 1 to 37) with neutral sentences being far more common than negative or positive ones. Fewer sentences dealt with life expectancy (median 4, 0-7) and in the UK and US commercial pamphlets, positive sentences were rare. When a carrier couple was identified, most (24/28) discussed prenatal diagnosis, but only about half (15/28) mentioned selective termination (or a similar term). Termination was mentioned more often in pamphlets from the UK than from the US (8/9 versus 7/19). In the United States, termination was never mentioned by commercial laboratories (0/7) but by more than half of non-commercial laboratories (7/12). The authors concluded that for information to be truly ‘balanced', it should contain both positive and negative comments intermixed with neutral comments.

Other evaluation criteria
Because the average literacy level of the population in the United States is between the 6^th and 8^th grade, it is important to consider reading level for patient materials. Simple and clear language is necessary so that the message is understood. The Suitability Assessment of Materials (SAM), a well-tested evaluation system for health-related educational materials for patients, is now in widespread use. It provides ratings based on 22 factors in six categories: content, literacy demands, graphics, layout and typography, learning stimulation and cultural appropriateness (Doak et al., 1995). The evaluation takes 45 minutes and provides a numerical measure of a document's suitability. Other tools are available for calculating ‘readability' of written materials, the most highly recommended and widely accepted of which is the Fry Formula (Fry, 1977). There are also several computerized formulae such as the Fog Index, the Flesch Reading Ease (Tefki, 1987) or the Flesch Grade Level. In addition to literacy level, it is important that patient materials be made available in languages other than English. Although most of the materials used in the pilot trials were originally written in English, alternative languages should be considered, depending on the population being served. It is always important to involve consumers in the planning and review of patient educational materials (Myers et al., 1994).

References

Cho MK, Arruda M, Holtzman NA. 1997. Educational material about genetic tests: Does it provide key information for patients and practitioners? Am J Med Genet 73:314-320.

Clayton EW, Hannig VL, Pfotenhauer JP, Parker RA, Campbell PW 3^rd, Phillips JA 3^rd. 1995. Teaching about cystic fibrosis carrier screening by using written and video information. Am J Hum Genet 57:171-181.

Dequeker E, Cuppens H, Dodge J, Estivill X, Goossens M, Pignatti PF, et al. 2000. Recommendations for quality improvement in genetic testing for cystic fibrosis. European Concerted Action on Cystic Fibrosis. Eur J Hum Genet 8 (suppl 2):S2-24.

DeQueker E, Cassiman J. 2000. Genetic proficiency testing in diagnostic laboratories – quality contol is the message. Am J Hum Genet 67:a247.

Doak C, Doak L, Root J. 1995. Teaching patients with low literacy skills. JB Lippincott.

Genetic testing under the clinical laboratory improvement amendments. 2000. Federal Register;65:25928-25934

Fry E. Fry's readability graph: clarifications, validity, and extensions to level 17. J Read 1977:242-252.

Grody WW, Cutting GR, Klinger KW, Richards CS, Watson MS, Desnick RJ. 2001. Laboratory standards and guidelines for population-based cystic fibrosis carrier screening. Genet Med 3:149-154.

Leist JC, Kristoco RE. 1990. The changing paradigm for continuing medical education: impact of information on the teachable moment. Bull Med Libr Assoc 78:173-179.

Loader S, Caldwell P, Kozyra A, Levenderon JC, Boehm CD, Kazazian HH et al. 1996. Cystic fibrosis carrier population screening in the primary care setting. Am J Hum Genet 59:234-247.

Loeben GL, Marteau TM, Wilfond BS. 1998. Mixed Messages: Presentation of information in cystic fibrosis-screening pamphlets. Am J Hum Genet 63:1181-1189.

Mennie ME, Liston WA, Brock DJH. 1992. Prenatal cystic fibrosis carrier testing: designing an information leaflet to meet the specific needs of the target population. J Med Genet 29:308-312.

Myers MF, Bernhardt BA, Tambor ES, Holtzman NA. 1994. Involving consumers in the development of an educational program for cystic fibrosis carrier screening. Am J Hum Genet 54:719-726.

Wilfond BS, Fost N. 1992. The introduction of cystic fibrosis carrier screening into clinical practical: Policy considerations. Milbank Quarterly 70:629-659.

Secretary's Advisory Committee on Genetic Testing. Federal Register 65:77631-77633.

Tekfi, Chafai (1987) Readability Formulas: An Overview, J Docum 43:261-73.

Woods P, Jeffrey B. Teachable moments. Open University Press, 1996.

The Fog Index can be computed at www.fpd.finop.umn.edu/groups/ppd/documents/information/writting_tips.cfm