Draft Genetic Test Review

---

Cystic Fibrosis
Analytic Validity

(1.14MB)

---

ANALYTIC VALIDITY

Question 8: Is the test qualitative or quantitative?
Question 9: How often is a test positive when a mutation is present?
Question 10: How often is the test negative when a mutation is not present?
Question 11: Is an internal QC program defined and externally monitored?
Question 12: Have repeated measurements been made on specimens?
Question 13. What is the within- and between-laboratory precision?
Question 14: If appropriate, how is confirmatory testing performed to resolve false positives in a timely manner?
Question 15: What range of patient specimens has been tested?
Question 16: How often does the test fail to give a useable result?
Question 17: How similar are results obtained in multiple laboratories using the same, or different, technology?

---

ANALYTIC VALIDITY

Question 11: Is an internal QC program defined and externally monitored?

Definition
Internal quality control is a set of laboratory procedures designed to ensure that the test method is working properly. An internal quality control program includes documentation that high standards are being practiced to ensure that:

• reagents used in all aspects of genetic testing are of high quality to allow successful test completion,
• all equipment is properly calibrated and maintained,
• good laboratory practices are being applied at every level of genetic testing. To the extent possible, all steps of the testing process must be controlled.

Quality control procedures
Techniques that are used for analyzing DNA in prenatal screening for cystic fibrosis are the same as those used for other molecular testing. These techniques are widely applied and well understood. As a result, it has been possible to design and publish generic internal quality control procedures, which many molecular laboratories already have in place. Table 2-8 lists published guidelines that, among other topics, describe reagent quality control, equipment calibration and maintenance, education of the technical staff, and other internal quality control procedures. The purpose of the quality control procedures is to rigorously control all steps of the DNA testing process to minimize the potential for test failure. Given that the internal procedures for establishing and maintaining good laboratory practice are readily available (Neumaier et al., 1998), the important next step will be to encourage, assist, and require laboratories to apply and document appropriate quality control procedures.

Table 2-8. Guidelines, Recommendations, and Checklists that Address Internal Quality Control Issues and Requirements.

Guidelines, Recommendations and Checklists	Source / Reference
Clinical Laboratory Improvement Amendments of 1988	Federal Register 1992;57:7002-3
Genetic Testing Under CLIA	Federal Register 2000;65: 25928-24934
New York State Laboratory Standards (9/00)	http://www.wadsworth.org/labcert/clep/Survey/standardsmenu.htm
Molecular Diagnostic Methods for Genetic Diseases: Approved Guidelines	National Committee for Clinical Laboratory Standards MM1-A Vol 20 #7
College of American Pathologists Checklist	www.cap.org
Standards and Guidelines for Clinical Genetics Testing	American College of Medical Genetics http://genetics.faseb.org/cgi-bin/acmgm/rd.pl?pgm=2
European Concerted Action on Cystic Fibrosis	(BMH-4-CT96-0462)
Laboratory Standards and Guidelines for Population-based Cystic Fibrosis Carrier Screening	Grody, et al. 2001. Genet Med 3:149-154
Technical Standards and Guidelines for Cystic Fibrosis	Supplement to the ACMG Standards and Guidelines for Clinical Genetics Laboratories (in preparation)

 

External monitoring
All clinical laboratories performing genetic testing must comply with general regulations under the Clinical Laboratory Improvement Amendments (CLIA) and a CLIA certification should be considered the minimum acceptable level of external monitoring. One shortcoming of having only a CLIA certification is that CLIA inspectors often have less experience in evaluating genetic testing laboratories than other certifying organizations. CLIA is in the process of upgrading its regulations regarding genetic testing. The Task Force on Genetic Testing concluded that the current CLIA requirements are insufficient to ensure quality of molecular genetic testing. Laboratories certified by the College of American Pathologists or by New York State Health Department will have undergone a more rigorous external monitoring that requires specific procedures and documentation.

The need for positive cystic fibrosis assay controls
The issue of positive controls deserves particular attention. Positive controls for the standard 25 cystic fibrosis mutations must be utilized to validate the assay and for each lot of reagents. These controls (or a subset of positive controls) are recommended to be routinely included in each assay run. However, obtaining these positive controls can be difficult. Not all of the 25 recommended cystic fibrosis mutations (Grody et al., 2001) are readily available through the American Type Culture Collection (ATCC, Rockville, MD www.atcc.org) or the Coriell Institute for Medical Research (Camden, NJ http:://arginine.umdnj.edu) repositories. Thus, laboratories are left to find another source for some of the positive controls. The following are four possible sources for obtaining positive controls:

• Asking colleagues who possess validated DNA aliquots of one or more of the cystic fibrosis mutations to share a portion of their material ("clone by phone"). For such sharing to take place, consent issues may need to be considered
• Synthesizing positive controls based on sequence alterations, using well designed primers, and then validating these synthetic controls using sequence analysis (sequence analysis is the "gold standard").
• Encouraging commercial reagents manufacturers to produce sets of validated positive controls for all of the 25 recommended cystic fibrosis alleles and provide them to laboratories using their reagents.
• Encouraging a foundation (e.g., the Cystic Fibrosis Foundation), a professional organization (e.g., the College of American Pathologists), or a governmental organization (e.g., the Centers for Disease Control and Prevention) to support an initiative aimed at developing and distributing positive controls to all testing laboratories.

The first two sources listed above are currently the only available options for laboratories involved with cystic fibrosis testing, and both have important drawbacks and limitations. There is a limit, for example, to how much DNA an investigator should be asked to provide. Even when benevolent investigators are found, stumbling blocks may be encountered in fulfilling the requirements of institutional materials transfer agreements or in obtaining Institutional Review Board (IRB) approvals. The second approach listed is also limited, as it requires a greater development effort than many laboratories could manage. The third approach has appeal, since manufacturers (including those supplying Analyte Specific Reagents) must validate their product prior to introduction and must, therefore, already have access to all necessary positive controls. Lastly, organizations such as those listed could embark on a coordinated initiative aimed at developing and distributing cystic fibrosis positive controls to testing laboratories.

References
Grody WW, Cutting GR, Klinger KW, Richards CS, Watson MS, Desnick RJ. 2001. Laboratory standards and guidelines for population-based cystic fibrosis carrier screening. Genet Med 3:149-154.

Holtzman NA, Watson MS. 1997. Promoting Safe and Effective Genetic Testing in the United States. Final report of the Task Force on Genetic Testing. http://www.nhgri.nih.gov /ELSI/TFGT_final/, pp1-72.

Neumaier M, Braun A, Wagener N. 1998. Fundamentals of quality assessment of molecular amplification methods in clinical diagnosis. International Federation of Clinical Chemistry Scientific Division Committee on Molecular Biology Techniques. Clin Chem 44:12-26.

---

ANALYTIC VALIDITY

Question 12: Have repeated measurements been made on specimens?

Measurements made on the same specimen in different laboratories
Multiple laboratories have made repeated measurements on the same specimen, utilizing a variety of technologies. A collaborative external proficiency testing program, jointly administered by the American College of Medical Genetics and the College of American Pathologists (ACMG/CAP) provides up to six cystic fibrosis DNA challenges each year, along with a summary report of the results. An earlier section in Analytic Validity (Questions 12 and 13) provides more details about the results of this program. In the first distribution of 2000, 32 of 36 participating laboratories (89 percent) correctly reported the results for all three challenges. Two of the challenges were wild-type and one was heterozygous for a cystic fibrosis mutation; all 36 laboratories correctly classified the two wild-type specimens. In 1999, 42 of 43 participating laboratories (98 percent) correctly classified two specimens heterozygous for a common cystic fibrosis mutation, and two specimens (one heterozygous and one homozygous) for another mutation. In 1998, 51 laboratories reported their analyses of three cystic fibrosis challenges, one homozygous and two compound heterozygous specimens. The correct response rates were 88 percent, 92 percent, and 100 percent, respectively. Although there may be some explanations for the poor between-laboratory performance on some challenges (difficulties with distinguishing delI507 and delF508 have been clearly documented), these data indicate that there is also unexplained variability between laboratories when measuring the same specimen. In summary, the between-laboratory replication of a single specimen is between 90 percent and 98 percent, when detectable mutations are present.

Measurements made repeatedly on the same sample within a laboratory
It is common practice for repeated measurements to be made on the same specimen (a control specimen) within a laboratory. For each assay, a positive control is usually included for testing. This internal documentation will remain within the laboratory but will be available for on-site inspections by certifying agencies. Thus, one avenue for collection of these data would again be to use laboratory survey instruments. In one laboratory offering prenatal screening for cystic fibrosis (Knight GJ, Foundation for Blood Research, ME, personal communication), two multi-mutation controls (three in one control; two in the other) were run on 60 consecutive clinical assays in 2000 and early 2001. Overall, two failures occurred (both were the three-mutation control) for a failure rate of 2.0 percent (6/300). In another laboratory offering testing for cystic fibrosis (Roa B, Baylor University Medical Center, personal communication), the delF508 control did not fail in 52 assays in 2000. Nearly all laboratories will have these data available, even though they may not be routinely collated and analyzed.