Draft Genetic Test Review

---

Hereditary Hemochromatosis
Analytic Validity

(119KB)

---

ANALYTIC VALIDITY

Question 8: Is the test qualitative or quantitative?
Question 9: How often is a test positive when a mutation is present (analytic sensitivity)?
Question 10: How often is the test negative when a mutation is not present (analytic specificity)?
Question 11: Is an internal QC program defined and externally monitored?
Question 12: Have repeated measurements been made on specimens?
Question 13. What is the within- and between-laboratory precision?
Question 14: If appropriate, how is confirmatory testing performed?
Question 15: What range of patient specimens has been tested?
Question 16: How often does the test fail to give a useable result?
Question 17: How similar are results obtained in multiple laboratories using the same, or different, technology?

---

ANALYTIC VALIDITY

Question 11: Is an internal quality control program defined and externally monitored?

Definition
Internal quality control is a set of laboratory procedures designed to ensure that the test method is working properly. An internal quality control program includes documentation that high standards are being practiced to ensure that:

• reagents used in all aspects of genetic testing are of high quality to allow successful test completion,
• all equipment is properly calibrated and maintained,
• good laboratory practices are being applied at every level of the genetic testing process

Quality control procedures
Techniques that are used for analyzing DNA in screening for HHC are the same as those used for other molecular testing. These techniques are widely applied and well understood. As a result, it has been possible to design and publish generic internal quality control procedures, which many molecular laboratories already have in place. Table 2-12 lists published guidelines that, among other topics, describe reagent quality control, equipment calibration and maintenance, education of the technical staff, and other internal quality control procedures. The purpose of the quality control procedures is to rigorously control all steps of the DNA testing process to minimize the potential for test failure. Given that the internal procedures for establishing and maintaining good laboratory practice are readily available (Neumaier et al., 1998), the important next step will be to encourage, assist, and require laboratories to apply and document appropriate quality control procedures.

Table 2-12. Guidelines, Recommendations, and Checklists that Address Internal Quality Control Issues and Requirements.

Guidelines, Recommendations and Checklists	Source / Reference
Clinical Laboratory Improvement Amendments of 1988	Federal Register 1992;57:7002-3
Genetic Testing Under CLIA	Federal Register 2000;65: 25928-24934
New York State Laboratory Standards (9/00)	http://www.wadsworth.org/docs/clrs.shtml (last accessed 1/2008)
Molecular Diagnostic Methods for Genetic Diseases: Approved Guidelines	National Committee for Clinical Laboratory Standards MM1-A Vol 20 #7
College of American Pathologists Checklist	www.cap.org
Standards and Guidelines for Clinical Genetics Testing	American College of Medical Genetics http://genetics.faseb.org/cgi-bin/acmgm/rd.pl?pgm=2
Technical Standards and Guidelines for Hereditary Hemochromatosis	Supplement to the ACMG Standards and Guidelines for Clinical Genetics Laboratories (in preparation by QA Committee)

External monitoring
All clinical laboratories performing genetic testing must comply with general regulations under the Clinical Laboratory Improvement Amendments (CLIA), and a CLIA certification should be considered the minimum acceptable level of external monitoring. One shortcoming of having only a CLIA certification is that CLIA inspectors often have less experience in evaluating genetic testing laboratories than other certifying organizations. CLIA is in the process of upgrading its regulations regarding genetic testing. The Task Force on Genetic Testing concluded that the current CLIA requirements are insufficient to ensure quality of molecular genetic testing. Laboratories certified by the College of American Pathologists or by New York State Health Department will have undergone a more rigorous external monitoring that requires specific procedures and documentation.

Positive HFE assay controls
Positive controls for HFE mutations must be utilized to validate the assay and each lot of reagents. Positive controls are recommended to be routinely included in each assay run. HFE controls are readily available through the American Type Culture Collection (ATCC, Rockville, MD www.atcc.org) or the Coriell Institute for Medical Research (Camden, NJ http:://arginine.umdnj.edu) repositories.

References:

Holtzman NA, Watson MS. 1997. Promoting Safe and Effective Genetic Testing in the United States. Final report of the Task Force on Genetic Testing. http://www.nhgri.nih.gov /ELSI/TFGT_final/, accessed June 30, 2003.

Neumaier M, Braun A, Wagener C. 1998. Fundamentals of quality assessment of molecular amplification methods in clinical diagnosis. International Federation of Clinical Chemistry Scientific Division Committee on Molecular Biology Techniques. Clin Chem 44:12-26.

---

ANALYTIC VALIDITY

Question 12: Have repeated measurements been made on specimens?

Measurements made on the same specimen in different laboratories
Multiple laboratories have made repeated measurements on the same specimen, utilizing a variety of technologies. A collaborative external proficiency testing program, jointly administered by the ACMG/CAP provides up to six HFE challenges each year, along with a summary report of the results. Earlier sections in Analytic Validity (Questions 10 and 11) provide more details about the results of this program. In summary, the between-laboratory replication of a single specimen's C282Y genotype is between 98.7% and 100% (Figure 2-1).

Measurements made repeatedly on the same sample within a laboratory
It is common practice for repeated measurements to be made on the same specimen within a laboratory. For each assay, a positive control is usually included for each mutation tested. This internal documentation will remain within the laboratory but will be available for on-site inspections by certifying agencies. Thus, one avenue for collection of these data would be to use laboratory survey instruments. Nearly all laboratories will have these data available, even though they may not be routinely collated and analyzed.

---

ANALYTIC VALIDITY

Question 13. What is the within- and between-laboratory precision?

This question is not applicable to the use of DNA tests in screening for HHC, since such testing is qualitative. This question is relevant to quantitative measurements such as transferrin saturation, an alternative strategy for this type of screening.

---

ANALYTIC VALIDITY

Question 14: If appropriate, how is confirmatory testing performed?

Definition
Confirmatory testing is defined here as any additional testing performed after two C282Y mutations are identified in an individual, to ensure that the original result is correct. As seen in Table 2-2 (Questions 10 and 11), the four false negative results would not have been corrected by confirmatory testing, since they were initially reported as negative. It would not be feasible to retest all individuals with negative test results to try to identify false negatives. However, by performing confirmatory testing of the relatively small number of individuals identified as being homozygous (less than 1%), the two false positive results shown in the table might have been identified and corrected.

Four distinct types of confirmatory testing could be utilized, depending on the testing protocols in place and the circumstances in which the positive test result is obtained.

• Repeating the same test protocol on another aliquot of the same specimen
• Repeating the same test protocol on a different specimen
• Performing a different test protocol on another aliquot of the same specimen
• Performing a different test protocol on a different specimen

Importance of confirmatory testing
The analytic specificity is currently estimated to be 99.9% (Question 11). It is important, therefore, to determine how often ‘false positive' results will be identified upon confirmatory testing. If the error is due to clerical or laboratory sample mix-up, simple retesting of an additional aliquot may be sufficient to identify and correct the error. Given that proficiency testing in Europe found a portion of the errors to be of this type (Dequeker and Cassiman, 2000), confirmatory testing might eliminate this type of false positive result. This issue is dealt with in more detail under Clinical Performance (Questions 18 and 19).

Gap in Knowledge: Proportion of Laboratories Performing Confirmatory Testing
Little or no information is available on whether clinical laboratories routinely perform confirmatory testing on samples found to be homozygous (or compound heterozygous). These data could be collected as part of the ACMG/CAP external proficiency testing program.

Gap in Knowledge: Performance of Confirmatory Testing
Little or no information has been found on the ability of confirmatory testing to identify false positive test results in a clinical setting. According to proficiency testing data, false positive results will occur and might be identified as part of routine confirmatory testing of individuals found to be homozygous for C282Y.

Analytic positive predictive value (aPPV)
Figure 2-2 shows the aPPV of testing for C282Y homozygosity, in a population of non-Hispanic Caucasians who have a prevalence of homozygosity of about 4/1000 (corresponding to an allele frequency of about 7%). In addition, the analytic sensitivity is 98.4% (Question 10), the analytic specificity is 99.8% (Table 2-2, column 2).

Figure 2-2. Analytic Positive Predictive Value for C282Y Homozygote Testing in Non-Hispanic Caucasians With a Prevalence of Homozygosity of Five per 1000

Among the 593 individuals identified as homozygous for C282Y, 394 (66%) are true positives. Thus, the aPPV is 66%. If confirmatory testing were routine, and if it were able to identify 90% of the false positive test results, the aPPV might be as high as 95% (388/408). An additional six true homozygotes may also be reclassified as negative.

In some populations, the prevalence of the C282Y genotype is much lower, and it should be expected that fewer of the positive test results would be true positives. Figure 2-3 shows a similar calculation to that shown in Figure 2-2, except that the prevalence of homozygosity is reduced to 1 per 1000.

Figure 2-3. Analytic Positive Predictive Power for C282Y Homozygote Testing in a Population with a Prevalence of Homozygosity of One per 1000

Among the 298 individuals identified as homozygous for C282Y, 98 (33%) are true positives. Thus, the aPPV is 33% in this population. I f confirmatory testing were routine, and if it were able to identify 90% of the false positive test results, the aPPV might be as high as 76% (96/126). An additional two true homozygotes may also be reclassified as negative.

Gap in Knowledge: Analytic specificity among samples with one or no C282Y mutations.
The estimates of analytic specificity among these two groups are expected to be high, and, therefore, errors are relatively rare. For this reason, many challenges are necessary to have confidence in the estimates. Analytic specificity estimates are based on a small number of errors, underscoring the preliminary nature of the current estimates.

Gene frequencies in different racial/ethnic groups
Many reports document differences in HFE gene frequencies, based on racial/ethnic groups and/or geography. A recent study (Steinberg et al., 2001) examined the prevalence of C282Y and H63D in the U.S. population, using samples from the Third National Health and Nutrition Examination Survey (NHANES III, 1992-1994). Samples were genotyped from 5,171 participants and analyzed with respect to race/ethnicity. The allele frequencies for C282Y and H63D are shown in Table 2-13. For a more complete analysis of allele and genotype frequencies by race/ethnicity, see Questions 18 and 19 in Clinical Validity.

Table 2-13. HFE Allele Frequencies in Selected Racial/Ethnic Groups in the United States

---

ANALYTIC VALIDITY

Question 15: What types of patient samples have been tested?

HFE mutation analysis has been successfully performed in a variety of types of specimens using available methodologies.

Screening can be performed on:

whole blood (purified DNA and lysates),

buccal lysates (cheekbrush, swab and mouthwash), or

dried blood spots.

Blood samples are the most reliable method of collecting large amounts of higher quality DNA, but a trained phlebotomist is needed, thereby increasing costs and requiring that specimens be collected at a medical facility. Buccal cells obtained by scraping, brushing or mouthwash yield adequate amounts of DNA for screening purposes, as documented in prenatal cystic fibrosis programs (Doherty et al., 1996; Loader et al., 1996; Witt et al., 1996; Grody et al., 1997). This technique can be used to collect samples at the physician's office or at home. Buccal samples have the disadvantage of less DNA, higher failure rates, and less documentation of chain of custody. Buccal lysates can be frozen and stored for years and still be tested successfully (Bradley et al., 1998). A comparison of test results from blood and buccal mouthwash samples showed consistent results (Baty et al., 1998). Dried blood spots can also be used for PCR-based testing. Guthrie cards from the New York State Newborn Screening Program have been used to amplify multiple genes to detect mutations that impact public health (Caggana et al., 1998). However, they have not routinely been used in hemochromatosis pilot screening programs. An informal survey of several commercial laboratories offering HFE testing determined that none accepted blood spots (Gasparini et al., 1999; S Richards, personal communication).

References

Baty D, Terron Kwiatkowski A, Mechan D, Harris A, Pippard MJ, Goudie D. 1998. Development of a multiplex ARMS test for mutations in the HFE gene associated with hereditary hemochromatosis. J Clin Pathol 51:73-74.

Bradley LA, Johnson DD, Palomaki GE, Haddow JE, Robertson NH, Ferrie RM. 1998. Hereditary haemochromatosis mutation frequencies in the general population. J Med Screen 5:34-36.

Caggana M, Conroy JM, Pass KA. 1998. Rapid, efficient method for multiplex amplification from filter paper. Hum Mutat 11:404-409.

Doherty RA, Palomaki GE, Kloza EM, Erickson JL, Haddow JE. 1996. Couple-based prenatal screening for cystic fibrosis in primary care settings. Prenat Diagn 16:397-404.

Gasparini P, Arbustini E, Restagno G, Zelante L, Stanziale P, Gatta L, et al. 1999. Analysis of 31 CFTR mutations by polymerase chain reaction/oligonucleotide ligation assay in a pilot screening of 4476 newborns for cystic fibrosis. J Med Screen 6:67-69.

Grody WW, Dunkel-Schetter C, Tatsugawa ZH, Fox MA, Fang CY, Cantor RM, et al. 1997. PCR-based screening for cystic fibrosis carrier mutations in an ethnically diverse pregnant population. Am J Hum Genet 60:935-947.

Loader S, Caldwell P, Kozyra A, Levenkron JC, Boehm CD, Kazazian HH, et al. 1996. Cystic fibrosis carrier population screening in the primary care setting. Am J Hum Genet 59:234-247.

Witt DR, Schaefer C, Hallam P, Wi S, Blumberg B, Fishbach A, et al. 1996. Cystic fibrosis heterozygote screening in 5,161 pregnant women. Am J Hum Genet 58:823-835.

---

ANALYTIC VALIDITY

Question 16: How often does the test fail to give a useable result?

Test ‘failures' in the pre-analytic phase of testing
In the pre-analytic phase, it may be determined that the sample is not suitable for testing because specific clinical criteria are not met, or because the sample is considered inadequate. While programs often monitor pre-analytic test cancellation rates as part of an overall quality assurance plan, these events are usually not considered a laboratory or methodologic ‘failure'. Table 2-14 lists criteria commonly used for deciding whether to reject a sample in the pre-analytic phase.

Table 2-14. Common Pre-analytic Criteria for Rejecting Samples Submitted for HFE C282Y Homozygote Testing as Part of Population Screening

Rejection Criteria Based on Clinical Information

Sample submitted for diagnostic testing (i.e., more than the C282Y mutation should be tested)

Rejection Criteria Based on Submitted Sample

Inadequate specimen quality (e.g., hemolyzed blood, dried buccal sample or obvious contamination)

Inappropriate sample (e.g., whole blood with no anticoagulant or wrong anticoagulant)

Inadequate specimen labeling

Inappropriate handling prior to laboratory receipt (e.g., sample too long in transit or exposed to extreme temperature)

Test failures during the analytic phase of testing
Failures of individual samples or assays occur when preset quality control standards are not met and test results are not reportable. Failures can arise for a number of reasons, such as improperly processed samples, problems with component reagents, or equipment malfunction. Many assay failures within the clinical molecular genetic laboratory are due to operator error. Automation and programs to properly train laboratory personnel can avoid most of these problems. Only a few medical technology programs, however, currently provide adequate molecular components in their programs. Documentation of failures and subsequent corrective action is required by regulatory agencies such as CLIA and the College of American Pathologists. Unfortunately, failure rates and other information on assay robustness are often not published as part of pilot trials or method evaluations. Available data suggest, however, that repeating the initial unsatisfactory analysis of an individual sample or assay run can often yield a satisfactory result.  An irretrievable assay failure occurs when an apparently suitable specimen is submitted and approved for testing, but the assay yields a result that is clinically uninterpretable. Failures of this type are most often related to the quality of the original sample. Procedural problems during specimen processing and DNA extraction can also be responsible. Success rates for obtaining clinically interpretable results are close to 100% for blood samples. Buccal samples have a somewhat lower success rate (98% to over 99%) as a result of poor sampling (inadequate number of cells), sample contamination, desiccation (exposure to extreme heat), or inadequate sensitivity of the testing methodology to account for the lower concentration and quality of the sample.  Test failure during the post-analytic phase of testing Post-analytic failures, such as incorrect or inadequately interpreted results, are considered separately from analytic test failures, as part of a review of overall quality assurance in the Clinical Utility Section (Question 34).

Gap in Knowledge: Overall, and method-specific, failure rates
Clinical laboratories are required to document test failures, as described above. For this reason, this type of information should be readily available from laboratories participating in external proficiency testing administered by the ACMG/CAP. This could be accomplished though the use of a supplemental question attached to a routine distribution or, alternatively, the data could be collected via an externally funded, independent project.