Methods-Based Proficiency Testing in Molecular Genetic Pathology

CME Accreditation Statement: This activity (“JMD 2014 CME Program in Molecular Diagnostics”) has been planned and implemented in accordance with the Essential Areas and policies of the Accreditation Council for Continuing Medical Education (ACCME) through the joint sponsorship of the American Society for Clinical Pathology (ASCP) and the American Society for Investigative Pathology (ASIP). ASCP is accredited by the ACCME to provide continuing medical education for physicians.The ASCP designates this journal-based CME activity (“JMD 2014 CME Program in Molecular Diagnostics”) for a maximum of 48 AMA PRA Category 1 Credit(s)™. Physicians should only claim credit commensurate with the extent of their participation in the activity.CME Disclosures: The authors of this article and the planning committee members and staff have no relevant financial relationships with commercial interests to disclose. CME Accreditation Statement: This activity (“JMD 2014 CME Program in Molecular Diagnostics”) has been planned and implemented in accordance with the Essential Areas and policies of the Accreditation Council for Continuing Medical Education (ACCME) through the joint sponsorship of the American Society for Clinical Pathology (ASCP) and the American Society for Investigative Pathology (ASIP). ASCP is accredited by the ACCME to provide continuing medical education for physicians. The ASCP designates this journal-based CME activity (“JMD 2014 CME Program in Molecular Diagnostics”) for a maximum of 48 AMA PRA Category 1 Credit(s)™. Physicians should only claim credit commensurate with the extent of their participation in the activity. CME Disclosures: The authors of this article and the planning committee members and staff have no relevant financial relationships with commercial interests to disclose. Proficiency testing (PT) is intended to be an external measure of clinical laboratory quality.1Kalman L. Lubin I.M. Barker S. du Sart D. Elles R. Grody W.W. Pazzagli M. Richards S. Schrijver I. Zehnbauer B. Current landscape and new paradigms of proficiency testing for molecular genetics.Arch Pathol Lab Med. 2013; 137: 983-988Crossref PubMed Scopus (40) Google Scholar In the United States, it is a requirement of accreditation by the Centers for Medicare and Medicaid Services. It is part of a quality assurance program to verify the accuracy and reliability of laboratory testing. Laboratories in the United States are certified under the Clinical Laboratory Improvement Amendments and accredited by professional organizations with deemed status, such as the College of American Pathologists (CAP). Participation in external quality assessment (EQA) may be through CAP PT programs or through another proficiency testing provider accepted by the Clinical Laboratory Improvement Amendments. For those analytes that do not have EQA surveys available, laboratories must implement an alternative EQA (PT) assessment procedure. Appropriate alternative performance assessment procedures may include split sample analysis with other laboratories, or, if that is not available, assessment of split samples with an established in-house method and previously assayed material, which are run and interpreted by laboratory personnel who do not have access to the prior results.2CLSIAssessment of Laboratory Tests When Proficiency Testing Is Not Available; Approved Guideline, ed 2. Clinical and Laboratory Standards Institute, Wayne, PA2008Google Scholar With the increase in diagnostic sequencing, however, the list of genes for which CAP PT is not available is rapidly growing, and it is virtually impossible to continue analyte-specific testing in a practice environment of large-scale sequencing of patients' DNA. Methods-based proficiency testing (MBPT) is a subset of overall PT and refers to an EQA approach that is based on method, rather than based on each individual analyte tested. MBPT is already well established for several pathology subspecialty areas, and the CAP offers a variety of PT products that are wholly methods based, for example, in cytogenetics, flow cytometry, and immunohistochemistry. Until recently, however, PT in the area of molecular testing has been entirely analyte specific. The CAP supports an MBPT approach for clinical molecular genetic testing, and the Centers for Medicare and Medicaid Services agrees with the concept of MBPT in molecular diagnostics, taking into account the cost and difficulty in obtaining materials for the many different possible mutations in any given gene. Thus, the concept of MBPT complies with federal laboratory regulations. Although MBPT in molecular diagnostics is currently limited to inherited genetic conditions, in the future, molecular cancer testing will probably include MBPT as well. Methods-based approaches are not limited to PT, but they also affect other aspects of developing and implementing laboratory testing. Several published guidelines and examples describe the steps involved in validating a molecular test, but they focus on analyte-specific tests.3Mattocks C.J. Morris M.A. Matthijs G. Swinnen E. Corveleyn A. Dequeker E. Müller C.R. Pratt V. Wallace A. EuroGentest Validation GroupA standardized framework for the validation and verification of clinical molecular genetic tests.Eur J Hum Genet. 2010; 18: 1276-1288Crossref PubMed Scopus (147) Google Scholar, 4Halling K.C. Schrijver I. Persons D.L. Test verification and validation for molecular diagnostic assays.Arch Pathol Lab Med. 2012; 136: 11-13Crossref PubMed Scopus (40) Google Scholar, 5Jennings L. Van Deerlin V.M. Gulley M.L. College of American Pathologists Molecular Pathology Resource CommitteeRecommended principles and practices for validating clinical molecular pathology tests.Arch Pathol Lab Med. 2009; 133: 743-755PubMed Google Scholar, 6Jennings L.J. Smith F.A. Halling K.C. Persons D.L. Kamel-Reid S. Molecular Oncology Resource Committee of the College of American PathologistsDesign and analytic validation of BCR-ABL1 quantitative reverse transcription polymerase chain reaction assay for monitoring minimal residual disease.Arch Pathol Lab Med. 2012; 136: 33-40Crossref PubMed Scopus (22) Google Scholar, 7Kamel-Reid S. Zhang T. Persons D.L. Nikiforova M.N. Halling K.C. Molecular Oncology Resource Committee of the College of American PathologistsValidation of KRAS testing for anti-EGFR therapeutic decisions for patients with metastatic colorectal carcinoma.Arch Pathol Lab Med. 2012; 136: 26-32Crossref PubMed Scopus (23) Google Scholar, 8Lacbawan F.L. Weck K.E. Kant J.A. Feldman G.L. Schrijver I. Biological and Molecular Genetic Resource Committee of the College of American PathologistsVerification of performance specifications of a molecular test: cystic fibrosis carrier testing using the Luminex liquid bead array.Arch Pathol Lab Med. 2012; 136: 14-19Crossref PubMed Scopus (6) Google Scholar, 9Pont-Kingdon G. Gedge F. Wooderchak-Donahue W. Schrijver I. Weck K.E. Kant J.A. Oglesbee D. Bayrak-Toydemir P. Lyon E. Biochemical and Molecular Genetic Resource Committee of the College of American PathologistsDesign and analytical validation of clinical DNA sequencing assays.Arch Pathol Lab Med. 2012; 136: 41-46Crossref PubMed Scopus (36) Google Scholar, 10Saxe D.F. Persons D.L. Wolff D.J. Theil K.S. Cytogenetics Resource Committee of the College of American PathologistsValidation of fluorescence in situ hybridization using an analyte-specific reagent for detection of abnormalities involving the mixed lineage leukemia gene.Arch Pathol Lab Med. 2012; 136: 47-52Crossref PubMed Scopus (13) Google Scholar, 11Schlaberg R. Mitchell M.J. Taggart E.W. She R.C. Microbiology Resource Committee of the College of American PathologistsVerification of performance specifications for a US Food and Drug Administration-approved molecular microbiology test: Clostridium difficile cytotoxin B using the Becton, Dickinson and Company GeneOhm Cdiff assay.Arch Pathol Lab Med. 2012; 136: 20-25Crossref PubMed Scopus (3) Google Scholar It is important to note that these same validation principles can also apply to methods-based testing. In fact, several molecular tests may be better approached through method-based validation rather than through analyte-specific validation, particularly when it may be impractical or even impossible to obtain positive samples of all targeted mutations. All validations should address the same critical question: given the intended use, is the test fit for its purpose? To answer this question, laboratories must assess the potential sources of error and must provide documented evidence that the test will consistently meet the predetermined analytical requirements. For example, validation of a new DNA extraction method may only require documentation that the method yields DNA of sufficient quantity, purity, and integrity for anticipated downstream applications. More complex methods, however, may also be amenable to method-based validation, at least for those processes that have been standardized. Multiplex ligation-dependent probe amplification is an example of a method that involves several steps, but these steps have been standardized across probe sets. After validating the analytical performance of the test system and verifying equivalent performance across multiple probe sets, the validation of a new probe set may be limited to actual probe validation.12Jennings L.J. Yu M. Fitzpatrick C. Smith F.A. Validation of multiplex ligation-dependent probe amplification for confirmation of array comparative genomic hybridization.Diagn Mol Pathol. 2011; 20: 166-174Crossref PubMed Scopus (6) Google Scholar Similarly, aspects of chromosomal microarray testing, Sanger sequencing, and next-generation sequencing (NGS) are amenable to method-based validation. The initial validation of the test system may be quite involved, but the addition of another targeted genomic region or gene sequence may only require probe or primer validation and a determination of the normal reference range. The advantages of MBPT are in part practical, because it is logistically challenging to develop separate proficiency challenges for each of the approximately 20,000 human genes that may be analyzed. More importantly, a widespread MBPT program in which all laboratories conducting nucleic acid sequencing may participate provides better comparisons of performance among laboratories to efficiently and thoroughly test the important aspects of analysis and interpretation that are central to laboratory performance. For example, the ability of laboratories to detect different types of mutations, including single nucleotide variants, insertions and deletions, and mutations in GC-rich genomic regions and to recognize technical reasons for false positive or false negative results (such as allele dropout) can systematically be tested by an MBPT program. MBPT testing can assess proficiency much more efficiently and effectively for thousands of individual genes than a disease-specific PT approach, in which the PT for each gene would incorporate numerous challenge events. In addition, MBPT is not analyte specific; therefore laboratories performing rare tests can participate in an EQA program, provided they are using comparable methodologies. By participating, the performance of laboratories performing rare genetic tests can be evaluated and even graded. MBPT for ultra-rare disorders (<2000 cases prevalence) has already been recommended by the American College of Medical Genetics and Genomics,13Maddalena A. Bale S. Das S. Grody W. Richards S. ACMG Laboratory Quality Assurance CommitteeTechnical standards and guidelines: molecular genetic testing for ultra-rare disorders.Genet Med. 2005; 7: 571-583Crossref PubMed Scopus (68) Google Scholar but, in light of the paradigm shift that results from widespread NGS-based genomic testing, the CAP extends this option to less rare conditions, as well. Of note, MBPT is only acceptable if provided by an external PT provider, as opposed to an internal scheme designed by individual laboratories. Although there are clear advantages to the integration of MBPT, laboratories are also required to participate in analyte-specific PT, if available, because specific aspects of test performance and interpretation can be optimally addressed by PT designed for the gene(s) or disease process being analyzed. Such aspects include the assessment of competency about the identification and interpretation of specific mutations in a given gene, as opposed to a more general nomenclature-specific competency. Thus, laboratories should subscribe to all formal assay/analyte-specific PT available for the assays they perform, because this evaluates gene- and mutation-specific competency. MBPT also has limitations compared with analyte-based PT. Traditional PT challenges have attempted to replicate clinical samples to assess performance of the entire test system. MBPT, by contrast, is focused on a step or process within the test system and may fail to assess other error-prone steps. For example, MBPT may be focused on the analytical step rather than preanalytical steps (eg, specimen processing and nucleic acid extraction) or the postanalytical steps (eg, interpretation and reporting). However, these challenges are not exclusive to MBPT. Even traditional PT samples sacrifice authenticity to provide samples that are stable, traceable, and homogenous, or which target rare, well-characterized mutations. Therefore, most PT samples, whether traditional or method-based, are variably artificial and cannot assess every single individual step in a process. Nevertheless, PT samples provide an external measure of quality and therefore are a critically important part of a laboratory's quality assurance program. The mentioned limitations are considered acceptable, provided the laboratory monitors error-prone steps in the test system in other ways (eg, quality controls, monitoring prevalence of mutations in a population, report review, etc), and these processes are incorporated in CAP molecular pathology checklists and reviewed during on-site peer-reviewed inspections. The rationale for implementing MBPT for molecular testing addresses several key aspects of sequencing-based technologies that are unique compared with other types of molecular testing: i) many clinical laboratories are currently performing genomic sequencing analysis for a variety of multigene panels by using similar (Sanger or NGS-based) methods, even though the specific gene(s) tested may be different; ii) important aspects of the technical performance and annotation of sequence variants are key to laboratory proficiency and can be tested systematically by MBPT but may be agnostic to the specific gene(s) being tested; iii) the ability of a laboratory to design, validate, perform, and interpret certain types of genomic sequencing analysis is not wholly dependent on the specific gene(s) or disease process being tested but rather on their ability to conduct the type of sequencing analysis in its totality; and iv) an MBPT program can be efficiently designed to test the ability of laboratories to analyze the wide spectrum and types of results that are crucial to proficient laboratory performance in genomic sequencing analysis. In 2010, a methods-based Sequencing Educational Challenge (SEC) survey was launched by the Biochemical and Molecular Genetics Committee of the CAP and the American College of Medical Genetics and Genomics. Of note, this was preceded (by years) by European laboratories' MBPT that involved many countries.14Ahmad-Nejad P. Dorn-Beineke A. Pfeiffer U. Brade J. Geilenkeuser W.J. Ramsden S. Pazzagli M. Meumaier M. Methodologic European external quality assurance for DNA sequencing: the EQUALseq PubMed Scopus Google Scholar these surveys have been and evaluated by the Biochemical and Molecular Genetics In the initial laboratories interpreted sequence in a variety of genes for which these laboratories not testing. With the of the provided with these laboratories the same methods for analysis and interpretation as they would for those genes they The of the CAP sequencing challenges is to test laboratory performance by MBPT, to assess analytical as well as the clinical interpretation for the of sequencing results that diagnostic laboratories may when testing by Sanger sequencing Laboratories in the survey each a that sequence that the PT normal sequence that the gene the and the a of the a sequence analysis to the and single nucleotide for and to Society and to describe complex a PubMed Scopus Google Scholar The laboratories to Society to report sequence In addition, laboratories to include all variants to for and to provide a interpretation for each include the of or of clinical to the of the American College of Medical Genetics and of Bale S. Das S. Grody W.W. Lyon E. Molecular of the ACMG Laboratory Quality Assurance for standards for interpretation and of sequence Med. PubMed Scopus Google Scholar After of the F.L. Lyon E. Feldman G.L. Biochemical and Molecular Genetics Resource of a methods-based external proficiency testing program for laboratories DNA sequencing for rare inherited disorders.Genet Med. PubMed Scopus Google Scholar it is clear that MBPT for molecular genetic testing can be and implemented and that the to laboratories from this survey has in and more interpretation of sequence The CAP PT program for sequencing was in to include a MBPT challenge which was designed to test not only analytical and clinical interpretation but also the technical of Sanger This survey DNA and for sequencing, in addition to the materials that are in the In the laboratories are to the sequence and to interpret results for nucleotide and Participation in the survey is for laboratories that sequencing in laboratories that the sequencing but interpret the are to subscribe to the The Biochemical and Molecular Genetics Committee will the performance of laboratories in this MBPT is a and approach in the of genomic CAP has the of a PT for a methods-based PT in which a to as the CAP has been by the commercial Laboratories will be evaluated for their proficiency to single nucleotide variants, deletions, and in this MBPT address multiple key aspects of the such as the individual steps of sequence sequence and annotation of are to be and evaluated in In the European Molecular Genetics Quality and the Quality for molecular launched a MBPT for laboratories and a well-characterized laboratories to run their or their gene. This scheme was designed to assess the technical ability of laboratories that NGS-based diagnostic testing, on the sequence quality rather than proficiency of The EQA scheme has been at several but has not been the scheme report is available only to European Molecular Genetics Quality PT European Molecular Genetics Quality With the of sequencing-based clinical testing, the of variants in every area of to which molecular diagnostic testing is is and is to In the of variants is approximately For that to A. S. and approaches to variants by 2011; PubMed Scopus Google Scholar With any test, a of variants will be of which will be disease For inherited of a single gene for a single disease has and that there are i) variants in a single gene (eg, which is for example, ii) multiple rare variants in a single gene (eg, to cancer 1 or iii) a variants in multiple genes with a single disease (eg, and of the complex such as type and iv) multiple rare variants in multiple genes with a single disease complex such as Therefore, mutations will the of genetic mutations that are currently tested in clinical laboratories. The MBPT approach the of laboratories to a test method and to the types of variants during genomic analysis nucleotide variants, deletions, and chromosomal rather than developing PT for each and every disease gene and its mutations. The ability of the to the of different of variants provides a assessment and whether the laboratory is to the to different types of genetic mutations by using a specific assay The process of a diagnostic by using is of a to a sequencing and to sequence the also is a which of analysis of sequence and interpretation of variants This process is complex and is being for the diagnostic of multigene and for and Clinical laboratories with in molecular testing are well for implementing the methods and for In contrast, the analysis of is a new for clinical and sequencing for gene identification in inherited an technical and Pathol Lab Med. 2013; 137: PubMed Scopus Google Scholar In this clinical laboratories may several approaches in the of approaches include i) laboratories the process within their ii) laboratories process samples and sequence but the analysis and interpretation iii) laboratories the laboratory but develop the in and iv) laboratories and in analysis and In laboratories that analysis the to to at a diagnostic different approaches considered when the CAP accreditation requirements. Thus, the to develop separate accreditation for the of and the analysis This the of laboratories that different approaches with to implementing By EQA testing for should also incorporate To address the CAP is developing an MBPT challenge that will address the process as it should be that the different approaches to the of a challenge when the and the are at different In the United States, is a of federal and under the it may not be possible to test the entire process under a single EQA that the not at a address but rather in the it will be to the interpretation of this in the PT challenges are being that will focus on the of the The challenges be of sequence (eg, from a or gene analysis with in sequence that variants, for example, single nucleotide variants or insertions and Such PT will a laboratory's ability to and sequence and to and In challenges are particularly in molecular testing or other mixed sample testing, including and in which the of detection of the assay is critical for By sequence that mutations of at with a set of sequence it will be possible to assess a laboratory's analytical to mutations, which will be in clinical advantages of testing are to with PT samples for sample would be a genomic DNA which multiple cancer mutations all in the same Such are being by commercial several different mutations are by within and the DNA from these each a different is This is an area of and area for PT will be to assess the ability of laboratories to variants from variants that are variants are in all with any type of testing panels or or therefore laboratories must all variants in relevant genes and to report on variants that may be the list to a set of those variants that are interpreted as or with to are in a of steps such as in for and clinical for previously In addition, laboratories may to and report on variants not with but as on the laboratory's policies for with the that approximately or more mutations that of within regions of the is even more than previously In the of genomic interpretation of variants has been as of the critical for sequence to clinical Therefore, in the future, PT challenges are to assess the laboratory's ability to analyze and interpret and In in the rapidly of molecular pathology with an increase in MBPT is a of traditional MBPT is to assess proficiency of molecular diagnostic laboratories that provide testing and can be in with analyte-specific testing to and laboratory performance. MBPT to be the most practical, and method to measure laboratory proficiency in sequencing