The Molecular Microscope® Diagnostic System meets eminence-based medicine: A clinician’s perspective

To the Editor: The recent editorial1Randhawa PS. The Molecular Microscope Diagnostic System (MMDx) in transplantation: a pathologist’s perspective [published online ahead of print 2020]. Am J Transplant. https://doi.org/10.1111/ajt.15887.Google Scholar on the Molecular Microscope® Diagnostic System (MMDx) presents an opportunity to reflect on the opportunities inherent in molecular platforms and machine learning. Diagnostic platforms are judged by precision and accuracy. Precision is the reproducibility of the assessments; accuracy is the degree to which the assessments reflect the true disease state in the patient. For example, “rejection” used to mean T cell–mediated rejection (TCMR), but that was inaccurate before the recognition of antibody-mediated rejection.2Halloran PF Wadgymar A Ritchie S Falk J Solez K Srinivasa NS. The significance of the anti-class I antibody response. I. Clinical and pathologic features of anti-class I-mediated rejection.Transplantation. 1990; 49: 85-91Crossref PubMed Google Scholar,3Einecke G Sis B Reeve J et al.Antibody-mediated microcirculation injury is the major cause of late kidney transplant failure.Am J Transplant. 2009; 9: 2520-2531Crossref PubMed Scopus (533) Google Scholar Histology relies on visual pattern recognition, based on experience taught from generation to generation. This has served us well but is imprecise because pathologist opinions differ.4Furness PN Taub N Assmann KJM et al.International variation in histologic grading is large, and persistent feedback does not improve reproducibility.Am J Surg Pathol. 2003; 27: 805-810Crossref PubMed Scopus (163) Google Scholar, 5Crespo-Leiro MG Zuckermann A Bara C et al.Concordance among pathologists in the second Cardiac Allograft Rejection Gene Expression Observational Study (CARGO II).Transplantation. 2012; 94: 1172-1177Crossref PubMed Scopus (91) Google Scholar, 6Arcasoy SM Berry G Marboe CC et al.Pathologic interpretation of transbronchial biopsy for acute rejection of lung allograft is highly variable.Am J Transplant. 2011; 11: 320-328Crossref PubMed Scopus (71) Google Scholar, 7Netto GJ Watkins DL Williams JW et al.Interobserver agreement in hepatitis C grading and staging and in the Banff grading schema for acute cellular rejection: the “hepatitis C 3” multi-institutional trial experience.Arch Pathol Lab Med. 2006; 130: 1157-1162Crossref PubMed Google Scholar, 8Regev A Molina E Moura R et al.Reliability of histopathologic assessment for the differentiation of recurrent hepatitis C from acute rejection after liver transplantation.Liver Transpl. 2004; 10: 1233-1239Crossref PubMed Scopus (0) Google Scholar For example, when 2 pathologists assessed histologic TCMR in the MMDx project, pathologist 1’s TCMR diagnosis only agreed with pathologist 2 in ≈50% of biopsies, and vice versa.9Reeve J Sellarés J Mengel M et al.Molecular diagnosis of T cell-mediated rejection in human kidney transplant biopsies.Am J Transplant. 2013; 13: 645-655Crossref PubMed Scopus (0) Google Scholar One was not always wrong: the results reflect the inherent interobserver differences in pattern recognition systems.10Topol E. Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again.1st edn. Basic Books, New York, NY2019: 341Google Scholar In one pattern recognition experiment, pigeons (which like humans excel at pattern recognition) were trained to read X-ray films and microscope slides.11Levenson RM Krupinski EA Navarro VM Wasserman EA. Pigeons (Columba livia) as trainable observers of pathology and radiology breast cancer images.PLoS ONE. 2015; 10: e0141357Crossref PubMed Scopus (58) Google Scholar Individual pigeons varied, and the group (or “ensemble”) of 12 pigeons was always more accurate than any single pigeon. Histology often refers dubious cases to an eminent pathologist, in the belief that central review would be more accurate than local standard-of-care assessment. This is eminence-based, not evidence-based. Central reviewers may be more adherent to guidelines but that is not proof of accuracy. In the clinical trials of immunosuppressive drugs, central review was often included but the primary endpoint was biopsy-proven rejection diagnosed by the local center because that determines treatment. Central review by a single observer obviously eliminates niterobserver variation, but will not improve accuracy. MMDx isolates mRNA and measures genomewide gene expression using microarrays with 50 000 probe sets, which are 99% reproducible. Machine learning–derived algorithms translate expression measurements into diagnostic probabilities in a report that expresses the 3-dimensional relationship of each new biopsy to the reference set with 99% precision.12Madill-Thomsen K Perkowska-Ptasińska A Böhmig GA et al.Discrepancy analysis comparing molecular and histology diagnoses in kidney transplant biopsies.Am J Transplant. 2020; 20: 1341-1350Abstract Full Text Full Text PDF PubMed Scopus (0) Google Scholar Like the pigeons, no single algorithm is right all the time—the No Free Lunch theorem13Wolpert DH Macready WG. No free lunch theorems for optimization.IEEE Trans Evolut Comput. 1997; 1: 67-82Crossref Scopus (0) Google Scholar—so we use ensembles.14Reeve J Böhmig GA Eskandary F et al.Generating automated kidney transplant biopsy reports combining molecular measurements with ensembles of machine learning classifiers.Am J Transplant. 2019; 19: 2719-2731Crossref PubMed Scopus (51) Google Scholar MMDx includes a text sign-out that in boundary cases requires modifiers such as “possible” and “probable.”12Madill-Thomsen K Perkowska-Ptasińska A Böhmig GA et al.Discrepancy analysis comparing molecular and histology diagnoses in kidney transplant biopsies.Am J Transplant. 2020; 20: 1341-1350Abstract Full Text Full Text PDF PubMed Scopus (0) Google Scholar In time this will be automated12Madill-Thomsen K Perkowska-Ptasińska A Böhmig GA et al.Discrepancy analysis comparing molecular and histology diagnoses in kidney transplant biopsies.Am J Transplant. 2020; 20: 1341-1350Abstract Full Text Full Text PDF PubMed Scopus (0) Google Scholar,14Reeve J Böhmig GA Eskandary F et al.Generating automated kidney transplant biopsy reports combining molecular measurements with ensembles of machine learning classifiers.Am J Transplant. 2019; 19: 2719-2731Crossref PubMed Scopus (51) Google Scholar but always acknowledging uncertainties.15Silver N. The Signal and the Noise: Why So Many Predictions Fail - but Some Don’t.1st edn. The Penguin Press, New York2012Google Scholar A few points should be noted:1The INTERLIVER study16Madill-Thomsen KS Wiggins RC Eskandary F Bohmig GA Halloran PF. The effect of cortex/medulla proportions on molecular diagnoses in kidney transplant biopsies: rejection and injury can be assessed in medulla.Am J Transplant. 2017; 17: 2117-2128Abstract Full Text Full Text PDF PubMed Scopus (31) Google Scholar found that standard-of-care histology reports often did not make a clear statement about TCMR. As a result, we had to compare MMDx scores to the histology lesion scores.2Limited challenge bias is not an issue in MMDx studies: all biopsy sets represent the frequency of phenotypes in the clinically relevant population.16Madill-Thomsen KS Wiggins RC Eskandary F Bohmig GA Halloran PF. The effect of cortex/medulla proportions on molecular diagnoses in kidney transplant biopsies: rejection and injury can be assessed in medulla.Am J Transplant. 2017; 17: 2117-2128Abstract Full Text Full Text PDF PubMed Scopus (31) Google Scholar3“Response-to-treatment” is not suitable for assessing accuracy because treatments are not standardized and often do not work (eg, for antibody-mediated rejection).4Machine learning does overcome errors in sample labeling, ie, biopsy diagnoses.17Reeve J Halloran P. Molecular classifiers can outperform the flawed histologic “gold standard” on which they are trained.Am J Transplant. 2018; 18: 496-497Google Scholar We will soon submit a detailed manuscript showing this result.5MMDx agreement with histology is mainly limited by the noise in histology. High agreement is neither expected nor desirable: it is what it is. Treatment is often a binary decision but must be framed in the context of probabilities, quality of evidence, and the consequences of positive errors vs negative errors. The best assessments assemble valid evidence, acknowledging the limitations of the platforms. For example, the MMDx platform cannot currently diagnose recurrent diseases such as IgA nephropathy. The clinician will always want to use all of the information available to guide good decisions for the patient. The authors of this manuscript have conflicts of interest to disclose as described by the American Journal of Transplantation. P.F. Halloran holds shares in Transcriptome Sciences Inc, a University of Alberta research company with an interest in molecular diagnostics; and has given lectures for Thermo Fisher and is a consultant for CSL Behring.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.