Abstract P5-02-01: Analytical validation and prognostic potential of an automated digital scoring protocol for Ki67: An International Ki67 Working Group study
Bibliographic record
Abstract
Abstract Background: The nuclear proliferation biomarker Ki67 has multiple potential roles in breast cancer, including aiding decisions based on prognosis, but has unacceptable between-laboratory variability. Here we tested an open source and calibrated automated digital image analysis (DIA) platform to: (i) Assess inter-laboratory reproducibility of automated Ki67 measurement among 17 participating labs and compare those with standardized pathologist-based visual scoring. (ii) Investigate the comparability of Ki67 measurement across corresponding core biopsy and whole section cases. (iii) Test prognostic potential of the built Ki67 scoring algorithms on an independent cohort. Methods: Two sets of 60 previously stained slides containing 30 core-cut biopsy and 30 corresponding whole tumor sections from 30 ER+ breast cancer cases were sent to 17 participating labs for automated assessment of average Ki67 expression. The blocks were centrally cut and stained for Ki67 using the Mib-1 antibody. The QuPath (open-source software) DIA platform was used to evaluate tumoral Ki67 expression. Calibration of the DIA method was performed in our previous study (Acs et al, Lab Invest 2019). A detailed guideline for building an automated Ki67 scoring algorithm was sent to the participating labs. Visual scoring of average Ki67 expression was performed by pathologists according to published standardized methods (Leung et al, NPJ Br Cancer 2016; Leung et al, Histopath 2019). Locked down DIA Ki67 scoring algorithms were applied to a validation cohort: 222 breast cancer cases from the Karolinska University Hospital in whole section format. Sufficient reproducibility to declare analytical validity was defined as an Intra Class Correlation (ICC) with lower limit of 95% credible interval (CI) >0.80. Markov Chain Monte Carlo routines for generalized linear mixed models were used to estimate ICCs and calculate corresponding CIs. Results: The same-section ICC was 0.902 (CI: 0.852-0.949) across 17 labs using calibrated DIA platform on core biopsy slides and 0.845 (CI: 0.778-0.912) on whole sections. The different-section ICC across the 17 labs was 0.873 (CI: 0.806-0.932) scoring on core biopsy slides and 0.777 (CI: 0.670-0.874) on whole sections. The pathologist-based visual Ki67 scoring showed ICC of 0.860 for all comparisons, respectively (CI: 0.795-0.927). Similar to what was observed for visual Ki67 scoring, the DIA scores are higher for core biopsy slides compared to paired whole sections (p≤0.001; median difference: 5.31%; IQR: 11.50%). Ki67 scores of all locked down DIA algorithms correlates significantly (p≤0.023) with outcome on the validation cohort (observed hazard ratios range: 2.518-2.922). Conclusions: Automated Ki67 evaluation using a calibrated, open-source DIA platform (QuPath) met the pre-specified criterion of success on core biopsies but not on whole sections in the multi-institutional setting. The systematic discrepancy between core biopsy and corresponding whole sections was likely due to pre-analytical factors (tissue handling, fixation) and intratumor heterogeneity. We found that different algorithms built according to calibrated DIA methods had similar prognostic potential. Assessment of clinical utility is planned. Citation Format: Balazs Acs, Samuel C.Y. Leung, Kelley M. Kidwell, Indu Arun, Renaldas Augulis, Sunil S. Badve, Yalai Bai, Anita L. Bane, John M.S. Bartlett, Jane Bayani, Gilbert Bigras, Annika Blank, Signe Borgquist, Henk Buikema, Martin C. Chang, Robin L. Dietz, Andrew Dodson, Anna Ehinger, Susan Fineberg, Cornelia M. Focke, Dongxia Gao, Allen M. Gown, Carolina Gutierrez, Johan Hartman, Judith C. Hugh, Zuzana Kos, Anne-Vibeke Lænkholm, Arvydas Laurinavicius, Richard M. Levenson, Rustin Mahboubi-Ardakani, Mauro G. Mastropasqua, Takuya Moriya, Sharon Nofech-Mozes, C. Kent Osborne, Liron Pantanowitz, Frédérique M. Penault-Llorca, Tammy Piper, Mary Anne Quintayo, Tilman T. Rau, Stefan Reinhard, Stephanie Robertson, Takashi Sakatani, Roberto Salgado, Melanie Spears, Jane Starczynski, Tomoharu Sugie, Bert van der Vegt, Giuseppe Viale, Shakeel Virk, Lila A. Zabaglo, Daniel F. Hayes, Mitch Dowsett, Torsten O. Nielsen, David L. Rimm, International Ki67 in Breast Cancer Working Group, BIG-NABCG. Analytical validation and prognostic potential of an automated digital scoring protocol for Ki67: An International Ki67 Working Group study [abstract]. In: Proceedings of the 2019 San Antonio Breast Cancer Symposium; 2019 Dec 10-14; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2020;80(4 Suppl):Abstract nr P5-02-01.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.063 | 0.067 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.002 | 0.002 |
| Science and technology studies | 0.001 | 0.002 |
| Scholarly communication | 0.003 | 0.001 |
| Open science | 0.003 | 0.002 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.002 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".