PD30-05 DEVELOPMENT AND EXTERNAL VALIDATION OF AN ARTIFICIAL INTELLIGENCE-BASED TOOL FOR PROGRESSION RISK ASSESSMENT IN NON-MUSCLE INVASIVE BLADDER CANCER (PROGRXN-BCA)
Bibliographic record
Abstract
You have accessJournal of UrologyBladder Cancer: Non-invasive II (PD30)1 May 2024PD30-05 DEVELOPMENT AND EXTERNAL VALIDATION OF AN ARTIFICIAL INTELLIGENCE-BASED TOOL FOR PROGRESSION RISK ASSESSMENT IN NON-MUSCLE INVASIVE BLADDER CANCER (PROGRXN-BCA) Jethro C. C. Kwong, Zizo Al-Daqqaq, Yashan Chelliahpillai, Soomin Lee, Kellie Kim, Maximiliano Ringa, Amna Ali, Andrew Feifer, Marian S. Wettstein, Wassim Kassouf, Peter C. Black, Rodney H. Breau, Michele Lodde, Adrian Fairey, Jean-Baptiste Lattouf, Claudio Jeldres, Ricardo Rendon, Nimira Alimohamed, Peter Chung, Neil E. Fleshner, Antonio Finelli, Alexandre R. Zlotta, Alistair E. W. Johnson, and Girish S. Kulkarni Jethro C. C. KwongJethro C. C. Kwong , Zizo Al-DaqqaqZizo Al-Daqqaq , Yashan ChelliahpillaiYashan Chelliahpillai , Soomin LeeSoomin Lee , Kellie KimKellie Kim , Maximiliano RingaMaximiliano Ringa , Amna AliAmna Ali , Andrew FeiferAndrew Feifer , Marian S. WettsteinMarian S. Wettstein , Wassim KassoufWassim Kassouf , Peter C. BlackPeter C. Black , Rodney H. BreauRodney H. Breau , Michele LoddeMichele Lodde , Adrian FaireyAdrian Fairey , Jean-Baptiste LattoufJean-Baptiste Lattouf , Claudio JeldresClaudio Jeldres , Ricardo RendonRicardo Rendon , Nimira AlimohamedNimira Alimohamed , Peter ChungPeter Chung , Neil E. FleshnerNeil E. Fleshner , Antonio FinelliAntonio Finelli , Alexandre R. ZlottaAlexandre R. Zlotta , Alistair E. W. JohnsonAlistair E. W. Johnson , and Girish S. KulkarniGirish S. Kulkarni View All Author Informationhttps://doi.org/10.1097/01.JU.0001008848.77629.6f.05AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: Current tools to predict progression risk in non-muscle invasive bladder cancer (NMIBC) perform poorly and do not completely reflect current practice. We aimed to develop and validate PROGRxN-BCa (PROGression Risk assessment in NMIBC) – an artificial intelligence tool to better predict progression. METHODS: PROGRxN-BCa, based on a gradient-boosted survival forest, was trained on NMIBC patients treated between 2005-2015 at one of three academic or community hospital networks: University Health Network, Sinai Health System, and Trillium Health Partners (n=2002). Internal validation was performed on patients treated between 2016-2022 at the same institutions (n=1321). External validation was performed on patients treated between 2012-2023 across 13 academic institutions affiliated with the Canadian Bladder Cancer Information System (n=3708). Primary outcome was time to progression, defined as development of muscle-invasive or metastatic disease. PROGRxN-BCa was compared to the European Association of Urology (EAU) risk calculator, the most widely used clinical prediction model for progression. RESULTS: During a median follow-up of 36 months (IQR 17-65), 1,006 out of 7,031 (14%) patients developed progression. PROGRxN-BCa achieved a c-index of 0.75-0.81, compared to 0.69-0.76 for the EAU risk calculator (p<0.001 for all cohorts). This performance benefit was consistent across clinically relevant subgroups, including age, sex, and tumor history. PROGRxN-BCa was well-calibrated for risks between 0-40%. At 5 and 10 years, PROGRxN-BCa demonstrated a higher net benefit (i.e. avoid unnecessary treatment escalation) compared to the EAU risk calculator for clinically relevant decision thresholds between 15-35%. When applied to intermediate risk patients (n=1555), PROGRxN-BCa identified 19% of patients with an observed average 5-year progression risk of 32% - revealing a subset of patients who may benefit from treatment intensification. Similarly, the model identified 32% of patients with an observed average 5-year progression risk of 2.5%. This approach outperformed sub-stratification based on intermediate risk factors. CONCLUSIONS: PROGRxN-BCa outperformed current tools in NMIBC prognostication in both academic and community settings, particularly in its ability to further sub-stratify intermediate risk patients. PROGRxN-BCa has the potential to further improve NMIBC risk stratification, inform clinical decision-making, and determine eligibility for clinical trials. Source of Funding: This project was supported by the MSH UHN AMO Innovation Fund, PSI Foundation Resident Research Grant, Data Sciences Institute Data Access Grant, CUASF-Bladder Cancer Canada Research Grant, and the University of Toronto Surgeon Scientist Training Program © 2024 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 211Issue 5SMay 2024Page: e626 Advertisement Copyright & Permissions© 2024 by American Urological Association Education and Research, Inc.Metrics Author Information Jethro C. C. Kwong More articles by this author Zizo Al-Daqqaq More articles by this author Yashan Chelliahpillai More articles by this author Soomin Lee More articles by this author Kellie Kim More articles by this author Maximiliano Ringa More articles by this author Amna Ali More articles by this author Andrew Feifer More articles by this author Marian S. Wettstein More articles by this author Wassim Kassouf More articles by this author Peter C. Black More articles by this author Rodney H. Breau More articles by this author Michele Lodde More articles by this author Adrian Fairey More articles by this author Jean-Baptiste Lattouf More articles by this author Claudio Jeldres More articles by this author Ricardo Rendon More articles by this author Nimira Alimohamed More articles by this author Peter Chung More articles by this author Neil E. Fleshner More articles by this author Antonio Finelli More articles by this author Alexandre R. Zlotta More articles by this author Alistair E. W. Johnson More articles by this author Girish S. Kulkarni More articles by this author Expand All Advertisement PDF downloadLoading ...
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.046 | 0.095 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.003 | 0.001 |
| Open science | 0.002 | 0.003 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.007 | 0.006 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".