Codebook VQ-VAE Approach for Prostate Cancer Diagnosis using Multiparametric MRI
Bibliographic record
Abstract
Multiparametric magnetic resonance imaging (mpMRI) plays an essential role in prostate cancer diagnosis as it can noninvasively localize and grade lesions based on their suspicion of representing clinically significant prostate cancer (csPCa). With the development of deep learning, automatic solutions for csPCa detection based on mpMRI have been developed; however, mpMRI data introduces several difficulties, including data scarcity, heterogeneity in image quality across institutions, and missing modalities. This work addresses these difficulties by building a radiology-based foundational model for prostate cancer mpMRI. Foundation models are deep learning models pretrained on a large-scale dataset and they have recently gained significant interest in computer vision and natural language applications. After pretraining, these models are often adapted for a variety of downstream tasks using smaller datasets from within the same domain. In this work, a large prostate multiparametric MRI (mpMRI) dataset was collected by combining data from our institution with two publicly available datasets. Joint modeling of all mpMRI modalities is essential for accurate prostate cancer diagnosis; however, some of these modalities may be missing. Using unsupervised learning, we pretrained modality-specific vector quantized variational autoencoders (VQ-VAE) to form a radiology foundational model. The learned codebook from VQ-VAE was then used to train a multimodal transformer to perform the diagnosis of clinically significant prostate cancer (csPCa). The proposed multimodal transformer models long-range dependencies between latent representations of input modalities and is augmented with modality-level dropout to increase the model robustness to incomplete modalities. Our framework outperforms previously published work and achieves an average AUC/sensitivity/specificity of 0.764/0.690/0.781. Our results show that pretraining on a larger dataset in combination with the power of transformer architecture can improve the accuracy of automatic prostate cancer detection.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.004 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.002 | 0.001 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".