Deep learning, data ramping, and uncertainty estimation for detecting artifacts in large, imbalanced databases of MRI images
Bibliographic record
Abstract
ABSTRACT Magnetic resonance imaging (MRI) is increasingly being used to delineate morphological changes underlying neurological disorders. Successfully detecting these changes depends on the MRI data quality. Unfortunately, image artifacts frequently compromise the MRI utility, making it critical to screen the data. Currently, quality assessment requires visual inspection, a time-consuming process that suffers from inter-rater variability. Automated methods to detect MRI artifacts could improve the efficiency of the process. Such automated methods have achieved high accuracy using small datasets, with balanced proportions of MRI data with and without artifacts. With the current trend towards big data in neuroimaging, there is a need for automated methods that achieve accurate detection in large and imbalanced datasets. Deep learning (DL) is the ideal MRI artifact detection algorithm for large neuroimaging databases. However, the inference generated by DL does not commonly include a measure of uncertainty. Here, we present the first stochastic DL algorithm to generate automated, high-performing MRI artifact detection implemented on a large and imbalanced neuroimaging database. We implemented Monte Carlo dropout in a 3D AlexNet to generate probabilities and epistemic uncertainties. We then developed a method to handle class imbalance, namely data-ramping to transfer the learning by extending the dataset size and the proportion of the artifact-free data instances. We used a 34,800 scans (98% clean) dataset. At baseline, we obtained 89.3% testing accuracy (F1 = 0.230). Following the transfer learning (with data-ramping), we obtained 94.9% testing accuracy (F1 = 0.357) outperforming focal cross-entropy (92.9% testing accuracy, F1 = 0.304) incorporated for comparison at handling class imbalance. By implementing epistemic uncertainties, we improved the testing accuracy to 99.5% (F1 = 0.834), outperforming the results obtained in previous comparable studies. In addition, we estimated aleatoric uncertainties by incorporating random flips to the MRI volumes, and demonstrated that aleatoric uncertainty can be implemented as part of the pipeline. The methods we introduce enhance the efficiency of managing large databases and the exclusion of artifact images from big data analyses. Highlights We address the difficulty in automatically detecting artifacts in a large, imbalanced image database. We reproduced high accuracy in detecting artifacts in small balanced datasets. We implemented Monte Carlo (MC) dropout in 3D AlexNet to generate uncertainty metrics. We transfer the learning from a small balanced dataset to a large imbalanced dataset, outperforming focal loss. We outperformed artifact detection in previous studies using comparable large imbalanced datasets The user can select an operating point to consider manual inspection vs. artifact detection error.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.007 | 0.020 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.001 | 0.002 |
| Open science | 0.002 | 0.002 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".