Development of a Novel Machine Learning Model for Automatic Assessment of Quality of Transvaginal Ultrasound Images From Multi‐Annotator Labels
Bibliographic record
Abstract
Objectives: Accurate diagnosis of pathology from ultrasound images is reliant upon images of a suitable diagnostic quality being acquired. This study aimed to create a novel machine learning model to automatically assess transvaginal ultrasound (TVUS) image quality for gynaecological ultrasound. Method: Six imaging professionals (two sonographers, two gynaecological sonologists and two radiologists) assigned a quality score to 150 TVUS images from 50 cases (50 uterus images and 100 ovary images). Images were given a score of 1-4 (1-reject/image inaccurate, 2-poor quality, 3-suboptimal quality or 4-optimal quality). As variation existed between the scores assigned by the labellers, we framed this problem as a multi-annotator noisy label problem. To address this, a new machine learning architecture was developed, combining a weighted ensemble algorithm to estimate consensus labels and a multi-axis vision transformer (MaxViT) to handle noisy labels, improving model accuracy in predicting image quality. Forty cases (120 images) were used for model training, while the remaining 10 cases (30 images) were reserved as a test set for model evaluation. Results: The novel machine learning architecture we created was able to successfully determine image quality with a validation accuracy of 80% and a macro average recall of 77%. This significantly improved upon the 57% accuracy of the baseline machine learning method (ResNet50). The MaxViTs were able to outperform human performance in most cases, with an accuracy of 80% surpassing four of the six human labellers. Conclusions: This novel machine learning model offers an automated method of assessing the quality of TVUS images. The tool has the potential to provide real-time feedback to those performing TVUS, reduce the need for repeated imaging, and improve the diagnosis of gynaecological pathology.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.002 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".