The transfer learning gap: quantifying transfer learning in a medical image case
Bibliographic record
Abstract
Transfer learning is a widely used technique in medical imaging and other research fields where a scarcity of available data limit the training of machine learning algorithms. Despite its widespread use and extensive supporting body of research, the specific mechanisms behind transfer learning are not completely understood. In this work, we quantify the effectiveness of transfer learning in medical image classification scenarios for different numbers of training set images. We trained ResNet50, a popular deep learning model used in medical image classification, using two scenarios: 1) applying transfer learning to a pre-trained network and 2) training the same model from scratch (i.e., starting with randomly selected weights). We analyzed the performance of the model under both scenarios as the number of training set images increased from 5,000 to 160,000 medical images. We introduced and evaluated a metric, the transfer learning gap (TLG), to quantify the differences between the two scenarios. The TLG measured the difference in the area under the loss curves (AULCs) when transfer learning was applied and when the model was trained from scratch. Our experiments show that as the training set size increases, the TLG trends to zero, suggesting that the advantage of using transfer learning decreases. The trend in the AULC suggests a training set size where the two scenarios would have equal losses. At this point, the model reaches the same performance regardless of if transfer learning or training from scratch was used. This study is important because it provides a novel metric to understand and quantify the effect of transfer learning.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.013 | 0.071 |
| Meta-epidemiology (narrow) | 0.002 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.004 | 0.002 |
| Science and technology studies | 0.001 | 0.004 |
| Scholarly communication | 0.002 | 0.005 |
| Open science | 0.002 | 0.004 |
| Research integrity | 0.003 | 0.002 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".