Evaluating AI-Generated Podcasts Versus Traditional Reading for Learning From Medical Articles: Protocol for a Mixed-Design Study Among Resident Physicians
Bibliographic record
Abstract
BACKGROUND: Podcasts have emerged as a popular medium in medical education over the past decade. Audio learning allows flexibility and may help residents engage with content in new ways. Reading scientific literature is a core skill for residents, yet many residents struggle to comprehend complex research articles. Advances in artificial intelligence (AI) have enabled the automatic generation of podcast-style summaries of documents. It remains unclear whether listening to AI-generated podcast summaries can match the educational value of reading the full text of medical papers, and whether this depends on the complexity of the article. OBJECTIVE: This study aims to compare comprehension of medical research papers when learning via an AI-generated audio podcast versus traditional reading. We will examine whether article complexity (narrative vs technical) moderates any difference. We hypothesize an interaction: for a highly complex article, residents who read the full text should achieve a better understanding than those who listen to a summary, whereas for an easier article, the difference between modalities should be smaller. METHODS: We designed a 2×2 mixed factorial study with 60 resident physicians preparing for the board certification in internal medicine or cardiology. All participants will engage with 2 peer-reviewed cardiology articles differing in complexity: a narrative case report on eosinophilic myocarditis and a technical research article on quantifying the vena contracta area using 3-dimensional echocardiography. Each participant will read 1 article and listen to an AI-generated podcast summary of the other, with the order and assignment counterbalanced to control for order effects. The podcasts are created using Google NotebookLM's experimental audio overview feature. Participants will complete a multiple-choice knowledge test for each article. The primary outcomes are comprehension scores for each modality. The secondary outcomes include intrinsic motivation, perceived learning gains, and cognitive load for each condition. Data will be analyzed using a mixed ANOVA to test the main effects of modality and article complexity, as well as their interaction. RESULTS: Data collection is expected to be completed by early 2026. We will report the trial results according to the CONSORT (Consolidated Standards of Reporting Trials) guidelines, and any deviations from this protocol will be documented and justified. No results are available at the time of publication of this protocol. CONCLUSIONS: This randomized trial will offer evidence on the effectiveness of AI-generated podcast summaries as a learning tool for medical literature. If listening to an AI-generated podcast yields comprehension comparable to or superior to reading the full article, it could validate an innovative, time-saving approach for busy medical trainees. Conversely, if significant deficits are observed in the podcast group (especially for complex content), the findings will highlight the limitations of AI summaries and the continued importance of traditional reading for thorough understanding. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/78505.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.060 | 0.070 |
| Meta-epidemiology (narrow) | 0.005 | 0.003 |
| Meta-epidemiology (broad) | 0.007 | 0.005 |
| Bibliometrics | 0.003 | 0.003 |
| Science and technology studies | 0.004 | 0.003 |
| Scholarly communication | 0.004 | 0.003 |
| Open science | 0.004 | 0.003 |
| Research integrity | 0.008 | 0.007 |
| Insufficient payload (model declined to judge) | 0.033 | 0.006 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".