Creation of novel pediatric academic curriculum and its evaluation using mixed methods
Bibliographic record
Abstract
Abstract Introduction The Royal College of Physicians and Surgeons of Canada and the American Accreditation Council for Graduate Medical Education require resident skills in Evidence-Based Medicine and participation in research activities. Our first-year pediatric residents (PGY1s) were required to attend a novel, call-protected, 4-week Academic Skills and Knowledge (ASK) rotation to improve their skills as consumers of medical literature. Objectives of the study were to describe this curriculum and summarize its mixed-methods evaluation. Methods After 14 months of curriculum development, three annual cohorts of PGY1s wrote identical pre- and post-ASK quizzes (2017-19). In 2018 and 2019, we assessed knowledge retention with PGY1s re-writing the quiz after 6 months. Mean test scores were compared using paired t-tests. In 2017, pre- and post-ASK focus groups assessed resident feelings about the rotation. Results All eligible PGY1s (n=32) participated. Mean exam scores demonstrated increased knowledge (time0 mean±SD 52.6±11.0%; vs. time1 80.2±9.0%, p <0.001). Knowledge retention at 6 months was intermediate (time2 70.2±12.0%; time0 vs time2 p<0.001). In the pre-rotation focus group, residents looked forward to ASK; goals centered around growing from learner to expert. Post-ASK, residents were very satisfied. Resident participation in our annual Research Institute poster competition increased linearly from 0% in 2014 to 8% in 2020 (r=0.74, p=0.01). Discussion The ASK curriculum was successfully implemented, and increased knowledge persisted over time. Residents were satisfied with ASK and appreciated the structured curriculum building on core knowledge that they could immediately apply to their clinical work. Statements and Declarations All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Martha Balicki, Darja Barr, Atul Sharma and Celia Rodd. The first draft of the manuscript was written by Martha Balicki and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript. Financial interests None of the authors have any relevant financial or non-financial interests to disclose. The authors did not receive support from any organization for the submitted work. Data availability The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.057 | 0.051 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.003 |
| Bibliometrics | 0.002 | 0.002 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.002 | 0.001 |
| Open science | 0.002 | 0.002 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.003 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".