From cue cards to code rot: In the end, it's not how you acquire the content, it’s what you do with it
Bibliographic record
Abstract
Cue cards, remember those?Mine were alphabetized, 5" x 8" cue cards.They were packed with lists, diagrams, and classifications, neatly squeezed into a large folio box.That's what I used for studying right up to the moment before my Royal College examinations in 1991.When I first started practice, I kept them in the office as a sort of security blanket in case I needed to access them for an urgent refresher.I still have them, except now they are at home on the top shelf of a closet next to a dusty box full of 8 mm Kodachrome II film reels containing the secrets of my childhood.I sometimes wonder if those cue cards would still be in my office if it weren't for the internet and smart phones.Part of the process of becoming a urologist is to become a content expert; in order to do that, trainees must access the content, learn it, and reproduce it on a standardized test.Over the years, I have had the privilege of contributing to the education of over 60 residents.During this time, I have observed the evolution of their study habits from cue cards and long, hand-written study notes to PowerPoint presentations, YouTube videos, social media, and digital files.Each has helped to parse urology into ever-smaller bits for the human brain to recognize and retrieve.In this issue of CUAJ, Skinner et al from Queen's University have provided us with an interesting descriptive study of two years of Canadian graduates, how they study, how much they study, and what motivates them to study throughout their residency. 1Not surprisingly, the reported time spent studying increased progressively throughout the years of training.The overwhelming motivator for studying in the final year was the Royal College certifying examination.It should be sobering to faculty that didactic lectures from us were rated as a rather mediocre method of content acquisition.As alluded to in the study, one of the central pillars supporting the shift from a timebased model to a competence by design (CBD) model of medical education is the desire to move away from content and towards competence.It has been recognized that being a content expert is necessary, but insufficient for the delivery of safe and appropriate care.In other words, when you lose control of the renal vein, the books are closed, the cue cards are in their box, your smart phone is inaccessible, and nobody is going to tweet you out of this situation.The unstated challenge in this paper then relates to how we teach and evaluate competence -not just the technical components of surgical competence, but the cognitive ones as well.How do we teach and assess communication, collaboration, and decision-making in the time-crunched ambulatory and operative setting?Decisionmaking -MD also stands for making decisions and making them under pressure based on incomplete information, and then taking responsibility for those decisions.No amount of content expertise, code rot or not, can replace that ability.How can we better teach and assess this?In addition, if the examinations are moved to the penultimate year of training, as is being proposed in the new CBD framework, what will replace the exam (if anything) to motivate trainees to study during the transition to practice phase of training?The members of the Urology Specialty Committee from across Canada have been hard at work trying to answer questions like these.Over the last 15 months, we have volunteered a total of nine days over three visits to the Royal College and an additional eight hours of teleconference time arm-wrestling our way toward some form of consensus on CBD implementation for urology.We are not there yet, and the result won't be perfect, but we are getting closer to accommodating the many institutional and regional differences of opinion.The go-live date is looking like July 1, 2018.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.005 | 0.027 |
| Meta-epidemiology (narrow) | 0.003 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.002 |
| Bibliometrics | 0.004 | 0.002 |
| Science and technology studies | 0.004 | 0.003 |
| Scholarly communication | 0.008 | 0.005 |
| Open science | 0.003 | 0.002 |
| Research integrity | 0.013 | 0.021 |
| Insufficient payload (model declined to judge) | 0.022 | 0.017 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".