How and Why Internal Medicine Clerkship Directors Use Locally Developed, Faculty-Written Examinations
Bibliographic record
Abstract
PURPOSE: To describe how and why internal medicine clerkship directors (CDs) use locally developed, faculty-written (LFW) examinations. METHOD: In 2009, the Clerkship Directors in Internal Medicine conducted an annual, online, confidential survey of its 107 U.S. and Canadian institutional members, including questions about LFW examinations. Data were analyzed using descriptive statistics and coding of free text. RESULTS: Sixty-nine of 107 members (64.5%) responded. The National Board of Medical Examiners (NBME) examination was administered by 93% (63/68), LFW examinations were used by 33% (22/67), and both types were used by 22% (17/67)-compared with 85%, 36%, and 12% in 2005. LFW examinations were frequently created by the CD alone (9/22; 41%) and consisted of one test (12/22; 52.2%), but some schools gave two (6/22; 26.1%), three (2/22; 8.6%), or four or more (3/22; 13%). Multiple-choice examinations were most common (26/38; 68.4%), followed by short-answer (8/38; 21.1%) and essay (4/38; 10.5%). Most were graded using preestablished criteria; half required a minimum passing score (60% most common). LFW exams were most commonly 5% to 10% of the total grade. Only a minority of CDs reported having reliability estimates or a control group for their exams. Most (70%) reported using LFW exams to cover content felt to be underrepresented by the NBME. CONCLUSIONS: Findings strongly suggest that a minority of internal medicine CDs use LFW examinations, mostly to measure achievement not assessed by the NBME. However, validity evidence is not consistently being gathered, which may limit judgments based on exam results.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.006 | 0.051 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.003 | 0.001 |
| Open science | 0.001 | 0.002 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".