Development of a validated exam to assess physician transfusion medicine knowledge
Bibliographic record
Abstract
BACKGROUND: There is evidence that physicians lack adequate transfusion medicine knowledge. To design needs-based educational interventions to address this gap, a validated assessment tool is required. Previously published exams have not been created or validated using rigorous psychometric methods. STUDY DESIGN AND METHODS: A modified Delphi method was used to achieve consensus regarding the essential knowledge and skills for physicians who transfuse blood products. To ensure content validity, members of an international organization of transfusion medicine experts (Biomedical Excellence for Safer Transfusion [BEST] Collaborative) participated in the exam design process. An exam, based on the most highly rated topics, was created and administered to individuals with a priori expected basic, intermediate, and expert levels of transfusion medicine knowledge. Rasch analysis, a psychometric technique used in high-stakes medical licensure and board testing, was used to determine exam accuracy and precision. RESULTS: Thirty-six topics achieved ratings sufficient to be considered for inclusion in the exam (content validity index > 0.8). A 23-question exam was administered to 49 individuals. Mean scores for individuals with expected basic, intermediate, and expert knowledge were 42, 62, and 82%, respectively (p < 0.0001). The exam achieved good fit with the Rasch model. CONCLUSION: A validated exam has now been created to accurately assess transfusion medicine knowledge. This exam can be used to determine knowledge deficits and assist in the design of curricula to improve blood product utilization.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.015 | 0.042 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".