The University of Michigan Imaging Assessment Tool (UM-IAT)
Bibliographic record
Abstract
Abstract Introduction Evidence in the literature is mixed regarding the competence of health professional trainees when interpreting imaging studies independently, as there are no widely accepted assessment standards for basic radiology skills. This is especially the case for trainees who are not pursuing radiology as their eventual specialty. Various methods have been used to assess learners' ability to interpret imaging, including peer review of image interpretation as a learning tool, and comparison between attending and learner interpretation of imaging studies. As with other performance-based clinical skills, there is a measurable learning curve for these image interpretation skills. This resource is a one-time competency-based assessment to determine baseline skills for interpretation skills. Methods This quiz follows a case-presentation format. Each case consists of a set of radiological images, and a brief query or description of a patient and his or her chief complaint. The quiz is administered in a test setting to assess interns' image interpretation skills. It is delivered online with 20-minutes allowed for completion. There are 14 images that must be identified, which are typical of those learners, or mid-level practitioners, are asked to interpret. The authors also provide a directed PowerPoint to highlight where learners should focus their attention in order to arrive at the correct interpretation in the future. Results An analysis of the quiz as a whole was conducted. After removal of individual items with negative discrimination indices, overall test-level discrimination persisted (f (2,365) = 20.04, p < 0.0001). Further, a poster presentation of this tool received the Top Poster award at the International Conference on Residency Education in Ottawa, Ontario Canada in 2012 and was published in Academic Radiology in 2014. Discussion This quiz provides actionable information to clinical supervisors and program directors related to residents' and mid-level practitioners' ability to begin the process of indirect supervision regarding interpretation of radiologic images. While this assessment is a useful tool for clinical supervisors and program directors to gain insight into trainees' ability to interpret images, we recommend this quiz be used in conjunction with formalized training rather than as a stand-alone tool. This tool is not intended to replace the ultimate determination of clinical interpretation of trained radiologists.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".