Learning medical professionalism with the online concordance-of-judgment learning tool (CJLT): A pilot study
Bibliographic record
Abstract
CONTEXT: Professionalism development entails learning to make judgments in ambiguous situations. A Concordance of Judgment Learning Tool (CJLT), comprised of 20 vignettes involving professionalism issues, was developed. Students obtained a measure of how concordant their judgments were with a panel of experts and learned from given explanations. METHOD: Twenty clinical vignettes implying professionalism issues were written including, for each, four possible courses of action. Expert panel, nominated by all clerkship students, was made up of attending physicians that best represented professionalism role models. Experts completed CJLT and gave explanations for their answers. All clerks were invited to answer each vignette, and then received automated expert feedback including explanations. RESULTS: Seventy-nine students sat for the activity. The optimized test included 20 cases and 54 questions (Cronbach's alpha coefficient of 0.64). Student - expert concordance scores ranged from 54 to 77 with a mean at 64.6 (standard deviation 5.1). Satisfaction survey results indicated high satisfaction and relevance of tool despite some pitfalls. Post-test focus group data revealed relevant experiential learning on professionalism issues. DISCUSSION: Students' scores and perceptions suggest pedagogic relevance of the CJLT in fostering professionalism development in clerkship. CJLT is user-friendly and shows promise as a situation experiential learning activity.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.005 | 0.015 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.003 |
| Insufficient payload (model declined to judge) | 0.006 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".