Professionalism Mini‐Evaluation Exercise for medical residents in Japan: a pilot study
Bibliographic record
Abstract
CONTEXT: Assessing medical professionalism among medical residents is of great importance. The Professionalism Mini-Evaluation Exercise (P-MEX) is a tool for assessing professionalism that was developed, tested for reliability and validated in Canada. Prior to the present study, no Japanese version of the P-MEX had been tested. METHODS: We modified the P-MEX for use in Japan and tested it on medical residents in a Japanese teaching hospital. For each resident, eight evaluators completed the P-MEX forms. A total of 184 P-MEX forms were completed on 23 senior residents. The construct validity of the P-MEX was analysed by confirmatory factor analysis through structural equation modelling. The reliability of the P-MEX was tested using generalisability theory and a decision study. After performing the assessment and providing feedback to the residents, we conducted a survey on the residents' perceptions of the assessment. RESULTS: Results indicate content and construct validity. A confirmatory factor analysis revealed that factor loadings ranged from 0.58 to 0.96, indicating good construct validity except for one item (P12: Maintained appropriate boundaries with patients and colleagues). Structural equation modelling showed that adding new items developed in Japan to the P-MEX provided adequate factor validity. A decision study showed confidence intervals sufficiently narrow with as few as 10 evaluations, slightly more than the eight forms verified in Canada. Most residents stated that the items were reasonable and appropriate, the results of the assessment were consistent with their own self-evaluation and the assessment enhanced their motivation. CONCLUSIONS: Our study demonstrated good evidence of adequate reliability and validity of the P-MEX for the assessment of professionalism among Japanese residents. Moreover, the addition of new items developed in Japan provided adequate factor validity.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.006 | 0.029 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".