A84 COMPETENCY BASED MEDICAL EDUCATION: O-SCORE CHARACTERISTICS OF PROCEDURAL AND COGNITIVE ASSESSMENTS IN GASTROENTEROLOGY RESIDENCY TRAINING
Notice bibliographique
Résumé
Abstract Background Competency based medical education has become the new standard for medical education which shifts the focus of training toward a competency, rather than time-based in framework known in Canada as ‘Competence by Design’ (CBD). CBD assesses a physician trainee’s ability to demonstrate competence in CanMEDS roles via entrustable professional activities (EPAs). EPAs utilize the O-SCORE as the metric for assessing competence. This score was developed and validated for surgical/procedural subspecialties; however, CBD currently coopts this scale for both procedural and non-procedural (cognitive) EPAs. Assessor expertise has also been shown to have an important role in performance assessments, but has not been studied in the context of CBD. Aims Our study aims to assess for differences in O-SCORE utilization between cognitive and procedural EPAs, and whether assessor characteristics are associated with trends in assessment. Methods Anonymized data for all Adult GI subspecialty EPAs completed from Jun 2019 to Jan 2023 at the University of Alberta was obtained. Evaluator sex, clinical vs academic practice, advanced training expertise, and EPA score was extracted. Locally a score of 5 denotes competence, while a 1-3 indicates competence was not yet achieved. A score of 4 may be accepted as evidence of competence (neutral score), at the discretion of the local competency committee. Data was analyzed via T-tests and ANOVA with post hoc Games-Howell testing with 95% confidence intervals (CI). A p-value of ampersand:003C0.05 was significant. Results 2264 EPAs were assessed including 1385 cognitive and 879 procedural EPAs. The number of EPAs completed by evaluators ranged from 11 to 165 with a mean of 60 (standard deviation: 40). Results of O-SCORE usage is summarized in Figure 1A-B. The majority of EPAs indicate competence, with 20-25% neutral, and ampersand:003C10% did not achieve competence. Less than one of third of evaluators utilized a score of 1 or 2 across all EPAs, and zero evaluators utilized a score of 1 for cognitive EPAs. Most commonly evaluators to utilized 3/5 options of the O-SCORE. Separated by EPA type, it was most common to utilize 2/5 and 4/5 options for cognitive and procedural EPAs respectively. Results of demographic comparisons are outlined if Figure 1C-E. Male and clinical evaluators submitted higher scores on average. Hepatologists submitted higher scores than all other advanced training areas for total, cognitive, and procedural EPAs. Conclusions Across total, cognitive, and procedural EPAs there are low rates in the utilization of the whole O-SCORE scale, and our study highlights a discrepancy between procedural and cognitive EPAs. In addition, there small but significant differences in the mean EPAs score awarded between different evaluator demographics (male, clinical, hepatologists providing higher scores). Figure 1. A) Number and proportion of Entrustable Professional Activities (EPA) stratified by type and competence evaluation. B) Number and proportion of Entrustable Professional Activities (EPA) stratified by type with scored 1-5 and percent (%) of staff utilizing each score stratified by EPA type C) Number, mean, and mean difference of Entrustable Professional Activities (EPA) stratified by evaluator sex and EPA type. D) Number, mean, and mean difference of Entrustable Professional Activities (EPA) stratified by evaluator academic vs clinical status and EPA type. E) Number, mean, and mean difference of Entrustable Professional Activities (EPA) stratified by evaluator advanced training and EPA type. CI: Confidence interval; SD: standard deviation; *: pampersand:003C0.05 Funding Agencies None
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Comment cette classification a été obtenuedéplier
Prédiction machine sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Le volet Gemma est une étiquette directe du modèle pour chaque travail de la base, lue sur la notice réduite au titre. Le volet Codex est un classifieur appris des 10 348 étiquettes directes de Codex et calibré sur les taux pondérés de l'échantillon; les champs sans appui suffisant ne portent aucun appel Codex. Le mode candidate est l'union des deux volets; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont pas des étiquettes humaines.
Scores du classifieur distillé par catégorie (deux têtes)
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,004 | 0,022 |
| Méta-épidémiologie (sens strict) | 0,000 | 0,000 |
| Méta-épidémiologie (sens large) | 0,000 | 0,001 |
| Bibliométrie | 0,002 | 0,002 |
| Études des sciences et des technologies | 0,000 | 0,001 |
| Communication savante | 0,001 | 0,001 |
| Science ouverte | 0,001 | 0,002 |
| Intégrité de la recherche | 0,000 | 0,001 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,002 | 0,000 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découleClassification
machine, non validéePrédiction automatique; un appel candidat d’une seule source (Gemma direct ou Codex distillé), pas un consensus.
Le détail, modèle par modèle et score par score, se trouve en fin de page sous « Comment cette classification a été obtenue ».