Using a commitment‐to‐change strategy to assess faculty development
Notice bibliographique
Résumé
‘Cabin Fever’ is an established University of Calgary faculty development programme offered annually to help rural educators improve their teaching and assessment skills. Teachers participate in five 80-minute workshops (out of 22) and one plenary session and interact informally with colleagues during lunches and dinners. Workshop topics have included: teaching procedural skills; teaching professionalism; teaching empathy; teaching multiple levels of learners; the resident as teacher; assessment and feedback; learning technologies; patient safety, and building effective workplaces. Post-course evaluations indicated the course was successful. There was no information about the programme’s ongoing impact on teaching staff and therefore potentially on learners. Data were required to defend the programme’s continuance after 10 years because it requires significant resources. We adopted ‘commitment-to-change statements’ (CTCs) to enhance course evaluation. At the end of the programme, doctors were asked to identify three changes they planned to make in the next 3 months and their level of commitment to making each change (using a scale of 1–4, where 1 = low and 4 = high). Three months later, participants were asked whether the intended changes were complete, still in progress, incomplete or would not be undertaken. We analysed the quantitative data descriptively and themed the commitments. Approximately 40% (35/81) of the participants completed CTCs at the end of the programme. Over 75% (n = 27) provided complete data for 79 changes (2.9 per doctor) 3 months after the course. Initial commitment to making the changes was high at 3.36/4. At 3 months, 33 (41.8%) of the changes had been implemented, 24 (30.4%) had been partially implemented, 21 (26.6%) could not be implemented, and one (1.3%) had been abandoned. Making changes in approaches to teaching was identified in 31 CTCs. These included improving communication skills (by spending more time on listening), enabling residents to teach students, and adopting new competency-based assessment tools and new approaches to teaching procedural skills. A total of 19 CTCs related to assessment and feedback (e.g. adopting feedback frameworks taught, giving specific or regular feedback, planning feedback). There were 15 commitments to improve the use of information technology (IT) and electronic resources (e.g. setting up a website for practice, using websites, creating lists of websites for resident or patient teaching). There were nine commitments to improve role-modelling, most of which related to demonstrating the benefits of family medicine as a career. The final five commitments focused on office management for effectiveness and error disclosure. The CTCs that the doctors were able to implement and carry out themselves (e.g. improving communication or observation of trainees) were likely to have been adopted fully or partially. By contrast, many of the 22 changes the doctors had been unable to implement or had abandoned were more complex and required additional time or the opportunity to implement. Almost half (7/15) of the plans for IT and electronic resources were incomplete or had been abandoned, whereas only a third (12/31) of plans relating to approaches to teaching had not succeeded. The CTC process provided new data with which to assess our programme because the CTCs were directly associated with the content provided. Changes were in the directions intended. Encouraging all attendees to complete their CTC forms is critical to understanding programme impact.
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Comment cette classification a été obtenuedéplier
Prédiction machine sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Le volet Gemma est une étiquette directe du modèle pour chaque travail de la base, lue sur la notice réduite au titre. Le volet Codex est un classifieur appris des 10 348 étiquettes directes de Codex et calibré sur les taux pondérés de l'échantillon; les champs sans appui suffisant ne portent aucun appel Codex. Le mode candidate est l'union des deux volets; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont pas des étiquettes humaines.
Scores du classifieur distillé par catégorie (deux têtes)
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,028 | 0,073 |
| Méta-épidémiologie (sens strict) | 0,001 | 0,001 |
| Méta-épidémiologie (sens large) | 0,001 | 0,002 |
| Bibliométrie | 0,007 | 0,004 |
| Études des sciences et des technologies | 0,002 | 0,003 |
| Communication savante | 0,003 | 0,003 |
| Science ouverte | 0,002 | 0,006 |
| Intégrité de la recherche | 0,002 | 0,003 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,004 | 0,001 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découleClassification
machine, non validéePrédiction automatique; un appel candidat d’une seule source (Gemma direct ou Codex distillé), pas un consensus.
Le détail, modèle par modèle et score par score, se trouve en fin de page sous « Comment cette classification a été obtenue ».