Use of an Objective Structured Assessment of Technical Skill (OSATS) after a Sports Rotation
Notice bibliographique
Résumé
Objectives: While the acquisition of competent technical skill is a defining characteristic of surgery, it is not measured systematically in residency. As all post-graduate medical training slowly shifts to a competency-based model, effective assessments of surgical and other technical skills after modules will become necessary. However, the best method for of assessing competence in technical skill in this setting is unknown, and is limited by both cost and access to resources. We hypothesized that a multi-station Objective Structured Assessment of Technical skill (OSATS), using sawbones models, would be a valid and reliable method of assessing resident competence in surgical skills after a sports medicine rotation. Methods: At the start of their three-month sports medicine rotation, each resident was provided a list of 10 surgical skills in which they were expected to demonstrate competence (Table 1). At the end of the rotation, each resident undertook an OSATS comprised of six randomly chosen stations - low-fidelity sawbones models were used in all stations. Residents were evaluated by faculty / staff surgeons using a previously validated global rating scale (the Arthroscopic Surgical Skill Evaluation Tool (ASSET)), as well as task-specific checklists created using a modified Delphi procedure, and a final five-point global rating scale (GRS) using the Drefus model of skill acquisition (1=novice, 2=advanced beginner, 3=competent, 4=proficient, 5=expert). All arthroscopic procedures were recorded, and all hand movements were videotaped - the videos were reviewed by a single, blinded observer, and correlation sought between the faculty ratings and the observer ratings. Results: Over 18 months, 27 residents (19 junior, 8 senior) sat the OSATS after their rotation, as well as seven sports medicine staff and seven fellows, for a total of 41 participants. The overall reliability of the OSATS as measured by Cronbach's Alpha was very high (0.9). A significant difference by year in training was seen for the overall GRS, the total ASSET score, and the total checklist score, as well as for each individual technical skill (p<0.001) - this difference was also seen for all stations. Post hoc analysis demonstrated a significant difference in the total ASSET score between junior (PGY1-3) and senior (PGY4&5) residents, senior residents and fellows, as well as between fellows and faculty (p<0.05)(Figure 1). A high correlation was seen between the faculty assessments and the blinded observer assessments for each station (>0.8). Conclusion: The results of this study demonstrate that an OSATS using dry models is a valid and reliable means of assessing technical skill in orthopaedic residents after a sports medical rotation. Interestingly, junior residents were not able to perform technical skills as well as senior residents despite an identical rotation, suggesting that overall surgical experience and exposure is as important as intensive teaching.
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Comment cette classification a été obtenuedéplier
Prédiction machine sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Le volet Gemma est une étiquette directe du modèle pour chaque travail de la base, lue sur la notice réduite au titre. Le volet Codex est un classifieur appris des 10 348 étiquettes directes de Codex et calibré sur les taux pondérés de l'échantillon; les champs sans appui suffisant ne portent aucun appel Codex. Le mode candidate est l'union des deux volets; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont pas des étiquettes humaines.
Scores du classifieur distillé par catégorie (deux têtes)
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,003 | 0,007 |
| Méta-épidémiologie (sens strict) | 0,001 | 0,000 |
| Méta-épidémiologie (sens large) | 0,000 | 0,001 |
| Bibliométrie | 0,001 | 0,000 |
| Études des sciences et des technologies | 0,000 | 0,000 |
| Communication savante | 0,000 | 0,001 |
| Science ouverte | 0,000 | 0,001 |
| Intégrité de la recherche | 0,000 | 0,000 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,002 | 0,001 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découleClassification
machine, non validéePrédiction automatique; un appel candidat d’une seule source (Gemma direct ou Codex distillé), pas un consensus.
Le détail, modèle par modèle et score par score, se trouve en fin de page sous « Comment cette classification a été obtenue ».