MP35-19 UTILITY AND FEASIBILITY OF VIDEO-BASED ASSESSMENT FOR URETEROSCOPY AND LASER LITHOTRIPSY
Notice bibliographique
Résumé
You have accessJournal of UrologySurgical Technology & Simulation: Training & Skills Assessment II (MP35)1 Apr 2019MP35-19 UTILITY AND FEASIBILITY OF VIDEO-BASED ASSESSMENT FOR URETEROSCOPY AND LASER LITHOTRIPSY Yuding Wang*, Kelly Dore, Dana Russell, Jen Hoogenes, Bobby Shayegan, Nathan Wong, and Edward Matsumoto Yuding Wang*Yuding Wang* More articles by this author , Kelly DoreKelly Dore More articles by this author , Dana RussellDana Russell More articles by this author , Jen HoogenesJen Hoogenes More articles by this author , Bobby ShayeganBobby Shayegan More articles by this author , Nathan WongNathan Wong More articles by this author , and Edward MatsumotoEdward Matsumoto More articles by this author View All Author Informationhttps://doi.org/10.1097/01.JU.0000556009.17548.baAboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVES: Assessment plays a central role in competency-based medical education. The operating room (OR) is a demanding environment with many competing priorities placed on the primary surgeon, one being the assessment of resident performance. Previous studies show that assessment reliability decreases with increased task complexity. The objective of this study was to explore the utility of video-based assessment of resident performance for ureteroscopy and laser lithotripsy. METHODS: Over a 6-month period, 15 ureteroscopy and laser lithotripsy cases performed by urology residents under staff surgeon guidance were captured on video using a 3-camera setup capturing the surgeon’s view, the endoscopic view, and a 360-degree view of the OR. Following the case, the staff surgeon assessed resident performance using the 11-item entrustment-based Ottawa Surgical Competency Operating Room Score Evaluation (O-SCORE), and evaluated his or her own cognitive load during the case using the Surgical Task Load Index (SURG-TLX). Two independent staff urologists reviewed each video using the same assessment tools. Raters’ results were compared with those of the case surgeon. A post-assessment survey and interview were conducted with the case surgeon and the video raters. RESULTS: Over 15 cases, 630 minutes of video was collected of 7 residents and 5 staff surgeons. Six cases were completed by junior residents and 9 by senior residents. The interrater reliability between video assessors was highly correlated (k=0.82), while comparing between intraoperative assessment and video-based assessment, mean O-SCOREs were also highly correlated (r=0.7). When assessing O-SCORE domains for intraoperative performance, there was a slight decrease in correlation between the intraoperative assessment and video-based assessment (r=0.68). All assessors expressed low cognitive load during viewing or while scrubbed into the case with mean SURG-TLX scores of 5 (SD ±1.8). Video-based reviewers frequently rewound the video during critical steps of each case to assess resident performance. CONCLUSIONS: This study showed that video-based assessment of resident performance during ureteroscopy and laser lithotripsy using the O-SCORE is useful and feasible, with high interrater reliability among the case surgeon and independent video reviewers. Use of the SURG-TLX provided new insight into the cognitive load of both case surgeons and video reviewers. Additional research with other types of surgical cases are required to further explore the use of video-based assessment in the OR. Source of Funding: McMaster University Surgical Associates Hamilton, Canada© 2019 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 201Issue Supplement 4April 2019Page: e513-e513 Advertisement Copyright & Permissions© 2019 by American Urological Association Education and Research, Inc.MetricsAuthor Information Yuding Wang* More articles by this author Kelly Dore More articles by this author Dana Russell More articles by this author Jen Hoogenes More articles by this author Bobby Shayegan More articles by this author Nathan Wong More articles by this author Edward Matsumoto More articles by this author Expand All Advertisement PDF downloadLoading ...
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Comment cette classification a été obtenuedéplier
Prédiction machine sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Le volet Gemma est une étiquette directe du modèle pour chaque travail de la base, lue sur la notice réduite au titre. Le volet Codex est un classifieur appris des 10 348 étiquettes directes de Codex et calibré sur les taux pondérés de l'échantillon; les champs sans appui suffisant ne portent aucun appel Codex. Le mode candidate est l'union des deux volets; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont pas des étiquettes humaines.
Scores du classifieur distillé par catégorie (deux têtes)
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,005 | 0,023 |
| Méta-épidémiologie (sens strict) | 0,000 | 0,000 |
| Méta-épidémiologie (sens large) | 0,000 | 0,001 |
| Bibliométrie | 0,001 | 0,000 |
| Études des sciences et des technologies | 0,000 | 0,000 |
| Communication savante | 0,001 | 0,001 |
| Science ouverte | 0,001 | 0,001 |
| Intégrité de la recherche | 0,000 | 0,000 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,036 | 0,006 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découleClassification
machine, non validéePrédiction automatique; un appel candidat d’une seule source (Gemma direct ou Codex distillé), pas un consensus.
Le détail, modèle par modèle et score par score, se trouve en fin de page sous « Comment cette classification a été obtenue ».