Effect of artificial intelligence-augmented human instruction on surgical simulation performance: A randomized controlled trial
Notice bibliographique
Résumé
BackgroundWith current surgical teaching models’ lack of standardization and reliance on subjective assessments by human experts rather than quantitative performance data, training novices to master surgical technical skills remains challenging. To mitigate this issue, we developed an artificial intelligence (AI) application known as the Intelligent Continuous Expertise Monitoring System (ICEMS) capable of assessing bimanual surgical skills at 0.2-second intervals and providing continuous, action-oriented verbal feedback.ObjectivesThe objective of this study is to determine the effect of AI-augmented personalized expert instruction versus AI tutor instruction alone on surgical performance, skill transfer, and affective-cognitive responses.MethodsA multi-institutional randomized controlled trial was conducted wherein medical students performed subpial brain tumour resection tasks on the NeuroVR and received real-time feedback on their performance. Students were stratified based on their year in medical school and block randomized to one of three groups. Group 1 received AI tutor instruction delivered by the ICEMS, group 2 received expert feedback in identical words to the ICEMS, and group 3 received AI data-informed personalized expert feedback. Trainees performed six practice subpial resection tasks to assess learning followed by a complex realistic brain tumour resection scenario to assess skill transfer. The ICEMS quantitatively evaluated trainee performance. Participants self-reported emotions before, during, and after training and cognitive load after training via questionnaires.ResultsEighty-seven medical students from four Quebec institutions were randomly assigned to the AI instruction (n = 30), expert instruction (n = 29), and personalized expert instruction (n = 28) groups. The ICEMS assessed and scored 522 practice resections and 87 realistic resections. During the practice tasks, personalized expert instruction resulted in significantly greater expertise scores than AI tutor instruction across several trials, including trial 5 (mean difference, 0.26 [95% CI, 0.09 to 0.43]; P = 0.01). During the realistic task, the personalized instruction group had significantly higher expertise scores than both the AI tutor instruction (mean difference, 0.20 [95% CI, 0.06 to 0.34]; P = 0.02) and expert instruction (mean difference, 0.18 [95% CI, 0.03 to 0.32]; P = 0.049) groups. The personalized expert instruction group also achieved significantly higher scores than the other two groups in certain metrics, such as bleeding and injury risk. Emotions and cognitive load demonstrated significant differences.ConclusionPersonalized expert instruction resulted in enhanced surgical performance and skill transfer compared with intelligent tutor instruction, highlighting the importance of human input and active participation in AI-based surgical training
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Comment cette classification a été obtenuedéplier
Prédiction machine sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Le volet Gemma est une étiquette directe du modèle pour chaque travail de la base, lue sur la notice réduite au titre. Le volet Codex est un classifieur appris des 10 348 étiquettes directes de Codex et calibré sur les taux pondérés de l'échantillon; les champs sans appui suffisant ne portent aucun appel Codex. Le mode candidate est l'union des deux volets; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont pas des étiquettes humaines.
Scores du classifieur distillé par catégorie (deux têtes)
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,006 | 0,008 |
| Méta-épidémiologie (sens strict) | 0,002 | 0,001 |
| Méta-épidémiologie (sens large) | 0,004 | 0,002 |
| Bibliométrie | 0,001 | 0,001 |
| Études des sciences et des technologies | 0,001 | 0,003 |
| Communication savante | 0,001 | 0,001 |
| Science ouverte | 0,002 | 0,001 |
| Intégrité de la recherche | 0,003 | 0,003 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,008 | 0,001 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découleClassification
machine, non validéePrédiction automatique; un appel candidat d’une seule source (Gemma direct ou Codex distillé), pas un consensus.
Le détail, modèle par modèle et score par score, se trouve en fin de page sous « Comment cette classification a été obtenue ».