Effect of artificial intelligence-augmented human instruction on surgical simulation performance: A randomized controlled trial
Bibliographic record
Abstract
BackgroundWith current surgical teaching models’ lack of standardization and reliance on subjective assessments by human experts rather than quantitative performance data, training novices to master surgical technical skills remains challenging. To mitigate this issue, we developed an artificial intelligence (AI) application known as the Intelligent Continuous Expertise Monitoring System (ICEMS) capable of assessing bimanual surgical skills at 0.2-second intervals and providing continuous, action-oriented verbal feedback.ObjectivesThe objective of this study is to determine the effect of AI-augmented personalized expert instruction versus AI tutor instruction alone on surgical performance, skill transfer, and affective-cognitive responses.MethodsA multi-institutional randomized controlled trial was conducted wherein medical students performed subpial brain tumour resection tasks on the NeuroVR and received real-time feedback on their performance. Students were stratified based on their year in medical school and block randomized to one of three groups. Group 1 received AI tutor instruction delivered by the ICEMS, group 2 received expert feedback in identical words to the ICEMS, and group 3 received AI data-informed personalized expert feedback. Trainees performed six practice subpial resection tasks to assess learning followed by a complex realistic brain tumour resection scenario to assess skill transfer. The ICEMS quantitatively evaluated trainee performance. Participants self-reported emotions before, during, and after training and cognitive load after training via questionnaires.ResultsEighty-seven medical students from four Quebec institutions were randomly assigned to the AI instruction (n = 30), expert instruction (n = 29), and personalized expert instruction (n = 28) groups. The ICEMS assessed and scored 522 practice resections and 87 realistic resections. During the practice tasks, personalized expert instruction resulted in significantly greater expertise scores than AI tutor instruction across several trials, including trial 5 (mean difference, 0.26 [95% CI, 0.09 to 0.43]; P = 0.01). During the realistic task, the personalized instruction group had significantly higher expertise scores than both the AI tutor instruction (mean difference, 0.20 [95% CI, 0.06 to 0.34]; P = 0.02) and expert instruction (mean difference, 0.18 [95% CI, 0.03 to 0.32]; P = 0.049) groups. The personalized expert instruction group also achieved significantly higher scores than the other two groups in certain metrics, such as bleeding and injury risk. Emotions and cognitive load demonstrated significant differences.ConclusionPersonalized expert instruction resulted in enhanced surgical performance and skill transfer compared with intelligent tutor instruction, highlighting the importance of human input and active participation in AI-based surgical training
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.006 | 0.008 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.004 | 0.002 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.003 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.002 | 0.001 |
| Research integrity | 0.003 | 0.003 |
| Insufficient payload (model declined to judge) | 0.008 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".