Video-analysis for the assessment of practical skill
Bibliographic record
Abstract
To review the literature on the application of video review and analytics on surgical education and quality improvement. Analysis of past performance is a mandatory component in many industries, yet the idea is in its infancy in surgical assessment. Evaluation of surgical skill, both technical and non-technical, is possible through video analysis methods. Adverse outcomes in surgery are related in part to errors committed by the surgical team, and review of intraoperative footage allows for detailed analysis and improvement of skills and systems that contribute to patient safety. In this article we review the literature pertaining to post hoc assessment of surgical performance, including technical skill and error, and non-technical skill and surgical coaching. We describe our group’s novel ‘OR blackbox’ method of detailed video analysis, and how we synthesize multiple metrics of performance collected through audiovisual pathways into compartmentalized, usable data. Qualitative and Quantitative video analysis has been applied to multiple fields of surgery. Tools for assessment of metrics across the spectrum of intraoperative factors exist and their validity and reliability is supported in the literature. Emerging evidence supports the use of retrospective evaluation of surgical technique in ensuring optimal outcomes for patients. Educators are using video analysis to identify aspects of surgical procedures to target for deliberate practice, and the role of coaching in surgery is greatly enhanced with review of cases. Identification of crucial events related to safety and surgical skill are not always identifiable in real time. Video analysis allows surgeons and educators to assess intraoperative factors that influence the patient’s surgical outcomes and safety. Assessment of technical skill, non-technical skill, and surgical error are possible through comprehensive video recording and analysis techniques.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.007 | 0.037 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.009 | 0.005 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.003 | 0.003 |
| Open science | 0.001 | 0.002 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.019 | 0.007 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".