PD58-04 MODELING AUTOMATED ASSESSMENT OF SURGICAL PERFORMANCE UTILIZING COMPUTER VISION: PROOF OF CONCEPT
Bibliographic record
Abstract
You have accessJournal of UrologySurgical Technology & Simulation: Training & Skills Assessment II1 Apr 2018PD58-04 MODELING AUTOMATED ASSESSMENT OF SURGICAL PERFORMANCE UTILIZING COMPUTER VISION: PROOF OF CONCEPT Amir Baghdadi, Lora Cavuoto, Ahmed Aly Hussein, Youssef Ahmed, and Khurshid Guru Amir BaghdadiAmir Baghdadi More articles by this author , Lora CavuotoLora Cavuoto More articles by this author , Ahmed Aly HusseinAhmed Aly Hussein More articles by this author , Youssef AhmedYoussef Ahmed More articles by this author , and Khurshid GuruKhurshid Guru More articles by this author View All Author Informationhttps://doi.org/10.1016/j.juro.2018.02.2792AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookTwitterLinked InEmail INTRODUCTION AND OBJECTIVES Thorough lymph node dissection (LND) is an integral part of robot-assisted radical cystectomy (RARC). There is a lack of consensus about what constitutes adequate LND. Current methods are subject to inter-rater variability. In this context, we sought to use computer vision methods to identify and extract valid measures to develop and validate an automated scoring system for LND. METHODS 20 recorded LNDs were included with a total of 200 frames/case from the console feed before and after LND with near-equivalent view and zoom. The quality of lymph node clearance was assessed based on the features derived from a computer vision algorithm: the number and area of the nerve/vessels (N-Vs) detected using the proposed Automated Structure Detection (ASD) method; image median Color Map by the assumption of decrease in yellow color after lymphatic and fatty tissue removal; and mean entropy, which measures the level of disorganization in the image. Each video frame was pre-processed for N-Vs detection using a series of image processing operations including binary conversion, edge and line detection, and object identification while considering geometrical characteristics of target objects, e.g. aspect ratio, width, height, and orientation. The N-Vs were labeled by fusing the information from both line and object detection processes (Figure 1). The automated scores (AS) were compared to Pelvic Lymphadenectomy Appropriateness and Completion Evaluation (PLACE), which is a subjective evaluation based on objective measures scored by a panel of expert surgeons. Logistic regression analysis was employed to compare AS and PLACE scores. RESULTS 14 cases were used to develop the automated scoring algorithm. A logistic regression model was trained and validated using the aforementioned features with 30% holdout cross validation. This model was applied to the remaining 6 previously unseen cases for testing and the accuracy of predicting the PLACE scores was 83.3% (5 correct score allocation across the 6 test cases). CONCLUSIONS To our knowledge,this is the first automated surgical skill assessment tool that provides objective evaluation of surgical performance with high accuracy compared to expert surgeon assessment. © 2018FiguresReferencesRelatedDetails Volume 199Issue 4SApril 2018Page: e1134-e1135 Advertisement Copyright & Permissions© 2018MetricsAuthor Information Amir Baghdadi More articles by this author Lora Cavuoto More articles by this author Ahmed Aly Hussein More articles by this author Youssef Ahmed More articles by this author Khurshid Guru More articles by this author Expand All Advertisement Advertisement PDF downloadLoading ...
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.003 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.001 |
| Bibliometrics | 0.001 | 0.000 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.025 | 0.006 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".