Artificial Intelligence Assessment of Expertise in Virtual Reality Spine Pedicle Screw Insertion

IMPORTANCE: Our understanding of the composites of technical expertise during spinal procedures including the insertion of pedicle screws is incomplete. Datasets generated from surgical simulation allows the quantitation of psychomotor skills, which can be analyzed using machine learning algorithms which allows a more complete understanding of surgical performance.OBJECTIVE: The primary aim of this study was to identify important features distinguishing skilled and less skilled levels of expertise during simulated pedicle screw insertion. The secondary aim was to benchmark the classification accuracy of surgical performance through the implementation of machine learning algorithms.DESIGN: Participants from four universities were recruited between July 15, 2022, and May 31, 2023, to participate in a case-series study. Data were collected over a single time point and no follow-up data were collected. Participants were classified a priori as either skilled or less skilled based on their experience in performing human pedicle screw insertion procedures.SETTING: McGill University Neurosurgical Simulation and Artificial Intelligence Learning Centre.PARTICIPANTS: Forty-three neurosurgery and orthopedic spine surgeons, spine fellows, and neurosurgery and orthopedic residents.INTERVENTION: Insertion of bilateral L5 and L4 pedicle screw insertions on a virtual reality platform resulting in 172 inserted screws for analysis. These 172 datapoints were divided into training set (70% - 121 data points) and testing set (30% -51 data points) for algorithm’s training and testing. We used 5-fold cross validation to validate the algorithm.EXPOSURES: All participants performed a simulated virtual reality L5-L4 bilateral pedicle screw insertion during which they each inserted 4 screws.MAIN OUTCOMES AND MEASURES The main outcomes and measures were determined through an iterative process, wherein features related to instrument movement, force application, and tissue resection were chosen from the raw simulator data output. This selection was achieved through a combination of four feature selection methods, wrapper-based, embedded, filter-based, and weight-based, in conjunction with Support Vector Machine (SVM), Random Forest, K-Nearest Neighbor (KNN), and Artificial Neural Network (ANN) models. The objective was to accurately assess the skill levels of participants in simulated pedicle screw insertion.RESULTS A cohort of 43 participants, including 5 women and 38 men with a mean age of 33.6 years (SD 9.5), was evaluated. Machine learning models demonstrated varying accuracies on the test set: SVM achieved 78%, Random Forest 80%, KNN 82.3%, and ANN 82.3%. Analysis revealed 24 common features across Random Forest, KNN, and ANN, each achieving a classification accuracy of over 80%.CONCLUSIONS AND RELEVANCE By employing machine learning algorithms, our study identified key features that may determine components of expertise during simulated pedicle screw insertion. We introduced a combined approach for feature selection that could enhance the accuracy of classifying skilled versus less skilled performance in future experiments. This method may prove valuable in the assessment and training of various surgical procedures

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.