A19 DEVELOPING A COMPETENCY-BASED PERFORMANCE METRIC OF COLONOSCOPY SKILLS ACQUISITION USING MOTION ANALYSIS - STEP 1: LOW-FIDELITY BENCHTOP MODEL
Bibliographic record
Abstract
Colonoscopy is essential for the diagnosis and treatment of colonic diseases. Simulation is increasingly prevalent in colonoscopy skills acquisition as it allows controlled experiential learning without risk to patients. However, objectively assessing when trainees achieve the competence required to proceed to patients remains an outstanding challenge. To establish a novel metric using advanced motion analysis to objectively assess colonoscopy skills acquisition across multiple simulation models. This pilot study assessed the difference between experienced (N=9) and novice endoscopists (N=20) before and after training on a low-fidelity bench-top colonoscopy simulator developed by Walsh et al. (2009) which is designed to teach basic endoscope handling skills. Experienced colonoscopists were asked to scope four different courses with two repetitions in a random sequence to define the benchmark workflow. Novice endoscopists were asked to scope the same course prior to and after completing a 1-hour practice session. Execution time in seconds, hand path length in meters and number of hand movements were recorded using electromagnetic position sensors attached to the left and right hands and forearms. Before practice novice endoscopists had average right hand (RH) path length, number RH movements, and total time of 6.42m, 48.7, and 153s, respectively. This was significantly higher than expert endoscopist values of 2.91m, 13.6 and 39.9s (p < 0.001 for all). Additionally, pre-practice values were significantly higher than post-practice values of 3.24m, 21.1 and 59.0s (p=0.002, p=0.003, p=0.002, respectively). Fifteen of 20 novice endoscopists reached competency after 1-hour of practice on this model; competency was defined using the second lowest performing expert’s scores. Figure 1 shows execution time of the top and bottom quartile of novice endoscopists after practice compared to the mean of experienced endoscopists. We are the first group to propose a novel platform to assess colonoscopy skills acquisition using motion analysis. Novice endoscopists reliably achieve competency in basic endoscopic handling using a benchmark developed from experienced colonoscopists on a low-fidelity colonoscopy model. The range of novice endoscopist performances after 1-hour of practice reinforces the importance of competency-based assessment. We aim to apply our competency metric to objectively assess all stages of colonoscopy skills acquisition. Southeastern Ontario Academic Medical Organization
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.008 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.001 |
| Bibliometrics | 0.001 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.000 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".