Development of an Operative Performance Rating System for Plastic Surgery Residents
Bibliographic record
Abstract
PURPOSE A standardized measure of operative performance is an essential component of the Patient Care Competency and is critical to the training of plastic surgery residents. The Operative Performance Rating System (OPRS) developed and validated by the Department of General Surgery at Southern Illinois University consists of procedure-specific evaluations for resident intraoperative performance. The OPRS provides an objective measure of procedure-specific resident performance that is not currently being assessed in plastic surgery training programs. The purpose of this study is to describe OPRS for plastic surgery residents and propose methodology for assessing the reliability, validity, and feasibility of this instrument. METHODS Ten procedure-specific rating instruments were developed for sentinel cases, each consisting of critical procedure-specific steps based on literature review and faculty focus group consensus. Sentinel cases were chosen based on review of the American Council of Graduate Medical Education Milestones and resident logs of the most commonly performed plastic surgery procedures, both at our institution and nationally. The degree of guidance required from the attending surgeon is recorded for each step. General operative performance competency is evaluated from validated items developed by the University of Toronto.1 All items use a 5-point Likert scale with behavioral anchors. The OPRS assessments will be incorporated into the internet-based resident management platform New Innovations. Sentinel procedures for evaluation will be identified on a weekly basis by the residency coordinator, based on resident operative assignments (postgraduate year 2–6) organized by the chief resident. OPRS assessment forms will be available electronically immediately following the procedures, with an e-mail reminder notification 24 hours later to help encourage compliance. In addition, resident self-assessment using the same OPRS will be conducted and correlated with faculty OPRS evaluations. Each OPRS assessment will be evaluated for internal consistency reliability and inter-item correlation. Inter-rater reliability will be measured by faculty assessment of videotaped sentinel procedures using the appropriate OPRS instrument. Performance variation based on resident PGY level will be analyzed using 1-way analysis of variance. Feasibility will also be determined based on attending and resident response rates and time to completion for the OPRS evaluations, as well as a short written survey to assess resident and attending satisfaction and obtain feedback following OPRS implementation. CONCLUSIONS A web-based OPRS provides timely and objective feedback to improve residents’ technical and decision-making skills, as demonstrated by the experiences of other surgical specialties.2 This instrument will provide both formative and summative resident feedback, encouraging faculty and residents to focus on demonstrated competencies and areas for improvement.3 Furthermore, resident operative performance can be monitored across time and residents, allowing program directors to have a long-term objective method of evaluating resident technical performance.3 A reliable and valid OPRS may provide a feasible method of intraoperative assessment that could be implemented across all plastic surgery training programs.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.005 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".