A comparison of methods to obtain a composite performance indicator for evaluating clinical processes in trauma care
Bibliographic record
Abstract
BACKGROUND: Process performance indicators that evaluate trauma centers in clinical case management provide information essential to the improvement of trauma care. However, multiple indicators are needed to adequately evaluate process performance, which renders comparisons cumbersome. Several methods are available for generating composite indicators that measure global performance. The goal of this study was to compare three composite methods that are widely used in other health care domains to identify the most appropriate for trauma care process performance evaluation. METHODS: In this retrospective, multicenter cohort study, 15 process performance indicators were implemented using data from a Canadian provincial trauma registry (19,853 patients; 59 centers) on patients with an Injury Severity Score (ISS) greater than 15. Composite scores were derived using three methods as follows: the indicator average, the opportunity model, and a latent variable model. Composite scores were evaluated in terms of discrimination, construct validity (association with an indicator of trauma center structural performance), criterion predictive validity (association with clinical outcomes), and forecasting (correlation over time). RESULTS: All composite scores discriminated well between trauma centers. Only the average indicator score was correlated with improved structure (r = 0.29; 95% confidence interval [CI], 0.07-0.53), lower risk-adjusted mortality (r = -0.22; 95% CI, -0.46 to 0.04), and lower risk-adjusted complication rate (r = -0.48; 95% CI, -0.65 to -0.25). Composite scores calculated with 1999 to 2002 data all correlated with those calculated with 2003 to 2006 data (r = 0.49, 0.87, and 0.84 for the indicator average, the opportunity model, and the latent variable model, respectively). CONCLUSION: Results suggest that of the three composite scores evaluated, only the indicator average demonstrates content and predictive criterion validity, discriminates between centers, and has good forecasting properties. In addition, this score is simple and intuitive and not subject to variation in weights over trauma systems and time. The observed association between higher indicator average scores and lower risk-adjusted mortality and complication rates suggests that improving process performance may improve patient outcome.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".