Agreement and prognostic accuracy of three ED vulnerability screeners: findings from a prospective multi-site cohort study
Bibliographic record
Abstract
Abstract Objectives To evaluate the agreement between three emergency department (ED) vulnerability screeners, including the InterRAI ED Screener, ER2, and PRISMA-7. Our secondary objective was to evaluate the discriminative accuracy of screeners in predicting discharge home and extended ED lengths-of-stay (> 24 h). Methods We conducted a nested sub-group study using data from a prospective multi-site cohort study evaluating frailty in older ED patients presenting to four Quebec hospitals. Research nurses assessed patients consecutively with the three screeners. We employed Cohen's Kappa to determine agreement, with high-risk cut-offs of three and four for the PRISMA-7, six for the ER2, and five for the interRAI ED Screener. We used logistic regression to evaluate the discriminative accuracy of instruments, testing them in their dichotomous, full, and adjusted forms (adjusting for age, sex, and hospital academic status). Results We evaluated 1855 older ED patients across the four hospital sites. The mean age of our sample was 84 years. Agreement between the interRAI ED Screener and the ER2 was fair (K = 0.37; 95% CI 0.33–0.40); agreement between the PRISMA-7 and ER2 was also fair (K = 0.39; 95% CI = 0.36–0.43). Agreement between interRAI ED Screener and PRISMA-7 was poor (K = 0.19; 95% CI 0.16–0.22). Using a cut-off of four for PRISMA-7 improved agreement with the ER2 (K = 0.55; 95% CI 0.51–0.59) and the ED Screener (K = 0.32; 95% CI 0.2–0.36). When predicting discharge home, the concordance statistics among models were similar in their dichotomous (c = 0.57–0.61), full (c = 0.61–0.64), and adjusted forms (c = 0.63–0.65), and poor for all models when predicting extended length-of-stay. Conclusion ED vulnerability scores from the three instruments had a fair agreement and were associated with important patient outcomes. The interRAI ED Screener best identifies older ED patients at greatest risk, while the PRISMA-7 and ER2 are more sensitive instruments.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.013 | 0.029 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".