Mortality predictions in the intensive care unit: Comparing physicians with scoring systems*
Bibliographic record
Abstract
OBJECTIVE: Risk-prediction models offer potential advantages over physician predictions of outcomes in the intensive care unit (ICU). Our systematic review compared the accuracy of ICU physicians' and scoring system predictions of ICU or hospital mortality of critically ill adults. DATA SOURCE: MEDLINE (1966-2005), CINAHL (1982-2005), Ovid Healthstar (1975-2004), EMBASE (1980-2005), SciSearch (1980-2005), PsychLit (1985-2004), the Cochrane Library (Issue 1, 2005), PubMed "related articles," personal files, abstract proceedings, and reference lists. STUDY SELECTION: We considered all studies that compared physician predictions of ICU or hospital survival of critically ill adults to an objective scoring system, computer model, or prediction rule. We excluded studies if they focused exclusively on the development or economic evaluation of a scoring system, computer model, or prediction rule. DATA EXTRACTION AND ANALYSIS: We independently abstracted data and assessed study quality in duplicate. We determined summary receiver operating characteristic curves and areas under the summary receiver operating characteristic curves+/-se and summary diagnostic odds ratios. DATA SYNTHESIS: We included 12 observational studies of moderate methodological quality. The area under the summary receiver operating characteristic curves for seven studies was 0.85+/-0.03 for physician predictions compared with 0.63+/-0.06 for scoring system predictions (p=.002). Physicians' summary diagnostic odds ratios derived from the area under the summary receiver operating characteristic curves were significantly higher (12.43; 95% confidence interval 5.47, 27.11) than scoring systems' summary diagnostic odds ratios (2.25; 95% confidence interval 0.78, 6.52, p=.001). Combined results of all 12 studies indicated that physicians predict mortality more accurately than do scoring systems: ratio of diagnostic odds ratios (95% confidence interval) 1.92 (1.19, 3.08) (p=.007). CONCLUSIONS: Observational studies suggest that ICU physicians discriminate between survivors and nonsurvivors more accurately than do scoring systems in the first 24 hrs of ICU admission. The overall accuracy of both predictions of patient mortality was moderate, implying limited usefulness of outcome prediction in the first 24 hrs for clinical decision making.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.066 | 0.308 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.007 | 0.011 |
| Bibliometrics | 0.008 | 0.009 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.005 | 0.004 |
| Open science | 0.002 | 0.001 |
| Research integrity | 0.002 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".