Abstract 4357888: Evaluating the Temporal and Sociodemographic Generalizability of the Emergency Heart Failure Mortality Risk Grade
Bibliographic record
Abstract
Background: The Emergency Heart Failure Mortality Risk Grade (EHMRG) models are a series of clinically validated models to predict (7-day and 30-day) mortality in acute heart failure (AHF) patients. Although EHMRG models were validated in 2015, it is unknown whether EHMRG maintained temporal validity, and if they maintain accuracy in different sociodemographic strata. Objectives: To assess the temporal validity of EHMRG models using recently collected data and determine if EHMRG models perform differently across sociodemographic subgroups Methods: We selected a sample of 7537 patients presenting to 10 emergency departments with an AHF diagnosis between 2017-2019 in Ontario Canada, from the Comparison of Outcomes and Access to Care for Heart Failure Trial (COACH). We linked our cohort to national databases to obtain census-derived sociodemographic factors. We calculated the EHMRG-7 (7-day mortality), and EHMRG30-ST (30-day mortality) scores for each patient, and assessed the performance of those models using discrimination metrics and calibration plots comparing the observed and predicted probabilities of death across the deciles of risk. We then compared the discriminatory performance across several demographic subgroups. Results: In our independent temporal validation cohort (median age 80 [IQR, 70-87] years, 48% females), EHMRG-7 had a high discrimination c-statistic = 0.795 [95%CI, 0.764 – 0.824] and demonstrated good calibration with minor overestimation of the 7-day risk of death in the highest decile. EHMRG30-ST had a good discrimination, with a c-statistic = 0.777 [95%CI, 0.759 – 0.7969], but overestimated the risk of 30-day death across the upper five deciles. The discrimination of both EHMRG models was robust across subgroups of sex, marital status, income quintiles, and census derived factors, as demonstrated by comparable c-statistics in these subgroups. However, the c-statistic was lower for patients residing in retirement or long-term care homes. (Figure 1) Conclusions: EHMRG models retained good discrimination comparable to previous validations. Risk was overestimated at 30 days, which would lead to more conservative recommendations tending towards hospital admission. EHMRG models were robust over the majority of sociodemographic characteristics.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.019 | 0.061 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.002 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".