Derivation and Internal Validation of a Health Administrative Data Algorithm for Identifying Sepsis
Bibliographic record
Abstract
Objectives: The primary objective was to externally validate a Canadian ICD-coded sepsis case definition developed by Jolley et al. by examining its sensitivity and specificity in health administrative data among intensive care unit (ICU) patients at The Ottawa Hospital (TOH). We also evaluated whether two modifications (the addition of antimicrobial information and the removal of infection-related ICD codes) to the Jolley algorithm could improve sensitivity while maintaining specificity. The secondary objective was to examine the impact of these modifications on the algorithm's positive and negative predictive values. Methods: We identified adults (18 years and older) admitted to TOH's Civic and General campus ICUs between April 1, 2014 and March 31, 2019. From this population, a random sample of 870 medical charts was selected for manual chart review, with sepsis positive and negative reference cases classified according to the Sepsis-3 criteria. Chart review data were linked to a health administrative dataset, containing pharmacy records, diagnosis and procedure codes. We then evaluated the sensitivity and specificity of the Jolley algorithm and its modified versions incorporating antimicrobial information and/or excluding infection codes. Results: Among 10,407 eligible ICU patients, 833 of the 870 charts met the eligibility criteria. Of these, 391 patients met Sepsis-3 criteria through chart review, while 364 patients met Jolley sepsis criteria. The original Jolley algorithm had a sensitivity of 72.6% (95% CI: 69.6% - 75.7%) and specificity of 81.9% (95% CI: 79.3% - 84.5%). Incorporating antimicrobial information increased sensitivity to a range of 80.8% (95% CI: 78.1% - 83.5%) to 99.5% (95% CI: 99.0% -100.0%), but resulted in a marked decline in specificity, ranging from 12.2% (10.0% -14.4%) to 39.6% (95% CI: 36.3% - 42.9%). Removing infection-related ICD codes increased sensitivity to 83.4% (95% CI: 80.8% - 85.9%) but further reduced specificity to 26.9% (95% CI: 23.9% - 29.9%). Combining both modifications raised sensitivity to a range of 88.8% (95% CI: 86.6% - 90.9%) to 99.7% (95% CI: 99.4% -100.0%), while specificity dropped to 4.8% (95% CI: 3.3% - 6.2%) to 13.6% (95%CI: 11.2% -15.9%). Conclusion: The Jolley algorithm demonstrated reasonable sensitivity and specificity in identifying sepsis patients among TOH ICU patients using administrative data. While adding antimicrobial information or removing infection codes substantially increased sensitivity, these modifications led to a marked decline in specificity. These findings highlight both the promise and limitations of using administrative data for sepsis surveillance and underscore the need for careful algorithm refinement to balance accuracy and utility.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".