194 Machine learning for early prediction of sepsis from electronic health records: a preliminary validation study at GOSH

Background Electronic Health Records (EHRs) are being leveraged to build Machine Learning (ML) models to help tackle the growing burden of neonatal sepsis. Regrettably, obstacles in external validation hinder their seamless integration into clinical care. We seek to explore the transferability of ML methods in diverse specialist clinical settings. Specifically, we aim to test the robustness to external validation of both the models and the methodologies published by Masino et al. (2019), using the original data from the Children’s Hospital of Philadelphia (CHOP) and comparable data from Great Ormond Street Hospital (GOSH).Methods We extracted ICU EHR data and employed transformation techniques to derive clinical features, ensuring dataset comparability. Following the computational framework established by Masino et al. (2019), we developed ML classifiers aimed at early sepsis prediction. We evaluated the influence of EHR data differences by analysing feature compatibility, data heterogeneity, and availability. Learning curve analysis was employed to explore overfitting within both datasets.Results We obtained the following preliminary results; the tested models reproduced the strong discriminatory power seen in the CHOP training dataset (AUC: 0.85 for Logistic Regression) but demonstrated a decline in predictive performance during external validation on GOSH data (AUC: 0.47 for Logistic Regression). Retraining on GOSH data improved performance (AUC: 0.71 for Logistic Regression), with statistically significant differences from initial validation results.Conclusion Disparities in EHR data signals between the GOSH and CHOP cohorts, including differences in baseline distributions, data availability, and biases from data imputation, largely explained these results. The classifiers learnt CHOP cohort-specific patterns, complicating deployment at GOSH. To combat overfitting in specialist hospitals with smaller datasets, simpler and more interpretable models may be more effective. We emphasise the importance of validation frameworks to build robust and reliable models capable of consistent performance amidst different data landscapes.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.