MétaCan
Menu
← Back to cohort
Record W4383313452 · doi:10.1101/2023.06.29.23292059

Geographical validation of the Smart Triage Model by age group

2023· preprint· en· W4383313452 on OpenAlexaff
Cherri Zhang, Matthew O. Wiens, Dustin Dunsmuir, Yashodani Pillay, Charly Huxford, David Kimutai, Emmanuel Tenywa, Mary Ouma, Joyce Kigo, Stephen Kamau, Mary Chege, Nathan Kenya‐Mugisha, Savio Mwaka, Guy A. Dumont, Niranjan Kissoon, Samuel Akech, J. Mark Ansermino

Bibliographic record

VenuemedRxiv · 2023
Typepreprint
Languageen
FieldMedicine
TopicEmergency and Acute Care Studies
Canadian institutionsBC Children's HospitalUniversity of British Columbia
Fundersnot available
KeywordsTriageGeneralizability theoryMedicineReceiver operating characteristicPediatricsProspective cohort studyCohortInternal medicineEmergency medicineStatistics

Abstract

fetched live from OpenAlex

Abstract Age is an important risk factor among critically ill children with neonates being the most vulnerable. Clinical prediction models need to account for age differences and must be externally validated and updated, if necessary, to enhance reliability, reproducibility, and generalizability. We externally validated the Smart Triage model using a combined prospective baseline cohort from three hospitals in Uganda and two in Kenya using admission, mortality, and readmission. We evaluated model discrimination using area under the receiver-operator curve (AUROC) and visualized calibration plots. In addition, we performed subsetting analysis based on age groups (< 30 days, ≤ 2 months, ≤ 6 months, and < 5 years). We revised the model for neonates (< 1 month) by re-estimating the intercept and coefficients and selected new thresholds to maximize sensitivity and specificity. 11595 participants under the age of five (under-5) were included in the analysis. The proportion with an outcome ranged from 8.9% in all children under-5 (including neonates) to 26% in the neonatal subset alone. The model achieved good discrimination for children under-5 with AUROC of 0.81 (95% CI: 0.79-0.82) but poor discrimination for neonates with AUROC of 0.62 (95% CI: 0.55-0.70). Sensitivity at the low-risk thresholds (CI) were 0.85 (0.83-0.87) and 0.68 (0.58-0.76) for children under-5 and neonates, respectively. Specificity at the high-risk thresholds were 0.93 (0.93-0.94) and 0.96 (0.94-0.98) for children under-5 and neonates, respectively. After model revision for neonates, we achieved an AUROC of 0.83 (0.79-0.87) with 13% and 41% as the low- and high-risk thresholds, respectively. The Smart Triage model showed good discrimination for children under-5. However, a revised model is recommended for neonates due to their uniqueness in disease susceptibly, host response, and underlying physiological reserve. External validation of the neonatal model and additional external validation of the under-5 model in different contexts is required. Author summary Clinical prediction model has become evermore popular in various medical fields as it can improve clinical decision-making by providing personalized risk estimate for patients. It is a statistical technique that incorporates patient-specific factors to personalize treatment and optimize health resources allocation. Clinical prediction models need to be validated in a different setting and population, and updated accordingly to ensure accuracy and relevance in clinical settings. We aim to evaluate one such model currently being implemented at the outpatient pediatric department at multiple hospitals in Uganda and Kenya. This model has been incorporated into a digital platform that is used to quickly identify critically ill children at triage. After validating the model against different age groups, we found the current model is not well suited for neonates and thus attempted to update the model. Our study provides new insight into clinical variables that impact neonatal outcome and we hope to improve neonatal morality for low-resource settings.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.

How this classification was reachedexpand

Full frame machine prediction

Teacher imitation

Not calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.

metaresearch head score (Codex)0.018
metaresearch head score (Gemma)0.043
Version: metacan-v3-hybrid-931329e0061cValidation status: machine_predicted_unvalidated
Candidate categoriesnone
Consensus categoriesnone
DomainCandidate signal: none · Consensus signal: none
Study designCandidate signal: Observational · Consensus signal: Observational
GenreCandidate signal: Empirical · Consensus signal: Empirical
Teacher disagreement score0.027
Threshold uncertainty score0.096

Distilled classifier scores by category (both heads)

CategoryCodexGemma
Metaresearch0.0180.043
Meta-epidemiology (narrow)0.0010.000
Meta-epidemiology (broad)0.0010.002
Bibliometrics0.0010.001
Science and technology studies0.0010.001
Scholarly communication0.0010.001
Open science0.0020.002
Research integrity0.0010.002
Insufficient payload (model declined to judge)0.0030.001

Machine scores (provisional)

The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.

Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.

Opus teacher head0.048
GPT teacher head0.304
Teacher spread0.256 · how far apart the two teachers sit on this one work
Validation statusscore_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it

Classification

machine, unvalidated

Machine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.

The models applied no category: nothing in the taxonomy fit this work.
Study designObservational
Domainnot available
GenreEmpirical

How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".

Quick stats

Citations0
Published2023
Admission routes1
Has abstractyes

Explore more

Same venuemedRxiv→Same topicEmergency and Acute Care Studies→French-language works237,207→