Investigating Ensemble Methods to Improve Risk Estimates for Chronic Disease in Canadian Primary Care: A Case Study with Prediction Models for Chronic Kidney Disease in People with Diabetes

Introduction: Clinical prediction models often suffer poor model transportability or subgroup performance resulting from using a single data source. However, many models developed across diverse populations often predict similar outcomes. We examined whether ensemble methods (machine learning that combines several models into a single, often better performing model) can improve model transportability and performance among subgroups. We used clinical prediction models for chronic kidney disease (CKD) in Canadian primary care patients with diabetes as a case study. We quantified the frequency of screening and diagnosis of CKD among patients with diabetes then assessed the performance of existing CKD models and compared with the performance achieved using ensemble methods. Methods: We conducted a retrospective cohort study using the Canadian Primary Care Sentinel Surveillance Network (CPCSSN) electronic medical record database (2014-2019). We included adult patients (18+) with diabetes without existing CKD. We determined the frequency of CKD screening using urine albumin to creatinine ratio (ACR) and estimated glomerular filtration rate (eGFR) testing and identified incident CKD cases over 5 years follow-up. We identified models that predict incident CKD based on two systematic reviews and included models with sufficient predictors in CPCSSN (≤1 unavailable) and eGFR-based CKD definitions. We estimated patients’ risks of incident CKD using each model and combined their unique risk estimates using many strategies (e.g., averaging or mixture-of-experts). For each model, we estimated the discrimination, precision, recall, and calibration. Results: Among 37,604 patients with diabetes, 14.6% were diagnosed with CKD within 5 years. Overall performance of 13 CKD prediction models was mixed: 3 models displayed moderate to strong discrimination whereas others performed poorly. After model updating, calibrations were heterogeneous with most models displaying some miscalibration. Ensemble methods performed well, but no better than the best performing component model. Conclusions: Three models displayed strong performance predicting CKD among CPCSSN patients. These models should be evaluated for their impact in clinical practice. However, our findings suggest ensemble methods may not improve predictive performance. Instead, the best performing component model may be implemented after evaluation in clinical practice—though further research should confirm these findings in additional settings.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.