Abstract WP182: Machine Learning Models are More Accurate Than Regression-based Models for Predicting Functional Impairment Risk in Acute Ischemic Stroke
Bibliographic record
Abstract
Background and Purpose: The burden of stroke-related functional impairment remains high among stroke survivors. Clinical prediction models are commonly used to estimate patient functional impairment risk. However, these models have been principally developed based on regression models, which are sensitive to multicollinearity. This study investigates whether there is any advantage in using machine learning models to develop stroke-related functional impairment risk prediction tools. Methods: Using data from a multi-center hospital-based cohort study (n = 614). Modified Rankin Scale (mRS) score was used to assess 90-day functional impairment status. The accuracy of machine learning models was used to predict the risk of patient-specific risk of 90-day functional impairment. Area under the receiver operating characteristic curve (AUC) was used to assess the predictive accuracy of these models via internal cross-validation and external validation in the ESCAPE randomized controlled trial data. Results: Of the 614 patients included in the analyses, 348(56.7%) had some form of functional impairment (i.e., mRS > 1), 313 (50.9%) were males, while the median and interquartile range (IQR) of age and baseline NIHSS scores were 72 years (IQR = 63-80) and 12 (IQR = 6-19), respectively. Internal cross-validation shows that the AUC for regression models were 68.3% (95%CI = [63.9% - 76.5%]) and 70.1% (95%CI = [63.5% - 76.1%]) while the AUC for machine learning models ranged between 62.7% to 68.8%. But when these models were externally validated in the ESCAPE data, the AUC for regression models were 39.6% (95%CI = [36.1% - 47.5%]) and 35.8% (95%CI = [30.4% - 41.5%]) while the AUC for machine learning models ranged between 61.6% (95%CI = [58.2% - 67.3%]) and 66.7% (95%CI = [61.3% - 72.3%]). Conclusions: This study shows that while there were negligible differences between risk prediction models based on machine learning and regression-based models when internally validated, the former are more accurate than the latter in predicting stroke-related functional impairment in externally validated data. Future research will use Monte Carlo methods to develop recommendations for selecting machine learning models under a variety of data characteristics.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.022 | 0.078 |
| Meta-epidemiology (narrow) | 0.002 | 0.000 |
| Meta-epidemiology (broad) | 0.002 | 0.002 |
| Bibliometrics | 0.002 | 0.002 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.004 | 0.003 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.009 | 0.003 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".