A highly scalable deep learning language model for common risks prediction among psychiatric inpatients
Bibliographic record
Abstract
BACKGROUND: There is a lack of studies exploring the performance of Transformers-based language models in common risks assessment among psychiatric inpatients. We aim to develop a scalable risk assessment model using multidimensional textualized data and test the stability, robustness, and benefit of this approach. METHODS: In this real-world cohort study, a deep learning language model was developed and validated using first hospitalized cases diagnosed with schizophrenia, bipolar disorder, and depressive disorder between January 2016 and March 2023 in three hospitals. The algorithm was externally validated on an independent testing cohort comprising 1180 patients. A total of 140 features, including first medical records (FMR), laboratory examinations, medical orders, and psychological scales, were assessed for analysis. The outcomes were short- and long-term impulsivity (STI and LTI), risk of suicide (STSS and LTSS), and need of physical restraint (STPR and LTPR) assessed by qualified nurses or clinicians. Analysis was carried out between August 2024 and June 2024. Models with different architectures and input settings were compared with each other. The area under the receiver operating characteristic curve (AUROC) was used to assess the primary performance of models. The clinical utility was determined by the net benefit under Youden's threshold. RESULTS: Of 7451 patients included in this study, 2982 (47.6%) were male, and the median (interquartile range) age was 42 (28-57) years. The overall incidence of outcomes was 635 (8.5%), 728 (10.5%), 659 (8.8%), 803 (10.8%), 588 (7.9%), and 728 (9.8%) for STPR, LTPR, STSS, LTSS, STI, and LTI, respectively. The multitask semi-structured Transformers-based language (SSTL) model showed more promising AUROCs (STPR: 0.915; LTPR: 0.844; STSS: 0.867; LTSS: 0.879; STI: 0.899; LTI: 0.894) in the prediction of these outcomes than single-tasked or multimodal language models and traditional structured data models. Combining FMR with other data from electronic health records led to significant improvements in the performance and clinical utility of SSTL models based on demographic, diagnosis, laboratory tests, treatment, and psychological scales. CONCLUSIONS: The SSTL model shows potential advantages in prognostic evaluation. FMR is a strong predictor for common risks prediction and may benefit other tasks in psychiatry with minimum requirements for data and data processing.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".