A Rule‐Based Framework to Identify Severity of Dementia from Unstructured Electronic Health Record Data
Bibliographic record
Abstract
Abstract Background The severity of Alzheimer’s disease and related dementias (ADRD) is mostly documented in unstructured textual data in electronic health records (EHR). This information is important for clinical decision‐making yet is often “hidden” in free text fields and then not as readily available as information in the structured fields for clinicians to act upon. This study assessed the feasibility and potential bias in using keywords and rules‐based matching for obtaining information about severity of ADRD from EHR. Method We used EHR data from a large academic healthcare system that included patients with a primary discharge diagnosis of ADRD based on ICD‐9/10 codes between 2014 and 2020. The severity of ADRD was determined using clinicians’ notes based on (1) scores from the Mini Mental State Examination and Montreal Cognitive Assessment, and (2) explicit terms for ADRD severity (e.g., “mild dementia”, “advanced AD”). A list of common ADRD symptoms, cognitive test names and diagnosis stage terms was compiled and iteratively refined based on prior literature and clinical expertise. We used the list together with rule‐based pattern matching to identify the context in which the word/phrase was mentioned. The algorithm was developed in python 3.8 using spaCy and pandas library. We assessed the prevalence of the documented ADRD severity and used logistic regression to examine whether the severity varies by patient characteristics. Result A total of 9,115 patients with over 65k providers’ notes were evaluated. Overall, 16.85% (N = 1,536) of patients were documented with mild ADRD, 17.95% (N = 1,636) were documented with moderate or severe ADRD, and 65.20% (N = 5,942) did not have any documentation of the severity of their ADRD. Compared with patients with mild ADRD, those documented with more advanced ADRD were older, more likely to be female, black, and receive their diagnoses in a primary care or in‐hospital setting. Relative to patients with undocumented ADRD severity, those documented with ADRD severity had a similar distribution regarding sex, race, and rural/urban living environment. Conclusion This study demonstrated the value of unstructured EHR data and the feasibility of using pattern matching algorithm in identifying severity of ADRD. Still, differences in the documentation may introduce bias in the algorithm.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.015 | 0.059 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.003 |
| Bibliometrics | 0.011 | 0.005 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.005 | 0.003 |
| Open science | 0.004 | 0.003 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.002 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".