Using natural language processing to identify signs and symptoms of dementia and cognitive impairment in primary care electronic medical records (EMR)
Bibliographic record
Abstract
Abstract Background Free‐text fields in electronic medical records (EMRs) are a rich source of information about persons with dementia. The signs and symptoms of dementia (e.g., responsive behaviours, cognitive impairment) can present to primary care providers many years before a formal diagnosis. We used natural language processing (NLP) to develop a list of features (i.e., dementia‐related key words) and compare classification algorithms to identify persons with dementia based on signs and symptoms documented in primary care EMRs. Method We used a validated algorithm based on administrative data to identify 526 persons with incident dementia (known positives) and 44,148 persons without (known negatives) aged 66+ from a primary care EMR database in Ontario, Canada between April 2010 and March 2018. A list of 900+ features associated with dementia was developed using literature review, clinician input and associated word embeddings. We trained a series of classification algorithms (e.g., gradient boosted models, neural networks, lasso and ridge regression) separately in progress notes and consult notes and compared their performance using nested 10‐fold cross validation. Result Persons with dementia were older (mean:80.3 vs. 74.6 years) and more likely to have 5+ chronic conditions (11.6% vs. 7.8%). Persons with dementia had a median of 30.3 features per progress note (IQR:23.8, 40.4) and 54.7 per consult note (IQR:26.6, 83.8) compared to 27.5 (IQR:21.3, 36.5) and 32.1 (IQR:14.0, 55.6) for persons without dementia. Out of eight thematic groups (cognition, social, health system use, function, medication‐dementia, medication, symptoms, other), persons with dementia showed substantially more features related to cognition, social and medication‐dementia in progress and consult notes compared to persons without dementia. Using progress notes, the classification algorithm involving neural networks showed the best performance (Sensitivity:66.2%, Positive Predictive Value [PPV]:81.3%). Using consult notes, the gradient‐boosted classifier performed best (Sensitivity:45.4%, PPV:66.5%). Conclusion We used NLP to discover informative features and develop classification algorithms to identify persons with dementia using free‐text EMR data. This could be used to improve recognition of early signs and symptoms of dementia by primary care providers to provide patients with appropriate interventions, including assessments, imaging and specialist referrals.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.011 | 0.046 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.001 |
| Bibliometrics | 0.005 | 0.003 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.002 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".