Comparative identification of Veterans with mild cognitive impairment or Alzheimer’s disease extracted from clinical notes and administrative data
Bibliographic record
Abstract
Abstract Background Registries that identify people with mild cognitive impairment (MCI) or Alzheimer’s dementia (AD) will be critical in targeting interventions to delay disease progression. Optimal methods to identify people for inclusion in such registries remain uncertain. We now compare approaches that rely on International Classification of Diseases (ICD)‐10 codes to the use of natural language processing (NLP) of clinicians’ notes. Method We used data from the Department of Veterans Affairs (VA) considering all Veterans aged 50 or older from the fiscal year 2019. People for the “ICD” cohorts were identified based on at least one code for either AD or MCI. For the “NLP” cohorts, we used electronic clinician notes and previously validated algorithms that search for keywords ("Alz*” or “Mild Cognitive Impairment”), excluding false positive cases identified through text screening such as “family history.” The number and overlap of people identified by the ICD‐ and NLP‐based approaches were compared. Result Overall, 144,942 Veterans documented with either ICD coding or clinical notes using NLP as probable AD patients were identified; 32,498 (22.4%) people were identified by ICD codes and 141,816 (97.8%) people by clinical notes. Only 20.3% of the Veterans were identified by both clinical notes and ICD; 77.5% were identified by clinical notes alone and only 2.2% by ICD alone. In contrast, for the 134,699 Veterans with MCI, 95,324 (70.8%) people were identified by ICD codes and 79,232 (58.8%) people by clinical notes. Veterans identified by both ICD and clinical notes consisted of 29.6% of the sample, compared to 41.2% by ICD and 29.2% by clinical notes. Conclusion Identification of patients with MCI or AD in electronic clinical notes using NLP is an alternative approach for developing a disease registry to the traditional methodology based on ICD coding in administrative databases. The differences in case identification between NLP‐ and ICD‐based approach may have reflected a clinical practice pattern in making clinical judgments regarding MCI and AD and documenting MCI and AD assessments in clinical notes as opposed to administrative records among the US Veterans.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.010 | 0.053 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.005 | 0.005 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.002 | 0.001 |
| Open science | 0.001 | 0.002 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".