A Clinical Decision‐Making Tool to Identify Red Flags for Remote Cognitive Assessment: An Expert Consensus Study from the Canadian Consortium on Neurodegeneration in Aging
Bibliographic record
Abstract
Abstract Background Remote diagnostic assessment of cognitively impaired individuals offers numerous potential benefits including increased access to care. However, remote cognitive and behavioral assessment also has limitations, and may not be appropriate for certain patients. Currently, evidence‐based guidance on virtual assessment readiness is lacking. Our goal was to develop a clinical decision‐making tool that outlines an approach to determining a patient’s suitability for undergoing remote cognitive and behavioral diagnostic assessment by identifying ‘red flags’ for remote assessment. To address this goal, a multidisciplinary workgroup was convened under the auspices of the Canadian Consortium on Neurodegeneration in Aging (CCNA). This workgroup was composed of experts in remote assessment and included behavioral neurologists, neuropsychiatrists, neuropsychologists, social workers, geriatricians, persons with lived experience and family medicine specialists. Methods The Delphi process is an iterative, systematic, group consensus method, used here to determine the features of the patient, caregiver, clinician and context/situation, or ‘red flags’, indicating that a remote cognitive diagnostic assessment should be avoided. The process consisted of anonymized data collection in three rounds among the multidisciplinary expert workgroup, culminating in two rounds of iterative scoring of potential red flags based on three quality indicators that assessed a potential red flag’s effectiveness, reproducibility, and efficiency. Red flags that received an overall mean score above the pre‐determined consensus threshold on the final round were included in the final clinical decision‐making tool. Result In the first round, 11 respondents, with an average of 12.4 years of clinical experience, generated 67 unique potential red flags. In the second and third rounds, 8 and 9 respondents, respectively, scored the flags on the three quality indicators. Applying consensus criteria yielded 14 red flags that achieved consensus. Conclusion To enhance the translation and implementation of these findings, we developed a clinical decision‐making tool and infographic describing the final set of red flags in collaboration with the CCNA knowledge translation team. This infographic is designed to help clinicians determine a patient’s readiness to undergo remote cognitive assessment. This study directly impacts the clinical care of cognitively impaired individuals by providing clinical decision‐making guidance on a patient’s suitability for remote neurobehavioral assessment.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.284 | 0.260 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.002 |
| Bibliometrics | 0.006 | 0.004 |
| Science and technology studies | 0.008 | 0.003 |
| Scholarly communication | 0.004 | 0.002 |
| Open science | 0.004 | 0.007 |
| Research integrity | 0.003 | 0.002 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".