Developing a more equitable language‐based automated assessment of cognition: CognoSpeak
Bibliographic record
Abstract
BACKGROUND: Current common cognitive assessment tools more frequently misdiagnose patients from minority ethnic groups and those who speak English as an additional language. CognoSpeak is a language-based memory assessment tool underpinned by artificial intelligence. Avoiding bias within this system requires exposure to a diverse training population. METHODS: Research champions from within South Asian, Somali, and Chinese community groups were trained to recruit and assess participants using different pen-and-paper (Rowland Universal Dementia Assessment Scale (RUDAS), Multicultural Cognitive Examination (MCE), Montreal Cognitive Assessment (MoCA)) and automated (CognoSpeak) cognitive assessments. RESULTS: To date, 146 participants (52 Somali, 53 South Asian, and 41 Chinese) have been recruited through these community centres, making up over 60% of the total number of BAME participants in CognoSpeak's training cohort (n >1500). Despite all participants being cognitively intact, preliminary data shows that the MoCA miscategorised 39.8% of the participants as cognitively impaired, compared to just 0.8% on the RUDAS and the MCE (p<.001). Initial analysis indicates that Somali participants scored significantly lower than monolingual English speakers across both CognoSpeak's verbal fluency tasks (Semantic: Animals, phonemic: 'P') (p<.001). However, Somali does not have the phoneme "P" meaning the fluency score may be artificially low, reflecting the importance of considering language background in cognitive assessments. We will present the relative accuracy of CognoSpeak's categorisation of participants from these cohorts compared to a White English Speaking monolingual cohort. CONCLUSION: Despite common usage, the MoCA may not be appropriate for minoritised communities and individuals who speak English as an additional language. This collaborative approach helped break down traditional barriers to research participation. We will expand these results to present the accuracy of CognoSpeak across different language backgrounds and ethnicities.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.003 | 0.006 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.002 | 0.001 |
| Open science | 0.001 | 0.002 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".