TD‐P‐003: Using the Digital Clock Drawing Test and Machine Learning to Improve Accuracy of Cognitive Screening
Bibliographic record
Abstract
The Clock Drawing Test is a rapid and inexpensive screening tool, but its evaluative value is limited by imprecise measurements and low inter-rater reliability. The Digital Clock Drawing Test (dCDT) uses novel software to analyze data from a digitizing ballpoint pen that reports its position with spatial and temporal precision; our software produces precise innovative measurements of both the drawing process and final product. We use machine learning (ML) techniques to select variables and construct prediction models that optimize detection and classification, and show that ML methods improve diagnostic accuracy compared to existing CDT scoring systems. We applied our novel dCDT software to categorize every pen stroke from 3994 neurologically impaired and cognitively healthy (CH) subjects, computing approximately 1000 novel variables on that data. Using ML techniques (e.g., regularized logistic regression) we created models that best classify subjects into four categories: memory impaired, vascular related disorders, Parkinson’s disease, CH. We embodied in code eight manual scoring systems (MSS), then adjusted their parameters to optimize their performance. We measured our ML model performances using 5-fold cross-validation, reporting AUCs averaged over the 5 folds and compared predictive performance to the MSS optimized versions. Classifiers produced by ML methods significantly outperformed MSS: ML methods had an AUC performance ranging from 0.87 to 0.92 (depending on the algorithm and condition being classified), while corresponding AUCs for MSS ranged from 0.63 to 0.74. The MSS AUC ranges were higher than what those manual scoring systems would produce in practice, as optimization ensured their best performance and embodying them in code eliminated all inter-rater reliability error. ML methods combined with precise spatial and temporal analysis of drawing behavior enables the creation of new ways to detect and classify cognitive functions with far greater accuracy, enabling improved diagnostic capabilities with little to no manual effort. Applying ML to theoretical models improves predictive value by using features not typically considered under existing frameworks. Combining new technology and advanced analytics with theoretical neurosciences enables new opportunities for early diagnosis and treatment.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.004 | 0.017 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".