A Novel Digital Digit-Symbol Substitution Test Measuring Processing Speed in Adults At Risk for Alzheimer Disease: Validation Study
Bibliographic record
Abstract
BACKGROUND: Assessing cognitive constructs affected by Alzheimer disease, such as processing speed (PS), is important to screen for potential disease and allow for early detection. Digital PS assessments have been developed to provide widespread, efficient cognitive testing, but all have been validated only based on the correlation between test scores. Best statistical practices dictate that concurrent validity should be assessed for agreement or equivalence rather than using correlation alone. OBJECTIVE: This study aimed to assess the concurrent validity of a novel digital PS assessment against a gold-standard measure of PS. METHODS: Adults aged 45-75 years (n=191) participated in this study. Participants completed the novel digital digit-symbol substitution test (DDSST) and the Repeatable Battery for the Assessment of Neuropsychological Status coding test (RBANS-C). The correlation between the test scores was determined using a Pearson product-moment correlation, and a difference in mean test scores between tests was checked for using a 2-tailed dependent samples t test. Data were analyzed for agreement between the 2 tests using Bland-Altman limits of agreement and equivalency using a two one-sided t tests (TOST) approach. RESULTS: A significant moderate, positive correlation was found between DDSST and RBANS-C scores (r=.577; P<.001), and no difference in mean scores was detected between the tests (P=.93). Bias was nearly zero (0.04). Scores between the tests were found to display adequate agreement with 90% of score differences falling between -22.66 and 22.75 (90% limits of agreement=-22.91 to 22.99), and the scores were equivalent (P=.049). CONCLUSIONS: Analyses indicate that the DDSST is a valid digital assessment of PS. The DDSST appears to be a suitable option for widespread, immediate, and efficient PS testing. TRIAL REGISTRATION: ClinicalTrials.gov NCT04559789; https://clinicaltrials.gov/ct2/show/NCT04559789.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".