P2‐233: ITEM RESPONSE THEORY ANALYSIS OF THE MONTREAL COGNITIVE ASSESSMENT
Bibliographic record
Abstract
The Montreal Cognitive Assessment (MoCA) is a brief screening test that samples a broad range of cognitive domains and is sensitive to Mild Cognitive Impairment (MCI) and Alzheimer's dementia (AD). A prior study examined its psychometric properties using one-parameter Rasch analysis (Koski et al., 2009), concluding that the MoCA is a reliable quantitative estimate of global cognitive ability with items that are sufficiently difficult for use in a geriatric clinical setting. Here, we seek to replicate those findings using a more flexible two-parameter item response theory (IRT) analysis and to investigate whether certain items are differentially difficult or discriminating by sex, education, or diagnosis. 275 participants (age = 72.3 ± 9.5, range = 50.9-93.5; 52.4% Female; 90.4% White; years of education = 16.5 ± 2.7) completed the MoCA as part of a longitudinal research study at the UCLA-ADRC, and were diagnosed as Normal Control (n=99), MCI (n=113), or AD (n=63) by consensus. We estimated a two parameter graded item response model using MoCA item-level data. Differential item functioning (DIF) was examined using the program IRTLRDIF. IRT analysis reveals that most MoCA items have: a) high factor loadings (standardized [N(0,1)] r's>0.7), suggesting that most are correlated well with the latent trait (Fig. A, y-axis), and b) a desirable spread in item difficulty (Fig. A, x-axis). DIF analysis of sex was significant for items Serial 7 and Contour (Figs. B-D): males were more likely to provide correct responses to Serial 7 regardless of cognitive ability, while females were more likely to provide correct responses to Contour except in lower levels of cognitive ability. DIF analysis of years of education was significant for City; individuals with >12 years of education were more likely to provide correct responses regardless of cognitive ability. We found no DIF by diagnosis.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.016 | 0.058 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.004 |
| Bibliometrics | 0.004 | 0.005 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.002 | 0.001 |
| Open science | 0.002 | 0.002 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.013 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".