The Cognitive Validity of the Script Concordance Test: A Processing Time Study
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
BACKGROUND: According to the theory on which the Script Concordance Test (SCT) is based, scripts contain expectations on features that are associated with each illness and about the range of values that are typical, atypical, or incompatible. PURPOSE: To document the construct validity of the SCT, we investigated the theory prediction that once a script is activated, new incoming information (e.g., additional clinical features) is processed faster if it is typical for that script than if it is atypical. If it is incompatible, processing time falls in between. METHODS: We presented 2 groups of participants (30 fourth-year medical students and 30 full-time geriatricians) with 64 clinical vignettes (divided over 5 types of prevalent clinical presentations in geriatrics), each accompanied by a diagnostic hypothesis aimed to instantiate an appropriate script. Next, we presented a new finding, which could be typical, atypical, or incompatible given the hypothesis. Participants had to decide as quickly and accurately as possible whether the new finding increased, decreased, of did not affect the likelihood of the diagnostic hypothesis. We administered the test on a computer. The dependent variable was processing time. We analyzed data with a repeated measure 2 x 3 analysis of variance. RESULTS: Typical information was processed faster than atypical and incompatible information (M = 10.6 sec vs. 19.2 sec and 16.4 sec, respectively; p lt; .001 for both). Incompatible information was processed faster than atypical information (16.4 sec vs. 19.2; p < .001). There was no significant difference between the groups of geriatricians and students. CONCLUSION: It is possible to predict what kind of information will be processed faster depending of the typicality and compatibility of clinical data for given hypotheses. Results support SCT construct validity.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.005 | 0.209 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.002 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it