Comparison of the English and French versions of the CASPer® Test in a bilingual population
Bibliographic record
Abstract
This article was migrated. The article was marked as recommended. Objective The University of Ottawa MD program has two different streams to which candidates may apply: a francophone stream and an anglophone stream. As the admissions office receives applications in both French and English, they are required to ensure that the tools used to assess candidates are psychometrically equivalent across both streams. CASPer is a standardized test they recently adopted to assess the non-cognitive competencies of applicants and is offered in both English and French. The objective of this study is to compare the psychometric properties of the English and French versions of CASPer. Methods We collected data from all CASPer test-takers across three cohorts (n = 12,463; entry 2016, entry 2017, entry 2018). We first compared the difficulty of the test between the French and English version using proxy indicators (i.e., time to completion, typing speed). We then compared the psychometric properties of the two versions based on their internal-consistency reliability and applicant acceptability. Results There were some indications that the French version may be slightly more difficult than the English version of the CASPer test. However, it is unclear whether this difficulty is due to the difficulty of the individual test items or to differences in the characteristics of the cohort. Nevertheless, a comparison of the psychometric indicators suggests that both French and English versions of CASPer are psychometrically sound and equivalent. Conclusion Although CASPer scores cannot be directly compared between the English and French versions, the psychometric properties of the assessment were retained across the two versions. These results provide preliminary evidence that the psychometric strengths of the English version of CASPer likely extend to the French version of the assessment.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.004 | 0.013 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.001 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.000 | 0.001 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".