Did we take the same test? Differing accounts of the Ontario Secondary School Literacy Test by first and second language test‐takers
Bibliographic record
Abstract
Within the context of increasing numbers of second language (L2) learners in Canadian schools and expanding standards‐driven testing frameworks, a passing score on the Ontario Secondary School Literacy Test (OSSLT) is a recently imposed secondary school graduation requirement in Ontario. There is evidence, however, that tests designed on the basis of first language (L1) populations may have lower reliability and validity for L2 students. This study elicited accounts of the OSSLT in 33 focus groups of 22 L1 students and 136 L2 students, attending 7 Ontario secondary schools, prior to and immediately after the March 2006 test administration. The results suggest important differences in L1 and L2 accounts of test constructs and suggest a gap between what is valued as literacy on the test and what is valued in classroom literacy practice, raising some concern regarding the test’s consequential validity. By examining how different groups of test‐takers interpret test constructs and the interaction between these interpretations, test design, and accounts of classroom practice, we may better address issues of fidelity in test construct representation (i.e., understand what may constitute construct under‐representation and construct‐irrelevant variance). This study highlights what may make a test more L2‐friendly, i.e. what supports (or impedes) L2 test performance. Although in the washback literature test‐taker accounts of tests have been the least researched, the results of this study suggest that such accounts have the potential to increase test fairness, enhance the validity of inferences drawn from test performance, improve the effectiveness of accommodation strategies, and promote positive washback.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.015 | 0.063 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.001 |
| Bibliometrics | 0.003 | 0.003 |
| Science and technology studies | 0.007 | 0.017 |
| Scholarly communication | 0.006 | 0.004 |
| Open science | 0.002 | 0.004 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".