A Logistic Regression for Differential Item Functioning Primer
Bibliographic record
Abstract
The purpose of this article is to describe a statistical methodology, logistic regression (hereafter referred to as LogR), for differential item fimctioning (hereafter referred to as DIF) for language testing.We wi11 illustrate Logh DM with an English test used fbr a placement purpose with newly entered students in a private university in Japan, With an eye toward our purpose we wi11 first provide a basic overview; including the definition, purpose.and metheds ofDIF.Second, to contextualize LogR DIF methods for language testers, we review some studies using DIF analysis in the field of 1anguage testing, VVe close with a step-by-step guide and demonstration of using LogR DIF statistical methods using a sample data ofthe aforememioned English test.At this point, two are notewonhy, First, we focus on LogR as a statistical method for DIF analyses because as n(rted by Svvaminathan (1994) Lbgh can be considered the most genera1 form of the contingency table and generalized linear modeling approaches to DIF detection (Camilli & Shepard, 1994; Zumbo & Htibley, 2003).Seconq it is these contingency tahle and generalized linear modeling approaches (and particularly the Mantel-Haenszel test) that are among the most widely used Dre statistical methods therefore LogR methods are a good foundation for bui1dmg ones knowledge of DIF methods.Therefore, given the lack of a literature on DIF among language testers in Japan, our goal is to illustrate how the statistical methodology of LogR DIF analyses can be a usefu1 tool in test developrnent and in establishing the validity of the inferences we make from our tcst scores.Readers interested in a more general overview of Dlf methods should see Camilli and Shepard (1994).
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.063 | 0.182 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.003 |
| Bibliometrics | 0.008 | 0.010 |
| Science and technology studies | 0.002 | 0.002 |
| Scholarly communication | 0.003 | 0.006 |
| Open science | 0.004 | 0.006 |
| Research integrity | 0.002 | 0.008 |
| Insufficient payload (model declined to judge) | 0.020 | 0.006 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".