Gender issues in fundamental physics: Strumia's bibliometric analysis fails to account for key confounders and confuses correlation with causation
Bibliographic record
Abstract
Alessandro Strumia recently published a survey of gender differences in publications and citations in high-energy physics (HEP). In addition to providing full access to the data, code, and methodology, Strumia (2020) systematically describes and accounts for gender differences in HEP citation networks. His analysis points both to ongoing difficulties in attracting women to high-energy physics and an encouraging-though slow-trend in improvement. Unfortunately, however, the time and effort Strumia (2020) devoted to collating and quantifying the data are not matched by a similar rigour in interpreting the results. To support his conclusions, he selectively cites available literature and fails to adequately adjust for a range of confounding factors. For example, his analyses do not consider how unobserved factors - e.g., a tendency to overcite well-known authors-drive a wedge between quality and citations and correlate with author gender. He also fails to take into account many structural and non-structural factors - including, but not limited to, direct discrimination and the expectations women form (and actions they take) in response to it-that undoubtedly lead to gender differences in productivity. We therefore believe that a number of Strumia's conclusions are not supported by his analysis. Indeed, we re-analyse a subsample of solo-authored papers from his data, adjusting for year and journal of publication, authors' research age and their lifetime "fame". Our re-analysis suggests that female-authored papers are actually cited more than male-authored papers. This finding is inconsistent with the "greater male variability" hypothesis Strumia (2020) proposes to explain many of his results.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.100 | 0.254 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.003 | 0.003 |
| Bibliometrics | 0.019 | 0.034 |
| Science and technology studies | 0.004 | 0.012 |
| Scholarly communication | 0.009 | 0.011 |
| Open science | 0.003 | 0.005 |
| Research integrity | 0.003 | 0.004 |
| Insufficient payload (model declined to judge) | 0.005 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".