Author Gender and Editorial Outcomes at Political Behavior
Bibliographic record
Abstract
Political science journals have, for good reason, faced increased scrutiny because of the potential for biases in the editorial process. The representation of women lags behind their distribution in the discipline. Given the importance of publication in hiring, tenure, and promotion, if there are biases in the editorial process, it is vital to the discipline that we determine where in the process these occur and do what is necessary to eliminate them.\nPolitical Behavior uses a double-blind review process. When manuscripts are submitted, the editor determines their fit for the journal in terms of both substance and quality to decide if it is going to be sent out for peer review. At this stage, the editor knows the identity of the author(s). This initial screen results in more than one quarter (30% by August 2017) of all submissions being rejected without external review. Obviously, this is one potential location of any potential bias in the process.\nIf the manuscript is determined to fit the journal and, in the editor’s view, has the potential to be recommended for publication by the reviewers, it is sent out for blind review. At this stage, the reviewers should not know the identity of the author(s). Of course, the review process is less than ideal and there are certainly instances when the reviewers know the identity of the author(s). It is certainly plausible that the reviewer recommendations might also be a source of any bias in the process.\nTo try to empirically evaluate this, an undergraduate research assistant coded the data for 851 submissions to Political Behavior from January 2015 until August 2017. For each of these manuscripts, she coded the gender of the author(s), the rank of the senior author, and the initial decision.1 For manuscripts that were submitted for external review, the research assistant coded the gender of the reviewer and the categorical rating he or she gave. Other editors have coded the methodological approach of the manuscript. For Political Behavior, this is not a meaningful distinction. All but a handful of the submissions are quantitative or formal.\nFollowing the model used by Ansell and Samuels, this report proceeds as follows. The next section reports the descriptive statistics. I then move to a series of statistical tests to determine if there are any statistically significant differences in the outcomes of the review process based on the gender of the authors. Finally, I examine how the gender of the reviewers results in any differences in either the recommendations of the reviewers or the editorial decision. I find no evidence that the gender of the authors influences the outcome of the review process at Political Behavior.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.022 | 0.176 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.001 |
| Bibliometrics | 0.002 | 0.002 |
| Science and technology studies | 0.003 | 0.002 |
| Scholarly communication | 0.007 | 0.004 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.033 | 0.008 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".