Replication Package - Understanding the Time to First Response In GitHub Pull Requests
Bibliographic record
Abstract
The pull-based development is widely adopted in<br> modern open-source software (OSS) projects, where developers<br> propose changes to the codebase by submitting a pull request<br> (PR). However, due to many reasons, PRs in OSS projects<br> frequently experience delays across their lifespan, including<br> prolonged waiting times for the first response. Such delays<br> may significantly impact the efficiency and productivity of the<br> development process, as well as the retention of new contributors<br> as long-term contributors.<br> In this paper, we conduct an exploratory study on the time-to-<br> first-response for PRs by analyzing 111,094 closed PRs from ten<br> popular OSS projects on GitHub. We find that bots frequently<br> generate the first response in a PR, and significant differences<br> exist in the timing of bot-generated versus human-generated<br> first responses. We then perform an empirical study to examine<br> the characteristics of bot- and human-generated first responses,<br> including their relationship with the PR’s lifetime. Our results<br> suggest that the presence of bots is an important factor contribut-<br> ing to the time-to-first-response in the pull-based development<br> paradigm, and hence should be separately analyzed from human<br> responses. We also report the characteristics of PRs that are more<br> likely to experience long waiting for the first human-generated<br> response. Our findings have practical implications for newcomers<br> to understand the factors contributing to delays in their PRs.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.003 | 0.002 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.002 |
| Science and technology studies | 0.002 | 0.000 |
| Scholarly communication | 0.002 | 0.000 |
| Open science | 0.004 | 0.003 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.003 | 0.112 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; both teacher heads agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".