From First Use to Final Commit: Studying the Evolution of Multi-CI Service Adoption (Replication Package)
Bibliographic record
Abstract
This repository contains the replication package for the paper "From First Use to Final Commit: Studying the Evolution of Multi-CI Service Adoption," accepted at the 41st IEEE International Conference on Software Maintenance and Evolution 2025 (ICSME'25). The package provides all resources needed to reproduce the analyses and results presented in the paper. How to Cite If you use this package, please cite our paper: Nitika Chopra and Taher A. Ghaleb. "From First Use to Final Commit: Studying the Evolution of Multi-CI Service Adoption." In Proceedings of the 41st IEEE International Conference on Software Maintenance and Evolution (ICSME), 2025. @inproceedings{chopra2025multici, title={From First Use to Final Commit: Studying the Evolution of Multi-CI Service Adoption}, author={Chopra, Nitika and Ghaleb, Taher A.}, booktitle={Proceedings of the 41st IEEE International Conference on Software Maintenance and Evolution (ICSME)}, year={2025}, organization={IEEE} } Package Structure project-root/ ├── data/ # Data used in the study │ ├── java_ci_services_existence_check.csv │ ├── java_ci_services_yml_stats.csv │ ├── java_commits_per_ci_files.csv │ ├── java_contributors.csv │ ├── java_repo_commit_counts.csv │ └── java_repo_details.csv ├── scripts/ # Analysis scripts │ └── script.ipynb ├── results/ # Generated visualizations ├── requirements.txt # Python dependencies └── README.md # Project documentation Installation This package was developed and tested with Python 3.13.2. Clone the repository and install the required dependencies: pip install -r requirements.txt Usage To run the analysis, install Jupyter if it's not already installed. You can do so via: pip install notebook Run Analysis: Open the Jupyter notebook and execute the analysis: jupyter notebook scripts/script.ipynb The notebook performs many analyses related to CI service adoption and usage evolution, particularly related to the reported findings of the two research questions our study addresses: RQ1: How do CI services differ from each other in terms of adoption, usage evolution, configuration complexity, and maintenance activity? RQ2: What are the patterns of CI service co-adoption and switching among GitHub projects? Save Results: Generated results and figures will be displayed in the notebook and are also saved in the results/ folder. Data The data/ folder contains all data used in the study. See the paper and script comments for details on each file. Results The results/ folder contains: Analysis visualizations as reported in the paper (and more plots that were not included in the paper due to space restrictions). License Code in this repository is licensed under the MIT License. See the LICENSE file. Data files in this repository are licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) license unless otherwise noted. You are free to share and adapt the data with appropriate credit.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.012 | 0.082 |
| Meta-epidemiology (narrow) | 0.001 | 0.002 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.004 | 0.006 |
| Science and technology studies | 0.002 | 0.001 |
| Scholarly communication | 0.005 | 0.006 |
| Open science | 0.003 | 0.005 |
| Research integrity | 0.001 | 0.003 |
| Insufficient payload (model declined to judge) | 0.133 | 0.092 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".