Using Decision Trees to Predict the Certification Result of a Build
Bibliographic record
Abstract
Large teams of practitioners (developers, testers, etc.) usually work in parallel on the same code base. A major concern when working in parallel is the introduction of integration bugs in the latest shared code. These latent bugs are likely to slow down the project unless they are discovered as soon as possible. Many companies have adopted daily or weekly processes which build the latest source code and certify it by executing simple manual smoke/sanity tests or extensive automated integration test suites. Other members of a team can then use the certified build to develop new features or to perform additional analysis, such as performance or usability testing. For large projects the certification process may take a few days. This long certification process forces team members to either use outdated or uncertified (possibly buggy) versions of the code. In this paper, we create decision trees to predict ahead of time the certification result of a build. By accurately predicting the outcome of the certification process, members of large software teams can work more effectively in parallel. Members can start using the latest code without waiting for the certification process to be completed. To perform our study, we mine historical information (code changes and certification results) for a large software project which is being developed at the IBM Toronto Labs. Our study shows that using a combination of project attributes (such as the number of modified subsystems in a build and certification results of previous builds), we can correctly predict 69% of the time that a build will fail certification. We can as well correctly predict 95% of the time if a build will pass certification
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.005 | 0.011 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.004 | 0.002 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.002 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.002 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".