Predicting mutation score using source code and test suite metrics
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Mutation testing has traditionally been used to evaluate the effectiveness of test suites\nand provide con dence in the testing process. Mutation testing involves the creation of\nmany versions of a program each with a single syntactic fault. A test suite is evaluated\nagainst these program versions (i.e., mutants) in order to determine the percentage\nof mutants a test suite is able to identify (i.e., mutation score). A major drawback\nof mutation testing is that even a small program may yield thousands of mutants\nand can potentially make the process cost prohibitive. To improve the performance\nand reduce the cost of mutation testing, we proposed a machine learning approach to\npredict mutation score based on a combination of source code and test suite metrics.\nWe conducted an empirical evaluation of our approach to evaluated its effectiveness\nusing eight open source software systems.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.002 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it