Systematic Review of Machine Learning-Based Open-Source SoftwareMaintenance Effort Estimation
Bibliographic record
Abstract
Background: Software maintenance is known as a laborious activity in the software lifecycle and is often considered more expensive than other activities. Open-Source Software (OSS) has gained considerable acceptance in the industry recently, and the Maintenance Effort Estimation (MEE) of such software has emerged as an important research topic. In this context, researchers have conducted a number of open-source software maintenance effort estimation (OMEE) studies based on statistical as well as machine learning techniques for better estimation. Objective: The objective of this study is to perform a systematic literature review (SLR) to analyze and summarize the empirical evidence of O-MEE ML techniques in current research through a set of five Research Questions (RQs) related to several criteria (e.g. data pre-processing tasks, data mining tasks, tuning parameter methods, accuracy criteria and statistical tests, as well as ML techniques reported in the literature that outperformed). Method: We performed a systematic literature review of 36 primary empirical studies published from 2000 to June 2020, selected based on an automated search of six digital databases. Results: The findings show that Bayesian networks, decision tree, support vector machines and instance-based reasoning were the ML techniques most used; few studies opted for ensemble or hybrid techniques. Researchers have paid less attention to O-MEE data pre-processing in terms of feature selection, methods that handle missing values and imbalanced datasets, and tuning parameters of ML techniques. Classification data mining is the task most addressed using different accuracy criteria such as Precision, Recall, and Accuracy, as well as Wilcoxon and Mann-Whitney statistical tests. Conclusion: This SLR identifies a number of gaps in the current research and suggests areas for further investigation. For instance, since OSS includes different data source formats, researchers should pay more attention to data pre-processing and develop new models using ensemble techniques since they have proved to perform better.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.029 | 0.156 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.007 | 0.008 |
| Bibliometrics | 0.034 | 0.020 |
| Science and technology studies | 0.001 | 0.002 |
| Scholarly communication | 0.003 | 0.005 |
| Open science | 0.004 | 0.003 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".