What Constitutes the Deployment and Runtime Configuration System? An Empirical Study on OpenStack Projects
Bibliographic record
Abstract
Modern software systems are designed to be deployed in different configured environments (e.g., permissions, virtual resources, network connections) and adapted at runtime to different situations (e.g., memory limits, enabling/disabling features, database credentials). Such a configuration during the deployment and runtime of a software system is implemented via a set of configuration files, which together constitute what we refer to as a “configuration system.” Recent research efforts investigated the evolution and maintenance of configuration files. However, they merely focused on a limited part of the configuration system (e.g., specific infrastructure configuration files or Dockerfiles), and their results do not generalize to the whole configuration system. To cope with such a limitation, we aim to better capture and understand what files constitute a configuration system. To do so, we leverage an open card sort technique to qualitatively study 1,756 configuration files from OpenStack, a large and widely studied open source software ecosystem. Our investigation reveals the existence of nine types of configuration files, which cover the creation of the infrastructure on top of which OpenStack will be deployed, along with other types of configuration files used to customize OpenStack after its deployment. These configuration files are interconnected while being used at different deployment stages. For instance, we observe specific configuration files used during the deployment stage to create other configuration files that are used in the runtime stage. We also observe that identifying and classifying these types of files is not straightforward, as five out of the nine types can be written in similar programming languages (e.g., Python and Bash) as regular source code files. We also found that the same file extensions (e.g., Yaml ) can be used for different configuration types, making it difficult to identify and classify configuration files. Thus, we first leverage a machine learning model to identify configuration from non-configuration files, which achieved a median area under the curve (AUC) of 0.91, a median Brier score of 0.12, a median precision of 0.86, and a median recall of 0.83. Thereafter, we leverage a multi-class classification model to classify configuration files based on the nine configuration types. Our multi-class classification model achieved a median weighted AUC of 0.92, a median Brier score of 0.04, a median weighted precision of 0.84, and a median weighted recall of 0.82. Our analysis also shows that with only 100 labeled configuration and non-configuration files, our model reached a median AUC higher than 0.69. Furthermore, our configuration model requires a minimum of 100 configuration files to reach a median weighted AUC higher than 0.75.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".