Session details: Special Issue on Challenges in Software Performance
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Software systems have become increasingly complex. Such complexity makes it extremely challenging to deliver systems that meet their performance requirements. In recent years, there have been several high profile instances of projects that failed due to performance-related issues, e.g., the rollout of the healthcare.gov Web site. This motivates the need for new approaches for software developers to address performance problems. This special issue presents 6 papers that address this theme. A brief summary of these papers follows. Gesvindr and Buhnova study the challenges of building Platform as a Service (PaaS) cloud applications that meet their performance requirements. Using two case studies involving the Microsoft Azure cloud platform, the authors show that the use of a cloud platform in itself does not guarantee good performance and scalability. The authors then identify several best practices for ensuring the performance and scalability of PaaS applications. The work of Heinrich considers the problem of managing\ the performance of software systems composed of thirdparty cloud services. The lack of control over such thirdparty services can cause deviations from performance objectives that can be hard to foresee and manage. The author outlines a model-based framework called iObserve that can help a system adapt and evolve so as to address such problems. Li, O'Brien, and Kihl address the lack of standard methodologies for experimental performance evaluation of software systems. The paper outlines a method for defining domainspecific methodologies for performance evaluation in a way that is consistent across domains. The authors demonstrate the utility of their approach using a cloud services case study. The paper by Grottke et al. considers the problem of handling cyber attacks on networked computer systems andconnected industrial control systems supporting critical infrastructuresuch as energy, water, and gas. Using analytic models, the authors study the performance of various attack mitigation approaches. In particular, the model shows how parameters such as malware scanning rates, network topology, and types of infections impact the effective processing capacity of a system. Rehmann et al. describe the performance monitoring in SAP HANA's continuous integration process. They describe how performance regression tests have been integrated in the delivery process and what services are available to detect and report performance anomalies. Being an extended version of a previous workshop paper, this paper now also presents the reporting of performance test results and anomaly detection. Finally, Nambiar et al. examine issues and experience with software performance, and consider the shortcomings and potential benefits of modeling from the perspective of a potential industrial user. In particular, it considers why modeling is little used, and what conditions it would have to satisfy to fulfill its potential. We sincerely thank all authors for their work and contribution to this special issue. Additionally, we thank our reviewers Davide Arcelli, Danilo Ardagna, Alberto Avritzer, Steffen Becker, Ian Gorton, Andre van Hoorn, Raffaela Mirandola, Juan F. Perez, and Murray Woodside for their timely and constructive reviews as well as editor-in-chief Nidhi Hedge for her support.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.008 | 0.005 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.001 | 0.003 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.002 |
| Open science | 0.002 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.001 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it