Goal-Oriented Prioritized Non-Functional Testing with Stakeholders' Priorities
Bibliographic record
Abstract
Non-functional testing is crucial in software product line engineering to ensure high-quality end products. However, software complexity, limited testing resources, and time constraints pose challenges in conducting comprehensive testing. Additionally, non-functional testing can be seen as a multi-criteria decision problem since it involves multiple quality attribute requirements. Stakeholders often prioritize certain quality attributes over others based on their specific needs, goals, interests, and the value they perceive each quality aspect holds for them. Therefore, incorporating these priorities during test planning is essential for guiding decision-making and aligning the testing process with their needs. This paper presents a methodology for addressing non-functional testing in goal-driven software product lines. The methodology employs a semi-automated process for non-functional testing that utilizes the domain goal model as a testing basis to identify areas (features) within the application within the test space of quality attribute of the application and employs a multi-criteria decision method for prioritizing those features based on stakeholders' assigned priority values. Our goal-oriented methodology optimizes testing efforts by focusing on critical areas, aligning priorities with stakeholders' expectations to enhance software quality and reliability. The methodology supports three key testing activities: capturing of testable non-functional requirements and use case definition for early testing, effective testing scopes design with stakeholders' priorities, and creation of standardized test cases using pre-designed templates. A prototype toolkit was implemented to support the proposed methodology.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.002 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".