Bibliographic record
Abstract
KPMG IT Advisory frequently reviews the requirements engineering (RE) process as part of a project review. To assist in these reviews, KPMG uses a project-wide method (PRAM) which contains RE specific questions. However, because this checklist only covers a small set of RE specific aspects and is focused specifically on a waterfall model, reviewers often have to fall back on personal knowledge and experience for a complete review. Based on brainstorm sessions with members of KPMG, we have concluded that this necessity to rely on personal experience often results in time-expensive reviews and uncertainty about the completeness of the assessment. Therefore, the goal of this research is to develop a method that focuses specifically on the assessment of the quality of the RE process. To reach this goal, the following research question is answered: • Which aspects can be used to assess the quality of the RE process for complex software development projects? A research is performed in order to answer this research question. The complete research is divided into two steps: a preliminary research and a main research. The goal of the preliminary research was to develop an initial version of the RE review method. The preliminary research is based on a literature study and on a study of the current approach. The goal of the main research was to test, improve and validate the developed method. Testing of the developed method is done by a case study of two real-life cases. Subsequently, the results of the tests are used to improve the method. Validation of the improved method is done by six members of KPMG, each with a case study of one real-life case. Based on the validation, conclusions are drawn about the performance of the new method and the new method is adapted accordingly. The results show us that it is important that the expectations concerning planning, budget and quality of the solution are realized. Therefore, the chances of not realizing these expectations have to be minimized. This can be done by assessing the performance of the RE process as well as the information in the RE documentation. For the review of the RE process, the following aspects are the most important: the scope of the project has to be clear to all stakeholders, roles and responsibilities has to be clearly communicated, an agreement has to be reached about the cost specification, the solution has to be in alignment with the business model and attention has to be paid to the influence of the solution on the business processes. For the review of the RE documentation, the following aspects are the most important: the planning has to be realistic, de planning should contain the right activities, the cost specification has to be realistic, possible risks should be described, and all requirements has to be specified on a high-level as well as on a detailed-level. The goal of this research is reached. A complete and useful method is developed, with three important benefits in comparison with the current review method: it focuses on assessing the quality of the RE process, it contains aspects that are not present in the current method and it gives examples of aspects where applicable, in order to make the judgment of these aspects easier. The focus of the method is not specifically on iterative processes, but special attention is paid to the aspects that are important in these types of projects.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.012 | 0.072 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.001 |
| Bibliometrics | 0.003 | 0.002 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.003 | 0.002 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".