Leveraging User Feedback for Requirements Through Trend and Narrative Analysis
Bibliographic record
Abstract
With the rapid rise of new mediums and surge of user discussion regarding software products, it is increasingly important to consider these user concerns to fulfill user needs, otherwise users may opt for alternatives. Traditional requirements elicitation approaches relied on interviews and surveys with stakeholders to elicit key and important requirements. Despite the advances from recent studies to leverage the “crowd” via increased involvement of crowd based discussions, we still lack empirical structured guidance and approaches to handle feedback from multiple sources and synthesize the themes that emerge from these sources. As providing feedback becomes more accessible for users, managing the volume of feedback is correspondingly challenging. Since development resources are often limited, organizations need to make trade-offs between different user concerns. Gaining more insights on the themes and the trends of these themes from feedback should help organizations conduct these requirement trade-offs. To better explain the emergent trends of user feedback, I use the concept of narratives from economics to explain the phenomenon of what and when users change in their user discussions. Narrative analysis can help explain the causes of feedback trends, which can support organizations determining the priority and validity of various themes from feedback. In this work, I describe my preliminary findings which indicate the profound role that trends and narratives in user feedback have on user perception and concerns. My work has shown that feedback sources, like social media, can provide an avenue to identify requirements. Finally, I outline my plan towards developing a framework to identify trends and narratives, and a tool to support automated identification of requirements in the form of user stories.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.028 | 0.095 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.015 | 0.007 |
| Science and technology studies | 0.002 | 0.002 |
| Scholarly communication | 0.007 | 0.013 |
| Open science | 0.001 | 0.004 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".