Exploring How Professionals Within Agile Health Care Informatics Perceive Visualizations of Log File Analyses: Observational Study Followed by a Focus Group Interview
Bibliographic record
Abstract
BACKGROUND: An increasing number of software companies work according to the agile software development method, which is difficult to integrate with user-centered design (UCD) practices. Log file analysis may provide opportunities for integrating UCD practices in the agile process. However, research within health care information technology mostly has a theoretical approach and is often focused on the researcher's interpretation of log file analyses. OBJECTIVE: We aimed to propose a systematic approach to log file analysis in this study and present this to developers to explore how they react and interpret this approach in the context of a real-world health care information system, in an attempt to answer the following question: How may log file analyses contribute to increasing the match between the health care system and its users, within the agile development method, according to agile team members? METHODS: This study comprised 2 phases to answer the research question. In the first phase, log files were collected from a health care information system and subsequently analyzed (summarizing sequential patterns, heat mapping, and clustering). In the second phase, the results of these analyses are presented to agile professionals during a focus group interview. The interpretations of the agile professionals are analyzed by open axial coding. RESULTS: Log file data of 17,924 user sessions and, in total, 176,678 activities were collected. We found that the Patient Timeline is mainly visited, with 23,707 (23,707/176,678; 13.42%) visits in total. The main unique user session occurred in 5.99% (1074/17,924) of all user sessions, and this comprised Insert Measurement Values for Patient and Patient Timeline, followed by the page Patient Settings and, finally, Patient Treatment Plan. In the heat map, we found that users often navigated to the pages Insert Measurement Values and Load Messages Collaborate. Finally, in the cluster analysis, we found 5 clusters, namely, the Information-seeking cluster, the Collaborative cluster, the Mixed cluster, the Administrative cluster, and the Patient-oriented cluster. We found that the interpretations of these results by agile professionals are related to stating hypotheses (n=34), comparing paths (n=31), benchmarking (n=22), and prioritizing (n=17). CONCLUSIONS: We found that analyzing log files provides agile professionals valuable insights into users' behavior. Therefore, we argue that log file analyses should be used within agile development to inform professionals about users' behavior. In this way, further UCD research can be informed by these results, making the methods less labor intensive. Moreover, we argue that these translations to an approach for further UCD research will be carried out by UCD specialists, as they are able to infer which goals the user had when going through these paths when looking at the log data.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.014 | 0.053 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.003 | 0.004 |
| Scholarly communication | 0.003 | 0.004 |
| Open science | 0.001 | 0.004 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.002 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".