Usability of a Co-designed eHealth Prototype for Caregivers: Combination Study of Three Frameworks
Bibliographic record
Abstract
BACKGROUND: Co-design (or the participation of users) has shown great potential in the eHealth domain, demonstrating positive results. Nevertheless, the co-design approach cannot guarantee the usability of the system designed, and usability assessment is a complex analysis to perform, as evaluation criteria will differ depending on the usability framework (or set of criteria) used. ISO (International Organization for Standardization) on usability (ISO 9241-210), Nielsen heuristic, and Garrett element of user experience inform different yet complementary aspects of usability. OBJECTIVE: This study aims to assess the usability and user experience of a co-design prototype by combining 3 complementary frameworks. METHODS: To help caregivers provide care for functionally impaired older people, an eHealth tool was co-designed with caregivers, health and social service professionals, and community workers assisting caregivers. The prototype was a website that aims to support the help-seeking process for caregivers (finding resources) and allow service providers to advertise their services (offering resources). We chose an exploratory study method to assess usability in terms of each objective. The first step was to assess users' first impressions of the website. The second was a task scenario with a think-aloud protocol. The final step was a semistructured interview. All steps were performed individually (with a moderator) in a single session. The data were analyzed using 3 frameworks. RESULTS: A total of 10 participants were recruited, 5 for each objective of the website. We were able to identify several usability problems, most of which were located in the information design and interface design dimensions (Garrett framework). Problems in both dimensions were mainly coded as effectiveness and efficiency (ISO framework) and error prevention and match between the systemand the real world (Nielsen heuristic). CONCLUSIONS: Our study provided a novel contribution about usability analysis by combining the 3 different models to classify the problems found. This combination provided a holistic understanding of the usability improvements needed. It can also be used to analyze other eHealth products. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.2196/11634.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".