ARIA index to assess the quality of mobile health apps: A usability and reliability study
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
PurposeThe ubiquity of smart devices has precipitated the growth of mobile health applications (mHealth apps), which aim to help users manage health conditions.Persons living with chronic health conditions, especially older adults, can benefit from the use of mHealth apps given their diverse uses.However, challenges persist with regulation of this industry.Thus, the quality of mHealth apps are often unknown.As no tools existed to assess the quality of mHealth apps for persons living with a health condition, care partners, or healthcare practitioners, we developed the Alberta Rating Index of Apps (ARIA), with a paper-based version validated in a previous study (Azad-Khaneghah, Roduta Roberts, & Liu, 2022).ARIA consists of 19 items rated on a Likert scale that ask users to reflect on the usability, trustworthiness, appropriateness, security, and satisfaction of the mHealth app.For example, one item reads "It would be easy for the user to understand the information provided by the app."Subsequently, we developed a web-based version of the index.A web-based version may act to enhance accessibility, broaden the user audience, and improve functionality through computerized features, such as customized font size and automatic saving of progress.Older adults may benefit from these digitized features given the accessibility features available online to accommodate for a range of age-related impairments.Method In this study, the usability and reliability of a web-based version of ARIA was assessed.Eighty-four (84) participants trialed two mHealth apps (Calm and Breathe2Relax) and completed the ARIA Online questionnaire based on their experiences.A retest interval was observed before completing a second round of the questionnaire, in addition to the System Usability Scale (SUS).Frequency distribution described demographics, and statistical analyses included factor analysis, and reliability coefficients (internal consistency, inter-rater, and test-retest); descriptive analysis were applied for SUS scores (mean and standard deviation).Results and Discussion 61 persons living with a health condition, 15 care partners, and 8 healthcare practitioners participated in the study.An exploratory factor analysis identified 11 questionnaire items to be appropriate according to the correlation matrix (coefficients >0.70, KMO 0.838, chi-square 1458.00,171 df, p<0.001).Reliability analyses show high internal consistency among ARIA items (Cronbach's = 0.819), agreement among raters (ICC = 0.819), and reliability over time (r = 0.590, p<0.01).The SUS scores indicated that ARIA Online was well-accepted (mean = 83.85,SD = 9.85).Our results suggest that ARIA Online is both usable and reliable when assessing the quality of mHealth apps.An established and usable index for mHealth apps could equip app users to accurately assess the quality of mHealth apps.ARIA online can empower users who are considering adopting an application, ultimately enhancing their health management journey.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.009 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it