Grouping Digital Health Apps Based on Their Quality and User Ratings Using K-Medoids Clustering: Cross-Sectional Study
Bibliographic record
Abstract
Background: Digital health apps allow for proactive rather than reactive health care and have the potential to take the pressure off health care providers. With over 350,000 digital health apps available on the app stores today, those apps need to be of sufficient quality to be safe to use. Discovering the typology of digital health apps regarding professional and clinical assurance (PCA), user experience (UX), data privacy (DP), and user ratings may help in determining the areas where digital health apps can improve. Objective: This study has two objectives: (1) discover the types (clusters) of digital health apps with regards to their quality (scores) across 3 domains (their PCA, UX, and DP) and user ratings and (2) determine whether the National Institute for Health and Care Excellence (NICE) Evidence Standard Framework's (ESF's) tier, target users of the digital health apps, categories, or features have any association with this typology. Methods: Data were obtained from 1402 digital health app assessments conducted using the Organisation for the Review of Care and Health Apps Baseline Review (OBR), evaluating PCA, UX, and DP. K-medoids clustering identified app typologies, with the optimal number of clusters determined using the elbow method. The Shapiro-Wilk test assessed normality of user ratings and OBR scores. Nonparametric Wilcoxon rank sum tests compared cluster differences in these metrics. Post hoc analysis examined the distribution of NICE ESF tiers, target users, categories, and features across clusters, using Fisher exact test with Bonferroni correction. Effect sizes were calculated using Cohen w. Results: A total of four distinct app clusters emerged: (1) apps with poor user ratings (220/1402, 15.7%), (2) apps with poor PCA and DP scores (252/1402, 18%), (3) apps with poor PCA scores (415/1402, 29.6%), and (4) higher quality apps with high user ratings and OBR scores (515/1402, 36.7%). While some statistically significant associations were found between clusters and NICE ESF tiers (2/3), target users (0/14), categories (4/33), and features (6/19), all had small effect sizes (Cohen w<0.3). The strongest associations were for the "Service Signposting" feature (Cohen w=0.24) and NICE ESF tier B (Cohen w=0.19). Conclusions: The largest cluster comprised high-quality apps with strong user ratings and OBR scores (515/1402, 36.7%). A significant proportion (415/1402, 29.6%) performed poorly in PCA despite performing well in other domains. Notably, user ratings did not consistently align with PCA scores; some apps scored highly with users but poorly in PCA and DP. The 4-cluster typology underscores areas needing improvement, particularly PCA. Findings suggest limited association between the examined app characteristics and quality clusters, indicating a need for further investigation into what factors truly influence app quality.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.010 | 0.030 |
| Meta-epidemiology (narrow) | 0.000 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.003 | 0.003 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.002 | 0.002 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".