Integrating CEN ISO/TS 82304-2 in the Catalan Health App Assessment Framework: Comparative Case Study
Notice bibliographique
Résumé
Background: Health apps are increasingly being used to promote health, manage diseases, and deliver health care services. Still, there is scarce objective information regarding their quality beyond the required Conformité Européenne mark for medical apps, leading to potential risks for users. To address these challenges, several authorities have developed health app assessment frameworks. In 2017, the TIC Salut Social Foundation (FTSS) in Catalonia developed its own health app assessment framework, which has been in use since that year. The publication of CEN ISO/TS 82304-2 (abbreviated as 82304-2)-a Technical Specification for assessing health apps-and the cocreation of the Label2Enable 82304-2 handbook for certified assessment organizations provide a unique opportunity to harmonize app assessments across the European Union. Objective: This study aimed to perform a comparative analysis of the FTSS assessment framework with 82304-2 to explore the integration of 82304-2 in Catalonia. Our broader aim was to provide this methodology for health authorities elsewhere to consider integrating 82304-2 or other evaluation frameworks. Methods: For the comparative analysis, a mixed methods approach was used, combining a qualitative case study with a quantitative analysis of the 2 frameworks. The qualitative evaluation covered rationale for assessment, framework characteristics, governance, workflows, quality aspects, and quality requirements. For the quantitative analysis, all FTSS and 82304-2 requirements were translated into concepts and subconcepts. A scoring system identified matches of the frameworks with these subconcepts, with scores ranging from 0 (no match) to 0.5 (partial match) and 1 (full match). Integration was evaluated considering several scenarios, including adopting the Label2Enable 82304-2 handbook, adopting the 82304-2 requirements, adapting the 82304-2 requirements to local needs, and maintaining the current FTSS framework. Results: The main difference between the frameworks was the app usage-based assessment (FTSS) versus evidence- and app usage-based assessment (82304-2). All 120 FTSS requirements and 74 quality aspect-related 82304-2 requirements were translated into 78 concepts and 97 subconcepts. Overall, 48% (47/97) of the subconcepts were found in both frameworks, 39% (37.5/97) were specific to 82304-2, and 13% (12.5/97) were specific to FTSS. All 82304-2-specific subconcepts and thus all 82304-2 requirements were found to be relevant to FTSS. FTSS decided to integrate (adopt and adapt) all 74 82304-2 requirements. In total, 5 FTSS-specific requirements were included in the Label2Enable 82304-2 handbook, while another 4 rigor-enhancing requirements, 1 scope-expanding requirement, and 1 context-specific requirement would be assessed on top. Conclusions: The comprehensive comparative analysis of the FTSS framework and 82304-2 enabled FTSS decision-making to integrate all 82304-2 quality requirements and adopt the Label2Enable 82304-2 handbook in the future. The many new and all relevant 82304-2 concepts, the rigor of the handbook, and the few remaining FTSS-specific requirements are expected to be indicative of 82304-2's potential to make harmonized, robust health app assessments common in Catalonia and elsewhere. FTSS encourages other authorities to perform a similar evaluation.
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Comment cette classification a été obtenuedéplier
Prédiction machine sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Le volet Gemma est une étiquette directe du modèle pour chaque travail de la base, lue sur la notice réduite au titre. Le volet Codex est un classifieur appris des 10 348 étiquettes directes de Codex et calibré sur les taux pondérés de l'échantillon; les champs sans appui suffisant ne portent aucun appel Codex. Le mode candidate est l'union des deux volets; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont pas des étiquettes humaines.
Scores du classifieur distillé par catégorie (deux têtes)
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,054 | 0,028 |
| Méta-épidémiologie (sens strict) | 0,001 | 0,000 |
| Méta-épidémiologie (sens large) | 0,001 | 0,001 |
| Bibliométrie | 0,004 | 0,004 |
| Études des sciences et des technologies | 0,005 | 0,004 |
| Communication savante | 0,008 | 0,003 |
| Science ouverte | 0,003 | 0,007 |
| Intégrité de la recherche | 0,003 | 0,002 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,003 | 0,000 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découleClassification
machine, non validéePrédiction automatique; un appel candidat d’une seule source (Gemma direct ou Codex distillé), pas un consensus.
Le détail, modèle par modèle et score par score, se trouve en fin de page sous « Comment cette classification a été obtenue ».