A Rule-Based Conversational Agent for Mental Health and Well-Being in Young People: Formative Case Series During the Rise of Generative AI
Notice bibliographique
Résumé
BACKGROUND: There is a shortage of services available to address the growing demand for mental health support in Australia and worldwide. Digital interventions, including conversational agents, can overcome barriers to accessing mental health support. The recent advances in large language models have led to an improvement in the perceived human-like naturalness of chatbot conversations, but there is little research on the experience of chatbots to support mental health. Manage your life online (MYLO) is a rule-based chatbot that was co-designed with young people and uses questions to help users explore their problems. In a case series conducted before the release of ChatGPT (OpenAI), users rated a new smartphone interface for MYLO as acceptable, and there was a large effect size for reduction in problem-related distress. OBJECTIVE: This study aimed to evaluate an improved version of MYLO and compare the user experience of MYLO in this case series to the previous version that was completed in November 2022. METHODS: We replicated and extended the previous 2-week case-series, conducted from September to November 2022, by testing 4-week usage of MYLO with a larger sample between October and December 2023. We recruited 24 young people living in Western Australia who self-described as having a lived experience of anxiety or depression. Participants had access to and used MYLO over a 4-week period while completing online weekly surveys that included a range of health and psychological questionnaires. After the 4-week testing phase, participants were invited to provide feedback on their experience of using MYLO through an interview or focus group discussion. RESULTS: In total, 13 of the 24 participants were retained throughout the study and took part in interviews. On average, participants had around 4 conversations with MYLO. They experienced both benefits and limitations of these conversations. They spoke about their recent experiences with ChatGPT (released in November 2022 after the previous case-series concluded) and other generative artificial intelligence (AI) tools, stating that they had expected MYLO to possess similar functionality, which it did not. Nonetheless, we found moderate to large effect sizes for improvements in problem-related distress (Cohen d=-1.07), anxiety (Cohen d=-0.41), and psychiatric impairment (Cohen d=-0.60) and some evidence of reliable improvement in clinical outcomes. CONCLUSIONS: These findings have implications for mental health chatbots in the age of ChatGPT and highlight a need for researchers to engage with new technologies to improve user experience, while maintaining the necessary safety and ethical standards that can be a significant challenge for generative AI.
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Comment cette classification a été obtenuedéplier
Prédiction machine sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Le volet Gemma est une étiquette directe du modèle pour chaque travail de la base, lue sur la notice réduite au titre. Le volet Codex est un classifieur appris des 10 348 étiquettes directes de Codex et calibré sur les taux pondérés de l'échantillon; les champs sans appui suffisant ne portent aucun appel Codex. Le mode candidate est l'union des deux volets; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont pas des étiquettes humaines.
Scores du classifieur distillé par catégorie (deux têtes)
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,008 | 0,026 |
| Méta-épidémiologie (sens strict) | 0,001 | 0,001 |
| Méta-épidémiologie (sens large) | 0,001 | 0,001 |
| Bibliométrie | 0,001 | 0,001 |
| Études des sciences et des technologies | 0,006 | 0,006 |
| Communication savante | 0,003 | 0,002 |
| Science ouverte | 0,003 | 0,006 |
| Intégrité de la recherche | 0,004 | 0,003 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,002 | 0,000 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découleClassification
machine, non validéePrédiction automatique; un appel candidat d’une seule source (Gemma direct ou Codex distillé), pas un consensus.
Le détail, modèle par modèle et score par score, se trouve en fin de page sous « Comment cette classification a été obtenue ».