Development and Usability Testing of a Chatbot to Promote Mental Health Services Use Among Individuals With Eating Disorders Following Screening
Bibliographic record
Abstract
Background Eating disorders (EDs) are complex mental illnesses with debilitating, pervasive psychological and physiological consequences when left untreated. Unfortunately, patients may face barriers to receiving treatment, such as stereotypes surrounding EDs, denial of illness severity, lack of motivation for treatment, and lack of knowledge about treatment resources. Barriers such as these result in a large treatment gap: only 20% of those with EDs will ever receive treatment. Digital tools like chatbots show potential to disseminate mental health–related interventions to large populations while offering a user-friendly, cost-effective, accessible, and anonymous means of tackling patient concerns. Objective This study developed and evaluated the usability of a chatbot designed for pairing with online ED screening. The tool aimed to promote mental health service utilization by improving motivation for treatment and self-efficacy among individuals with EDs. Methods A chatbot prototype, Alex, was designed using decision trees and theoretically informed components: psychoeducation, motivational interviewing, personalized recommendations, and repeated administration. Usability testing was conducted over 4 iterative cycles, with user feedback informing refinements to the next iteration. Postintervention, participants (N=21) completed the System Usability Scale (SUS), the Usefulness, Satisfaction, and Ease of Use Questionnaire (USE), and a semistructured interview. This process aimed to create an optimized chatbot by the final cycle for use in a randomized trial. Results Interview feedback detailed chatbot aspects participants enjoyed and aspects necessitating improvement. Feedback converged on four themes: user experience, chatbot qualities, chatbot content, and ease of use. Following refinements, users described Alex as humanlike, supportive, and encouraging. Content was perceived as novel and personally relevant. USE scores across domains were generally above average (~5 out of 7), and SUS scores indicated “good” to “excellent” usability across cycles, with the final iteration receiving the highest average SUS score. Conclusions Overall, participants responded well in interactions with Alex, including the initial version. Refinements between cycles further improved user experiences. This study provides preliminary evidence of the feasibility and acceptance of a chatbot designed to promote motivation for and use of services among individuals with EDs. Alex is the first chatbot designed for pairing with an ED or other mental health–related online screen, with the goal of ultimately increasing service utilization. Acknowledgments This research was supported by K08 MH120341 from the National Institute of Mental Health. Availability of Data, Materials, and Code The data will be made available by reasonable request to the corresponding author. Authors’ Contributions EEFC conceptualized and designed the study. OL and BD conducted the investigation process. BD and JS assisted with data curation and conducted formal thematic analyses. JS conducted formal statistical analyses. JS wrote the original manuscript, with contribution from BD. EEFC, CBT, DEW, and SSS designed the data collection instruments, and coordinated and supervised data collection, in addition to reviewing and editing the manuscript with LS, LMF, LAF, and LD. Conflicts of Interest None declared.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".