Developing a Theory of Change for a Digital Youth Mental Health Service (Moderated Online Social Therapy): Mixed Methods Knowledge Synthesis Study
Bibliographic record
Abstract
BACKGROUND: Common challenges in the youth mental health system include low access, poor uptake, poor adherence, and limited overall effectiveness. Digital technologies offer promise, yet challenges in real-world integration and uptake persist. Moderated Online Social Therapy (MOST) aims to overcome these problems by integrating a comprehensive digital platform into existing youth mental health services. Theory of change (ToC) frameworks can help articulate how and why complex interventions work and what conditions are required for success. OBJECTIVE: The objective of this study is to create a ToC for MOST to explain how it works, why it works, who benefits and how, and what conditions are required for its success. METHODS: We used a multimethod approach to construct a ToC for MOST. The synthesis aimed to assess the real-world impact of MOST, a digital platform designed to enhance face-to-face youth mental health services, and to guide its iterative refinement. Data were gathered from 2 completed and 4 ongoing randomized controlled trials, 11 pilot studies, and over 1000 co-design sessions using MOST. Additionally, published qualitative findings from diverse clinical contexts and a review of related digital mental health literature were included. The study culminated in an updated ToC framework informed by expert feedback. The final ToC was produced in both narrative and table form and captured components common in program logic and ToC frameworks. RESULTS: The MOST ToC captured several assumptions about digital mental health adoption, including factors such as the readiness of young people and service providers to embrace digital platforms. External considerations included high service demand and a potential lack of infrastructure to support integration. Young people and service providers face several challenges and pain points MOST seeks to address, such as limited accessibility, high demand, poor engagement, and a lack of personalized support. Self-determination theory, transdiagnostic psychological treatment approaches, and evidence-based implementation theories and their associated mechanisms are drawn upon to frame the intervention components that make up the platform. Platform usage data are captured and linked to short-, medium-, and long-term intended outcomes, such as reductions in mental health symptoms, improvements in functioning and quality of life, reductions in hospital visits, and reduced overall mental health care costs. CONCLUSIONS: The MOST ToC serves as a strategic framework for refining MOST over time. The creation of the ToC helped guide the development of therapeutic content personalization, user engagement enhancement, and clinician adoption through specialized implementation frameworks. While powerful, the ToC approach has its limitations, such as a lack of standardized methodology and the amount of resourcing required for its development. Nonetheless, it provides an invaluable roadmap for iterative development, evaluation, and scaling of MOST and offers a replicable model for other digital health interventions aiming for targeted, evidence-based impact.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.005 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.002 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".