Analyzing Digital Evidence From a Telemental Health Platform to Assess Complex Psychological Responses to the COVID-19 Pandemic: Content Analysis of Text Messages
Bibliographic record
Abstract
BACKGROUND: The novel COVID-19 disease has negatively impacted mortality, economic conditions, and mental health. These impacts are likely to continue after the COVID-19 pandemic ends. There are no methods for characterizing the mental health burden of the COVID-19 pandemic, and differentiating this burden from that of the prepandemic era. Accurate illness detection methods are critical for facilitating pandemic-related treatment and preventing the worsening of symptoms. OBJECTIVE: We aimed to identify major themes and symptom clusters in the SMS text messages that patients send to therapists. We assessed patients who were seeking treatment for pandemic-related distress on Talkspace, which is a popular telemental health platform. METHODS: We used a machine learning algorithm to identify patients' pandemic-related concerns, based on their SMS text messages in a large, digital mental health service platform (ie, Talkspace). This platform uses natural language processing methods to analyze unstructured therapy transcript data, in parallel with brief clinical assessment methods for analyzing depression and anxiety symptoms. RESULTS: Our results show a significant increase in the incidence of COVID-19-related intake anxiety symptoms (P<.001), but no significant differences in the incidence of intake depression symptoms (P=.79). During our transcript analyses, we identified terms that were related to 24 symptoms outside of those included in the diagnostic criteria for anxiety and depression. CONCLUSIONS: Our findings for Talkspace suggest that people who seek treatment during the pandemic experience more severe intake anxiety than they did before the COVID-19 outbreak. It is important to monitor the symptoms that we identified in this study and the symptoms of anxiety and depression, to fully understand the effects of the COVID-19 pandemic on mental health.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.004 | 0.002 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.001 | 0.004 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".