Explainable Depression Detection Using Large Language Models on Social Media Data
Bibliographic record
Abstract
Due to the rapid growth of user interaction on different social media platforms, publicly available social media data has increased substantially.The sheer amount of data and level of personal information being shared on such platforms has made analyzing textual information to predict mental disorders such as depression a reliable preliminary step when it comes to psychometrics.In this study, we first proposed a system to search for texts that are related to depression symptoms from the Beck's Depression Inventory (BDI) questionnaire, and provide a ranking for further investigation in a second step.Then, in this second step, we address the even more challenging task of automatic depression level detection, using writings and voluntary answers provided by users on Reddit.Several Large Language Models (LLMs) were applied in experiments.Our proposed system based on LLMs can generate both predictions and explanations for each question.By combining two LLMs for different questions, we achieved better performance on three of four metrics compared to the state-of-the-art and remained competitive on the one remaining metric.In addition, our system is explainable on two levels: first, knowing the answers to the BDI questions provides clues about the possible symptoms that could lead to a clinical diagnosis of depression; second, our system can explain the predicted answer for each question.Question Rephrased symptom Q1 how sad the user feels Q2 how discouraged the user is about future Q3 how much the user feels like a failure Q4 how much the user loses pleasure from things Q5 how often the user feels guilty Q6 how much the user feels punished Q7 how much the user feels disappointed about him/herself Q8 how often the user criticizes or blames him/herself Q9 how much the user thinks about killing him/herself Q10 how often the user cries Q11 how much the user feels restless or agitated Q12 how much the user loses interest in things Q13 how difficult the user to make decisions Q14 how much the user feels worthless Q15 how much the user loses energy Q16 how much the user experienced changes in sleeping Q17 how much the user feels irritable Q18 how much the user experienced changes in appetite Q19 how difficult the user to concentrate Q20 how much the user feels tired or fatigued Q21 how much the user loses interest in sexTable A5: Rephrased symptoms on the BDI questionnaire
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.009 |
| Meta-epidemiology (narrow) | 0.002 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.001 | 0.002 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.002 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".