O6.8. THE EXTERNAL VALIDITY OF EARLY PSYCHOSIS RESEARCH: IMPLICATIONS FOR UNDERSTANDING CLINICAL POPULATIONS AND MEASUREMENT-BASED CARE
Bibliographic record
Abstract
Research findings over the past two decades have improved our understanding of schizophrenia and related disorders, including their onset and early course. However, this growing uptake also necessitates attention to the representativeness of research samples. In order to assess the implicit assumption of generalizability, we examined characteristics of research participants from within a large, catchment-based early intervention program for first episode psychosis (EIP) and compared them across a broad range of demographic and clinical factors to EIP patients from the same program who did not participate in research. Within a well-established EIP clinical research infrastructure operating in Montreal, Canada since 2003, patients (ages 14–35) who consented to participate in one of two major services-oriented projects funded by a national health research agency (n = 300) were compared with patients who elected not to participate during the same time periods of recruitment (n = 214). All subjects were drawn from a large, geographically defined catchment area of approximately 300,000 individuals with no competing public or private services in the same region. Data was systematically collected from all patients (with approval from the local research ethics board) based on a desire to engage in ongoing program evaluation. Group representativeness was assessed in dimensions of sociodemographic measures, pathways to care, psychiatric symptoms (positive psychotic, negative psychotic, depression, anxiety), and functioning (global functioning, social and occupational functioning) at entry to the EIP program. Between-group differences were assessed using basic descriptive statistics including t-tests, chi-squared tests, and Mann-Whitney U tests, as appropriate. Patients who participated in research studies were more likely to be diagnosed with affective psychosis than non-participants (35% vs. 21%, respectively; p<0.001), to have proportionally longer median durations of untreated illness (9.11 months vs. 5.67 months, respectively; p<0.003, and to have higher baseline total symptom scores on the Scale for the Assessment of Positive Symptoms (35.58 vs. 30.80, respectively; p<0.001) and the Brief Psychiatric Rating Scale (67.21 vs. 63.67, respectively; p<0.001). Participants also trended towards being more engaged in post-secondary education than non-participants (50.67% vs. 42.99%, respectively; p=0.086) and came from environments of lower rather than higher socio-economic status (71.3% to 63.31%, respectively; p=0.084). Even in longstanding catchment-based EIP settings with a history of community outreach, research samples may be capturing subgroups that are not representative of the presenting clinical population in important yet potentially divergent ways. Given the recent ascendance of population-based approaches in mental health, researchers should be aware of the possibility of similar discrepancies in their own studies and careful to interpret study findings in light of questions regarding generalizability. Finally, these findings suggest strategies for ensuring representativeness in study recruitment, highlight the need to contextualize the reporting of recruited samples with comparators, and have implications for what is prioritized in measurement-based care efforts.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.395 | 0.540 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.003 | 0.004 |
| Bibliometrics | 0.005 | 0.008 |
| Science and technology studies | 0.003 | 0.009 |
| Scholarly communication | 0.005 | 0.006 |
| Open science | 0.003 | 0.006 |
| Research integrity | 0.003 | 0.003 |
| Insufficient payload (model declined to judge) | 0.006 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".