76 The predictive validity of MyHEARTSMAP for psychosocial screening in the emergency department
Bibliographic record
Abstract
Abstract Background Mental health concerns in childhood and adolescence are prevalent, affecting nearly one million Canadian youth. In the absence of screening, up to 98% of these concerns can go undiagnosed, leading to significant health, educational, and social consequences. Consequently, the American Academy of Pediatrics recommends the development of screening tools to facilitate early identification and access to treatment. The Emergency Department (ED) represents a unique environment to implement such universal screening, as it is immediately accessible and may be the only point of contact for some vulnerable youth with undiagnosed illness. However, there are few existing instruments which take into account commonly cited barriers such as time constraints, disruption of ED flow, limited resources, and patient privacy. Objectives To facilitate efficient screening with minimal impact on ED flow, our team developed MyHEARTSMAP, an electronic self-administered screening tool. The tool is adapted from HEARTSMAP, a previously validated computerized assessment and management tool used by ED clinicians. MyHEARTSMAP has previously been evaluated for face validity and inter-rater reliability. Here, we measured the sensitivity and specificity of MyHEARTSMAP in identifying mental health concerns in youth. Design/Methods A prospective cohort study was conducted at two tertiary care pediatric EDs. Eligible youth aged 10-17 years presenting for a non-mental health complaint were invited to self-screen using MyHEARTSMAP. An accompanying parent/guardian could also complete an assessment of their child. The sensitivity and specificity was measured as the proportion of screened youth with mental health concerns identified through self-assessment by MyHEARTSMAP compared to assessment performed by a clinician (criteron standard). Results 760 youth and/or parents completed the study intervention. The sensitivity at identifying any psychiatric concerns was comparable between youth and guardian assessments: 92.7% (95%CI: 89.1, 95.4%) and 93.1% (95%CI: 89.5, 95.8%) respectively. The specificity at identifying youth without any psychiatric issues was also comparable between youth and their guardians: 42.2% (95%CI: 37.3, 47.3) and 37.0% (95%CI: 32.2,42.1), respectively. Conclusion MyHEARTSMAP is sensitive for identifying youth with mental health concerns. While it showed only modest specificity, false positives were almost entirely (98%) mild issues identified by youth and deemed to be normal by clinicians. This would not place a burdensome demand on mental health services and could be effectively assessed without specialized psychiatric training. Thus, MyHEARTSMAP may be an effective tool for early identification and management of mental health concerns.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.004 | 0.019 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.001 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.003 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".