Assessing the quality of research examining change in children’s mental health in the context of COVID-19
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
In their policy brief on the impact of COVID-19 on children and youth, the United Nations identified the need for “a rapid accumulation of data on the scale and nature of impacts among children.”1(p14) Although an important goal, this call to action defies how research typically unfolds. Science is a slow, methodical process that requires careful consideration of prior evidence, ethics, measurement, sampling, analysis, and implications, to name a few. Still, we appreciate the call to shift priorities and allocate resources to conduct research about this global event. The stakes are high and information is needed to guide us on how children and youth are faring during this unprecedented time. One problem is that sub-standard studies, often released as non-peer reviewed preprints, are being promoted on social media and in news outlets, and this attention can influence the public’s perception of risk, the credibility of scientists, and policy makers’ decisions related to funding and programming. Some scholars and medical professionals see preprints as a necessity during the pandemic to circumvent the lengthy review process and to arm professionals with the most up-to-date data.2 Others see this growing trend as facilitating the spread of misinformation because, unlike scientists who approach non-peer reviewed research with caution, popular news outlets and the public may take preprints at face value.3,4 Our goal is thus to remind readers of what constitutes good science in the field of child and youth mental health.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.069 | 0.015 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.001 | 0.002 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.002 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it