← Back to cohort
Data-driven methods distort optimal cutoffs and accuracy estimates of depression screening tools: a simulation study using individual participant data
Parash Mani Bhandari, Brooke Levis, Dipika Neupane, Scott B. Patten, Ian Shrier, Brett D. Thombs, Andrea Benedetti, Chen He, Danielle B. Rice, Ankur Krishnan, Yin Wu, Marleine Azar, Tatiana Sanchez, Matthew J. Chiovitti, Nazanin Saadat, Kira E. Riehm, Mahrukh Imran, Zelalem Negeri, Jill Boruff, Pim Cuijpers, Simon Gilbody, John P. A. Ioannidis, Lorie A. Kloda, Roy C. Ziegelstein, Liane Comeau, Nicholas Mitchell, Marcello Tonelli, Simone N. Vigod, Franca Aceti, Rubén Alvarado, Cosme Alvarado‐Esquivel, Muideen O. Bakare, Jacqueline Barnes, Amar Bavle, Cheryl Tatano Beck, Carola Bindt, Philip Boyce, Adomas Bunevičius, Tiago Castro e Couto, Linda H. Chaudron, Humberto Corrêa, Felipe Pinheiro de Figueiredo, Valsamma Eapen, Nicolas Favez, Ethel Felice, Michelle Fernandes, Bárbara Figueiredo, Jane Fisher, Lluïsa García-Esteve, Lisa Giardinelli, Nadine Helle, Louise M. Howard, Dina Sami Khalifa, Jane Kohlhoff, Zoltán Kozinszky, Laima Kusminskas, Lorenzo Lelli, Angeliki Leonardou, Michaël Maes, Valentina Meuti, Sandra Nakić Radoš, Purificación Navarro García, Daisuke Nishi, Daniel Okitundu Luwa E‐Andjafono, Susan Pawlby, Chantal Quispel, Emma Robertson‐Blackmore, Tamsen Rochat, Heather Rowe, Bonnie W.M. Siu, Alkistis Skalkidou, Alan Stein, Robert C. Stewart, Kuan‐Pin Su, Inger Sundström Poromaa, Meri Tadinac, S. Darius Tandon, Iva Tendais, Pavaani Thiagayson, Annamária Töreki, A. Torres, Thach Tran, Kylee Trevillion, Katherine Turner, Johann M. Vega‐Dienstmaier, Karen Wynter, Kimberly A. Yonkers
Bibliographic record
VenueJournal of Clinical Epidemiology · 2021
Typearticle
Languageen
FieldMedicine
TopicMaternal Mental Health During Pregnancy and Postpartum
Canadian institutionsWomen's College HospitalUniversity of TorontoUniversity of AlbertaInternational Society for Equity in HealthConcordia UniversityUniversity of CalgaryMcGill UniversityJewish General Hospital
FundersEunice Kennedy Shriver National Institute of Child Health and Human DevelopmentNational Institute of Mental HealthCanadian Institutes of Health ResearchMerck CanadaH. Lundbeck A/SServierNational Center of Neurology and PsychiatryMinistério da SaúdeSchweizerischer Nationalfonds zur Förderung der Wissenschaftlichen ForschungDepartment of Families, Housing, Community Services and Indigenous AffairsJapan Society for the Promotion of ScienceConselho Nacional de Desenvolvimento Científico e TecnológicoUniversity of OxfordWerner Otto StiftungFaculty of Medicine, McGill UniversityPfizerAustralian GovernmentUniversity of ConnecticutChulalongkorn UniversityMedical Research CouncilDepartment of Health and Social CareMcGill University Health CentreProgramme Grants for Applied ResearchMcGill UniversityJewish General HospitalNovo NordiskNational Institute for Health and Care ResearchFundação de Amparo à Pesquisa do Estado de Minas GeraisWellcome TrustUniversity of SouthamptonNational Health and Medical Research CouncilAstraZenecaFundação de Amparo à Pesquisa do Estado de São PauloPatrick and Catherine Weldon Donaghue Medical Research FoundationNational Science Foundation
KeywordsCutoffStatisticsMathematicsPopulationSample size determinationSensitivity (control systems)Youden's J statisticSample (material)Accuracy and precisionMedicineReceiver operating characteristicPhysics
Abstract
OpenAlex records an abstract for this work, but it could not be fetched just now.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
metaresearch head score (Codex)0.107
metaresearch head score (Gemma)0.318
Version: metacan-v3-hybrid-931329e0061cValidation status: machine_predicted_unvalidated
Candidate categoriesMetaresearch
Consensus categoriesnone
DomainCandidate signal: Methods · Consensus signal: none
Study designCandidate signal: Simulation or modeling · Consensus signal: Simulation or modeling
GenreCandidate signal: Empirical · Consensus signal: Empirical
Teacher disagreement score0.893
Threshold uncertainty score0.566
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.107 | 0.318 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.003 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.002 |
| Scholarly communication | 0.002 | 0.002 |
| Open science | 0.003 | 0.002 |
| Research integrity | 0.003 | 0.003 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
Opus teacher head0.814
GPT teacher head0.662
Teacher spread0.152 · how far apart the two teachers sit on this one work
Validation status
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
Study designSimulation or modeling
DomainMethods
GenreEmpirical
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".
Quick stats
Citations20
Published2021
Admission routes2
Has abstractyes