“Impact of COVID-19 on lifestyle habits and mental health symptoms in children with attention-deficit/hyperactivity disorder in Canada”: Can we trust the numbers from this Internet survey?
Bibliographic record
Abstract
The authors of this article (pxab030) address the important issue of the impact of COVID-19 on children with attention-deficit hyperactivity disorder (ADHD) but use an Internet survey nonsampling strategy to do so. They do this by constructing a 113-item questionnaire and making it available online for several weeks until they accumulate 587 surveys. The authors state that their response of “587 exceeded the minimum sample size (N = 384) required to represent the pediatric ADHD population of Canada.” They also claim that they have a representative sample based on ethnicity, income and other factors but then list limitations that suggest this is not the case, i.e., the self-selecting nature of recruitment” and “families who were struggling less were likely more able to complete a 30 to 45 minute survey.” These limitations have consequences. Many parents of children with ADHD also have ADHD and it is likely that those parents who have the time and patience to complete a 113-item online questionnaire are substantially different from those who cannot. Note that this nonrepresentativeness bias cannot be eliminated by sample size adjustment. The authors must go beyond acknowledging these limitations in the discussion, address how those limitations are threats to validity and take reasonable steps to assess whether or not the bias might significantly affect the results. Rather than presenting the data as if it comes from a representative sample, they should warn the reader that the reported results may be substantially different from the population of ADHD families that they are trying to describe. There is a rich literature about non-representativeness and bias inherent in Internet surveys. Investigators those who wish to use such a potentially biased survey method should build into their study methodology some means of quantifying and minimizing those threats to validity. The author has no funding or conflicts of interest to disclose in relation to this letter.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.008 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.001 | 0.004 |
| Science and technology studies | 0.003 | 0.001 |
| Scholarly communication | 0.002 | 0.001 |
| Open science | 0.001 | 0.002 |
| Research integrity | 0.001 | 0.003 |
| Insufficient payload (model declined to judge) | 0.004 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".