Quality Assessment of Health Information on Social Media During a Public Health Crisis: Infodemiology Study
Bibliographic record
Abstract
Background: The quality of health information on social media is a major concern, especially during the early stages of public health crises. While the quality of the results of the popular search engines related to particular diseases has been analyzed in the literature, the quality of health-related information on social media, such as X (formerly Twitter), during the early stages of a public health crisis has not been addressed. Objective: This study aims to evaluate the quality of health-related information on social media during the early stages of a public health crisis. Methods: A cross-sectional analysis was conducted on health-related tweets in the early stages of the most recent public health crisis (the COVID-19 pandemic). The study analyzed the top 100 websites that were most frequently retweeted in the early stages of the crisis, categorizing them by content type, website affiliation, and exclusivity. Quality and reliability were assessed using the DISCERN and JAMA (Journal of the American Medical Association) benchmarks. Results: Our analyses showed that 95% (95/100) of the websites met only 2 of the 4 JAMA quality criteria. DISCERN scores revealed that 81% (81/100) of the websites were evaluated as low scores, and only 11% (11/100) of the websites were evaluated as high scores. The analysis revealed significant disparities in the quality and reliability of health information across different website affiliations, content types, and exclusivity. Conclusions: This study highlights a significant issue with the quality, reliability, and transparency of online health-related information during a public health challenge. The extensive shortcomings observed across frequently shared websites on Twitter highlight the critical need for continuous evaluation and improvement of online health content during the early stages of future health crises. Without consistent oversight and improvement, we risk repeating the same shortcomings in future, potentially more challenging situations.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.006 | 0.002 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.002 | 0.000 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".