Interpretation of health news items reported with or without spin: protocol for a prospective meta-analysis of 16 randomised controlled trials
Bibliographic record
Abstract
INTRODUCTION: We aim to compare the interpretation of health news items reported with or without spin. 'Spin' is defined as a misrepresentation of study results, regardless of motive (intentionally or unintentionally) that overemphasises the beneficial effects of the intervention and overstates safety compared with that shown by the results. METHODS AND ANALYSIS: We have planned a series of 16 randomised controlled trials (RCTs) to perform a prospective meta-analysis. We will select a sample of health news items reporting the results of four types of study designs, evaluating the effect of pharmacological treatment and containing the highest amount of spin in the headline and text. News items reporting four types of studies will be included: (1) preclinical studies; (2) phase I/II (non-randomised) trials; (3) RCTs and (4) observational studies. We will rewrite the selected news items and remove the spin. The original news and rewritten news will be appraised by four types of populations: (1) French-speaking patients; (2) French-speaking general public; (3) English-speaking patients and (4) English-speaking general public. Each RCT will explore the interpretation of news items reporting one of the four study designs by each type of population and will include a sample size of 300 participants. The primary outcome will be participants' interpretation of the benefit of treatment after reading the news items: (What do you think is the probability that treatment X would be beneficial to patients? (scale, 0 (very unlikely) to 10 (very likely)).This study will evaluate the impact of spin on the interpretation of health news reporting results of studies by patients and the general public. ETHICS AND DISSEMINATION: This study has obtained ethics approval from the Institutional Review Board of the Institut national de la santé et de la recherche médicale (INSERM) (registration no: IRB00003888). The description of all the steps and the results of this prospective meta-analysis will be available online and will be disseminated as a published article. On the completion of this study, the results will be sent to all participants. PROSPERO REGISTRATION NUMBER: CRD42017058941.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.230 | 0.335 |
| Meta-epidemiology (narrow) | 0.010 | 0.008 |
| Meta-epidemiology (broad) | 0.020 | 0.032 |
| Bibliometrics | 0.010 | 0.012 |
| Science and technology studies | 0.004 | 0.007 |
| Scholarly communication | 0.009 | 0.007 |
| Open science | 0.005 | 0.006 |
| Research integrity | 0.013 | 0.014 |
| Insufficient payload (model declined to judge) | 0.053 | 0.012 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".