Social Media Authentication and Users’ Assessments of Health Information: Random Assignment Survey Experiment
Bibliographic record
Abstract
BACKGROUND: In an effort to signal the authenticity of user accounts, social networking sites (SNSs) such as Facebook and X, formerly known as Twitter, use visual heuristics (blue checkmarks) to signify whether accounts are verified. While these verification badges are generally well recognized (and often coveted) by SNS users, relatively little is known about how they affect users' perceptions of accuracy or their likelihood of engaging with web-based information. This is particularly true in the case of information posted by medical experts and health care professionals. OBJECTIVE: This study aims to use an experimental survey design to assess the effect of these verification badges on SNS users' assessments of information accuracy as well as their proclivity to recirculate health information or follow verified medical experts in their social network. METHODS: A survey experiment using random assignment was conducted on a representative sample of 534 adult SNS users in Florida, United States. A total of 2 separate experimental scenarios exposed users to vaccine-related posts from verified medical experts on X. In each case, the original post contained a platform-issued verification badge (treatment group), which was subsequently edited out of the image as an experimental control. For each scenario, respondents were randomly assigned to either the treatment or control group, and responses to 3 follow-up questions were assessed through a series of chi-square analyses and 2 logit regression models. Responses were fielded using a stratified quota sampling approach to ensure representativeness of the state's population based on age, sex, race, ethnicity, and political affiliation. RESULTS: Users' assessments of information accuracy were not significantly impacted by the presence or absence of verification badges, and users exposed to the experimental treatment (verification badge) were not any more likely to repost the message or follow the author. While verification badges did not influence users' assessments or subsequent behaviors, reliance on social media for health-related information and political affiliation were substantial predictors of accuracy assessments in both experimental scenarios. In scenario 1, which included a post addressing COVID-19 vaccine efficacy, users who relied on social media "a great deal" for health information were 2 times more likely to assess the post as accurate (odds ratio 2.033, 95% CI 1.129-3.661; P=.01). In scenario 2, which included a post about measles vaccines, registered Republicans were nearly 6 times less likely to assess the post as accurate (odds ratio 0.171, 95% CI 0.097-0.299; P<.001). CONCLUSIONS: For health professionals and medical experts wishing to leverage social networks to combat misinformation and spread reliable health-related content, account verification appears to offer little by way of added value. On the basis of prior research, other heuristics and communication strategies are likely to yield better results.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.036 | 0.038 |
| Meta-epidemiology (narrow) | 0.002 | 0.002 |
| Meta-epidemiology (broad) | 0.002 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.003 | 0.004 |
| Scholarly communication | 0.002 | 0.003 |
| Open science | 0.003 | 0.002 |
| Research integrity | 0.004 | 0.003 |
| Insufficient payload (model declined to judge) | 0.010 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".