User Reputation Indicators to Reduce the Spread of False News on Social Media
Bibliographic record
Abstract
Social media provide invaluable opportunities for people to curate desirable social identities for themselves. Positive social identities afford people with numerous social benefits, from belonging to social status, and thus many people are motivated to manage the impressions they make on others however possible. This powerful desire to keep one's social identity in good standing could incentivize paying attention to information accuracy, being skeptical of misinformation, and inducing decision uncertainty to combat engagement with online misinformation and reduce its spread. This dissertation evaluated the effectiveness of the presence (vs. absence) of a potential user reputation system intervention that reflects the credibility of news that participants interact with. It was hypothesized that threat posed by receiving an undesirable reputation would incentivize more active processing of information, resulting in more efforts to verify news by clicking links and reduced engagement with false news. To evaluate these differences in behaviour, this dissertation concurrently developed and solicited participant feedback on the Mock Social Media Website Tool (see https://docs.studysocial.media/docs/intro/), research software that simulates natural social media environments and collects detailed behavioural data on how participants interact with them. In Study 1 (n=181), a prototype of the software was used to test the effectiveness of a user reputation system that gave participants individually computed scores that were calculated based on the proportion of true (vs. false) fact-checked news they engaged with. Study 2 (n=56) solicited feedback from social media researchers on how to improve the prototype, and informed the development of the current, public-release version of the Mock Social Media Website Tool. Study 3 (n=486) was used to collect initial validation metrics for the public release version. Study 4 (n=283) evaluated the effectiveness of a streamlined version of the user reputation system using the public release version. Results found that the user reputation systems did not increase attempts to verify news or reduce engagement with misinformation, likely underscored by low engagement overall. However, the Mock Social Media Website Tool emerged as a highly customizable and realistic way of simulating social media platforms that shows promise for enabling novel behavioural social media research.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.005 | 0.042 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.000 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.002 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.008 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".