The Online Vaccine Debate: Study of a Visual Analytics System
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Online debates, specifically the ones about public health issues (e.g., vaccines, medications, and nutrition), occur frequently and intensely, and are having an impact on our world. Many public health topics are debated online, one of which is the efficacy and morality of vaccines. When people examine such online debates, they encounter numerous and conflicting sources of information. This information forms the basis upon which people take a position on such debates. This has profound implications for public health. It necessitates a need for public health stakeholders to be able to examine online debates quickly and effectively. They should be able to easily perform sense-making tasks on the vast amount of online information, such as sentiments, online presence, focus, or geographic locations. In this paper, we report the results of a user study of a visual analytic system (VAS), and whether and how this VAS can help with such sense-making tasks. Specifically, we report a usability evaluation of VINCENT (VIsual aNalytiCs systEm for investigating the online vacciNe debaTe), a VAS previously described. To help the reader, we briefly discuss VINCENT’s design in this paper as well. VINCENT integrates webometrics, natural language processing, data visualization, and human-data interaction. In the reported study, we gave users tasks requiring them to make sense of the online vaccine debate. Thirty-four participants were asked to perform these tasks by investigating data from 37 vaccine-focused websites. Half the participants were given access to the system, while the other half were not. Selected study participants from both groups were subsequently asked to be interviewed by the study administrator. Examples of questions and issues discussed with interviewees were: how they went about completing specific tasks, what they meant by some of the feedback they provided, and how they would have performed on the tasks if they had been placed in the other group. Overall, we found that VINCENT was a highly valuable resource for users, helping them make sense of the online vaccine debate much more effectively and faster than those without the system (e.g., users were able to compare websites similarities, identify emotional tone of websites, and locate websites with a specific focus). In this paper, we also identify a few issues that should be taken into consideration when developing VASes for online public health debates.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it