Infodemics and Vaccine Confidence: Protocol for Social Listening and Insight Generation to Inform Action
Bibliographic record
Abstract
BACKGROUND: In the fall of 2020, the COVID-19 infodemic began to affect public confidence in and demand for COVID-19 vaccines in the United States. While polls indicated what consumers felt regarding COVID-19 vaccines, they did not provide an understanding of why they felt that way or the social and informational influences that factored into vaccine confidence and uptake. It was essential for us to better understand how information ecosystems were affecting the confidence in and demand for COVID-19 vaccines in the United States. OBJECTIVE: The US Centers for Disease Control and Prevention (CDC) established an Insights Unit within the COVID-19 Response's Vaccine Task Force in January 2021 to assist the agency in acting more swiftly to address the questions, concerns, perceptions, and misinformation that appeared to be affecting uptake of COVID-19 vaccines. We established a novel methodology to rapidly detect and report on trends in vaccine confidence and demand to guide communication efforts and improve programmatic quality in near real time. METHODS: We identified and assessed data sources for inclusion through an informal landscape analysis using a snowball method. Selected data sources provided an expansive look at the information ecosystem of the United States regarding COVID-19 vaccines. The CDC's Vaccinate with Confidence framework and the World Health Organization's behavioral and social drivers for vaccine decision-making framework were selected as guiding principles for interpreting generated insights and their impact. We used qualitative thematic analysis methods and a consensus-building approach to identify prevailing and emerging themes, assess their potential threat to vaccine confidence, and propose actions to increase confidence and demand. RESULTS: As of August 2022, we have produced and distributed 34 reports to >950 recipients within the CDC and externally. State and local health departments, nonprofit organizations, professional associations, and congressional committees have referenced and used the reports for learning about COVID-19 vaccine confidence and demand, developing communication strategies, and demonstrating how the CDC monitored and responded to misinformation. A survey of the reports' end users found that nearly 75% (40/53) of respondents found them "very" or "extremely" relevant and 52% (32/61) used the reports to inform communication strategies. In addition, our methodology underwent continuous process improvement to increase the rigor of the research process, the validity of the findings, and the usability of the reports. CONCLUSIONS: This methodology can serve as a diagnostic technique for rapidly identifying opportunities for public health interventions and prevention. As the methodology itself is adaptable, it could be leveraged and scaled for use in a variety of public health settings. Furthermore, it could be considered beyond acute public health crises to support adherence to guidance and recommendations and could be considered within routine monitoring and surveillance systems.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.099 | 0.150 |
| Meta-epidemiology (narrow) | 0.004 | 0.004 |
| Meta-epidemiology (broad) | 0.004 | 0.006 |
| Bibliometrics | 0.008 | 0.010 |
| Science and technology studies | 0.010 | 0.006 |
| Scholarly communication | 0.008 | 0.007 |
| Open science | 0.006 | 0.011 |
| Research integrity | 0.008 | 0.013 |
| Insufficient payload (model declined to judge) | 0.184 | 0.036 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".