Artificial Intelligence–Enabled Social Media Analysis for Pharmacovigilance of COVID-19 Vaccinations in the United Kingdom: Observational Study
Bibliographic record
Abstract
BACKGROUND: The rollout of vaccines for COVID-19 in the United Kingdom started in December 2020. Uptake has been high, and there has been a subsequent reduction in infections, hospitalizations, and deaths among vaccinated individuals. However, vaccine hesitancy remains a concern, in particular relating to adverse effects following immunization (AEFIs). Social media analysis has the potential to inform policy makers about AEFIs being discussed by the public as well as public attitudes toward the national immunization campaign. OBJECTIVE: We sought to assess the frequency and nature of AEFI-related mentions on social media in the United Kingdom and to provide insights on public sentiments toward COVID-19 vaccines. METHODS: We extracted and analyzed over 121,406 relevant Twitter and Facebook posts, from December 8, 2020, to April 30, 2021. These were thematically filtered using a 2-step approach, initially using COVID-19-related keywords and then using vaccine- and manufacturer-related keywords. We identified AEFI-related keywords and modeled their word frequency to monitor their trends over 2-week periods. We also adapted and utilized our recently developed hybrid ensemble model, which combines state-of-the-art lexicon rule-based and deep learning-based approaches, to analyze sentiment trends relating to the main vaccines available in the United Kingdom. RESULTS: Our COVID-19 AEFI search strategy identified 46,762 unique Facebook posts by 14,346 users and 74,644 tweets (excluding retweets) by 36,446 users over the 4-month period. We identified an increasing trend in the number of mentions for each AEFI on social media over the study period. The most frequent AEFI mentions were found to be symptoms related to appetite (n=79,132, 14%), allergy (n=53,924, 9%), injection site (n=56,152, 10%), and clots (n=43,907, 8%). We also found some rarely reported AEFIs such as Bell palsy (n=11,909, 2%) and Guillain-Barre syndrome (n=9576, 2%) being discussed as frequently as more well-known side effects like headache (n=10,641, 2%), fever (n=12,707, 2%), and diarrhea (n=16,559, 3%). Overall, we found public sentiment toward vaccines and their manufacturers to be largely positive (58%), with a near equal split between negative (22%) and neutral (19%) sentiments. The sentiment trend was relatively steady over time and had minor variations, likely based on political and regulatory announcements and debates. CONCLUSIONS: The most frequently discussed COVID-19 AEFIs on social media were found to be broadly consistent with those reported in the literature and by government pharmacovigilance. We also detected potential safety signals from our analysis that have been detected elsewhere and are currently being investigated. As such, we believe our findings support the use of social media analysis to provide a complementary data source to conventional knowledge sources being used for pharmacovigilance purposes.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.006 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.004 |
| Science and technology studies | 0.002 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".