Direct quantitative comparison of benefits and risks of COVID-19 vaccines used in National Immunization Technical Advisory Groups Guidance during the first two years of the pandemic
Bibliographic record
Abstract
INTRODUCTION: The balance of benefits and harms of vaccines are assessed by regulatory agencies and National Immunization Technical Advisory Groups (NITAGs) to inform vaccine authorization or guidance. The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach has been adopted by many NITAGs to develop recommendations. During the COVID-19 pandemic, several NITAGs additionally used direct quantitative comparisons (DQCs) between benefits and risk of vaccination with or without a GRADE framework to support timely decision-making relating to emerging safety signals. This study aimed to document the role of DQCs as novel tools in NITAGs' work by identifying situations where DQCs have been clearly leveraged in NITAG guidance, as well as identifying their strengths and limitations. METHODS: The MEDLINE database and NITAGs' websites listed in the Global NITAG Network were searched for NITAG publications on COVID-19 vaccines. Publications were included if a DQC between benefits and risks of any COVID-19 vaccine was explicitly used for NITAG decision-making. Two reviewers independently assessed publication eligibility and extracted data. A narrative description of the role of DQCs in NITAG guidance, DQCs' methods and limitations was conducted. RESULTS: Overall, 23 publications with 18 DQCs used by seven NITAGs were included. Situations prompting these publications included new safety signals (n = 7), additional information available on previously identified safety signals (n = 4) and changing contexts (n = 15) (e.g., vaccine supply, and epidemiology). DQC simplicity made them accessible, timely, and allowed for transparent communication. DQCs heavily relied on assumptions making them sensitive to changes in model parameters. DQCs limitations made them not easily transferable to other contexts and they quickly became obsolete in the evolving context of the COVID-19 pandemic. CONCLUSIONS: The use of DQCs by NITAGs during the COVID-19 pandemic allowed for rapid evidence-based decision-making in an evolving environment while maintaining public trust. However, if their use becomes standard practice, efforts should be made to address their limitations.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.132 | 0.461 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.003 | 0.005 |
| Bibliometrics | 0.036 | 0.026 |
| Science and technology studies | 0.001 | 0.002 |
| Scholarly communication | 0.007 | 0.006 |
| Open science | 0.002 | 0.004 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.009 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".