Utility of social media and crowd-sourced data for pharmacovigilance: a scoping review protocol
Bibliographic record
Abstract
INTRODUCTION: Adverse events associated with medications are under-reported in postmarketing surveillance systems. A systematic review of published data from 37 studies worldwide (including Canada) found the median under-reporting rate of adverse events to be 94% in spontaneous reporting systems. This scoping review aims to assess the utility of social media and crowd-sourced data to detect and monitor adverse events related to health products including pharmaceuticals, medical devices, biologics and natural health products. METHODS AND ANALYSIS: Our review conduct will follow the Joanna Briggs Institute scoping review methods manual. Literature searches were conducted in MEDLINE, EMBASE and the Cochrane Library from inception to 13 May 2016. Additional sources included searches of study registries, conference abstracts, dissertations, as well as websites of international regulatory authorities (eg, Food and Drug Administration (FDA), the WHO, European Medicines Agency). Search results will be supplemented by scanning the references of relevant reviews. We will include all publication types including published articles, editorials, websites and book sections that describe use of social media and crowd-sourced data for surveillance of adverse events associated with health products. Two reviewers will perform study selection and data abstraction independently, and discrepancies will be resolved through discussion. Data analysis will involve quantitative (eg, frequencies) and qualitative (eg, content analysis) methods. DISSEMINATION: The summary of results will be sent to Health Canada, who commissioned the review, and other relevant policymakers involved with the Drug Safety and Effectiveness Network. We will compile and circulate a 1-page policy brief and host a 1-day stakeholder meeting to discuss the implications, key messages and finalise the knowledge translation strategy. Findings from this review will ultimately inform the design and development of a data analytics platform for social media and crowd-sourced data for pharmacovigilance in Canada and internationally. REGISTRATION DETAILS: Our protocol was registered prospectively with the Open Science Framework (https://osf.io/kv9hu/).
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.149 | 0.128 |
| Meta-epidemiology (narrow) | 0.006 | 0.007 |
| Meta-epidemiology (broad) | 0.013 | 0.014 |
| Bibliometrics | 0.025 | 0.020 |
| Science and technology studies | 0.006 | 0.006 |
| Scholarly communication | 0.010 | 0.011 |
| Open science | 0.008 | 0.009 |
| Research integrity | 0.011 | 0.007 |
| Insufficient payload (model declined to judge) | 0.072 | 0.020 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".