Searching for Adverse Effects in MEDLINE and EMBASE Requires a Combined Approach for Efficient Retrieval
Bibliographic record
Abstract
A review of: Golder, Su, Heather M. McIntosh, Steve Duffy, and Julie Glanville. “Developing Efficient Search Strategies to Identify Reports of Adverse Effects in MEDLINE and EMBASE.” Health Information & Libraries Journal 23.1 (Mar. 2006): 3-12. Objective – To assess the sensitivity and precision of various search strategies for retrieving adverse effects studies from the MEDLINE and EMBASE databases. Design – Analytical survey. Subjects – A case study using a recently published systematic review of the effectiveness and adverse effects of seven new anti-epileptic drugs. Setting – MEDLINE and EMBASE searches performed by researchers at the Centre for Reviews and Dissemination and the UK Cochrane Centre Search Filters Design Group at the University of York, UK. Methods – Five key approaches to searching were defined. The first approach used either text words or controlled vocabulary to search for specific adverse effects. The second used subheadings or qualifiers either attached to drug names found in the controlled vocabulary (approach 2a) or ‘floating’ without drug names (approach 2b). The third approach used text words as synonyms for the phrase ‘adverse effects.’ The fourth used controlled indexing terms for adverse effects. The fifth and final approach used two published search strategies incorporating study design (Badgett et al., Loke et al.). These five approaches were used to search for studies of the adverse effects of seven new anti-epileptic drugs. 5,011 unique papers were retrieved. Of these, 236 were judged potentially relevant and 225 full text articles were obtained. The inclusion criteria from a previously published systematic review (Wilby et al.) were applied to the papers, and 79 met the criteria. Five papers were added to the set after being identified from reference lists, clinical experts, and other sources. This new set of 84 studies was used as a quasi gold standard (QGS) against which more than 300 combinations of the five approaches could be tested. To create the set of possible approaches, the researchers combined search strategies one through four in all possible ways, and used all available subheading combinations from 2a and 2b. The Badgett and Loke searches were tested separately. Main Results – Sensitivity and precision were determined for each combination. Formulas used to calculate sensitivity and precision were provided. In MEDLINE, search strategies using floating subheadings achieved the highest sensitivity. The most useful single subheading in both MEDLINE and EMBASE was “adverse effects,” with 79.1% and 79.5% sensitivity respectively. Of the more than 300 combinations tested, the most sensitive combination in MEDLINE included specified adverse effects in combination with the floating subheadings “adverse effects,” “complications,” and “drug effects,” together with text words for adverse effects. This strategy had 97.0% sensitivity, but low precision at 2.8%. The highest precision was achieved by using subheadings attached to drug indexing terms. In EMBASE, the strategy of Loke et al. provided the highest sensitivity at 86.3% and precision of 2.0%. Since researchers are not likely to know in advance all of the reported adverse effects of a particular drug therapy, the most sensitive strategies without specific adverse events were also identified. The search with the highest sensitivity in MEDLINE had 95.5% sensitivity, and 97.3% sensitivity in EMBASE. Conclusion – Searching for adverse effects requires a combination of approaches in both MEDLINE and EMBASE. In MEDLINE, the most sensitive combination yielded 97.0% sensitivity. Regardless of the approach used, precision remains low. An effective generic search filter for adverse effects searches may not yet be feasible. More research is needed on search strategies, as well as more consistent methods of reporting and indexing adverse effects.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.090 | 0.310 |
| Meta-epidemiology (narrow) | 0.003 | 0.003 |
| Meta-epidemiology (broad) | 0.015 | 0.007 |
| Bibliometrics | 0.082 | 0.077 |
| Science and technology studies | 0.002 | 0.004 |
| Scholarly communication | 0.009 | 0.013 |
| Open science | 0.005 | 0.008 |
| Research integrity | 0.004 | 0.004 |
| Insufficient payload (model declined to judge) | 0.106 | 0.034 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".