Additional Search Strategies May Not Be Necessary for a Rapid Systematic Review
Bibliographic record
Abstract
A Review of: Westphal, A., Kriston, L., Hölzel, L.P., Härter, M., & von Wolff, A. (2014). Efficiency and contribution of strategies for finding randomized controlled trials: a case study from a systematic review on therapeutic interventions of chronic depression. Journal of Public Health Research, 3(2), 177. doi: 10.4081/jphr.2014.177 Abstract Objective – To evaluate the efficiency and contribution of additional searching strategies for finding randomized controlled trials (RCTs) in a systematic review. Design – A methodological case study. Setting – Biomedical literature. Methods – A sensitive search (defined as “the ratio of the number of relevant reports identified to the total number of relevant reports in existence”) was conducted of electronic databases, Cochrane CENTRAL database, MEDLINE, EMBASE, PsycInfo, CINAHL, BIOSIS, and Web of Science databases (Science and Social Science Citation Indexes). The following additional searching strategies were conducted: hand-searching contents of relevant journals (Archives of General Psychiatry, Journal of Consulting and Clinical Psychology, and Journal of Affective Disorders), citation tracking (forwards tracking using Social Science and Science Citation Index and backwards tracking by looking through reference lists of included studies), screening reference lists of relevant systematic reviews, searching clinical trials registers (ClinicalTrials.gov and ICTRP registers), and contacting first authors of included studies to find any similar unpublished studies. The number of articles identified by each of these methods was recorded and screened for inclusion in the systematic review. The authors calculated what they labelled as the ‘efficiency’ of each searching strategy (the number of included studies identified by the search method as a proportion of the full text articles screened) and the ‘contribution’ of the search strategies (the ratio of included studies identified by that method to the final number of included studies in the systematic review). The methodological quality of each included study was assessed using the Cochrane Risk of Bias Tool, which is a critical appraisal checklist used to judge the study’s value in the systematic review. The meta-analysis in the systematic review was conducted with and without the studies identified by the additional searching strategies to assess their impact on the review’s findings. Main Results – In total 50 studies were identified, 42 from electronic database searches and 8 from additional search strategies. As illustrated by the results in Table 1, the most useful additional search strategy was screening reference lists of relevant systematic reviews. Journal hand-searching and contacting authors also contributed to the review. Of the eight studies identified by the additional search strategies none were judged to have a low risk of bias (four had high risk of bias and four were unclear). Of the 42 included studies from electronic searches only 11 were judged to have a low risk of bias, whereas 9 studies had a high risk of bias and 22 were unclear. Excluding the eight studies retrieved from additional search strategies in the systematic review meta-analysis did not influence the results on the effectiveness of the different interventions for chronic depression. These studies were found to be indexed correctly on the electronic databases, but were not identified in the initial search. Conclusion – Additional search strategies, especially screening reference lists of systematic reviews and hand-searching relevant journals, retrieved a substantial number of relevant studies for a systematic review of interventions for treating chronic depression. However, results of the review’s meta-analysis did not differ when these additional studies (rated as either high or unclear risk of bias) were not included and search methods were time consuming. It might be reasonable to rely on electronic searching strategies when resources for conducting a systematic review are limited or when doing a “rapid review.” The benefits and limitations of additional search strategies should be considered particularly when resources or time for conducting a systematic review are limited. If the electronic database search is sensitive and includes the Cochrane CENTRAL database additional search strategies may not be necessary, but these findings should be tested in other research areas.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.005 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.114 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.018 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".