Textual evidence systematic reviews series paper 2: challenges and strategies in developing a search strategy for systematic reviews of textual evidence
Bibliographic record
Abstract
OBJECTIVE: The objective of this paper is to highlight and address challenges as well as provide strategies for developing searches for systematic reviews of textual evidence. INTRODUCTION: When conducting a JBI review of textual evidence, it is important to consider different sources of published and unpublished material. While systematic search methodologies have been well-established for searching traditional peer-reviewed literature, applying those same rigorous methods to literature outside of academic journals can be more challenging. This paper highlights and addresses the challenges of developing searches for systematic reviews of textual evidence and provides strategies for how to conduct these. It takes into consideration the unique complexities of locating published material outside of academic journals and presents guidance for developing more robust searches incorporating textual evidence. DISCUSSION: Researchers should acknowledge the value of textual evidence, including opinions, narratives, and policies, as crucial for informing health care practices. It is also essential to clearly define the types of textual evidence needed and establish comprehensive search parameters to ensure thorough coverage. To enhance the search process, researchers should follow a structured 3-phase approach: first, identify relevant keywords; second, conduct tailored searches in bibliographic databases; and third, perform supplementary searches. Furthermore, it is recommended that researchers collaborate with information specialists and experts to refine and strengthen their search techniques. Researchers should also explore a variety of sources, including dedicated databases, conference proceedings, theses, dissertations, and media reports, to gather valuable textual evidence. Finally, it is important to systematically document all search processes to support transparency and reproducibility in the review. CONCLUSION: Searching broadly across bibliographic databases and including textual evidence from non-academic journals may provide the best available and most appropriate evidence to address specific questions.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.286 | 0.564 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.016 | 0.002 |
| Bibliometrics | 0.001 | 0.003 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.002 | 0.005 |
| Open science | 0.004 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; both teacher heads agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".