New Search Strategies Optimize MEDLINE Retrieval of Sound Studies on Treatment or Prevention of Health Disorders. A review of: Haynes, R. Brian, K. Ann McKibbon, Nancy L. Wilczynski, Stephen D. Walter, and Stephen R. Were. “Optimal search strategies for retrieving scientifically strong studies of treatment from Medline: analytical survey.” BMJ 330.7501 (21 May 2005): 1179.
Bibliographic record
Abstract
<b>Objective</b> – To develop and test search strategies for retrieving clinically sound studies from the MEDLINE database on the prevention or treatment of health disorders. <br><b>Design</b> – Analytical survey. <br><b>Subjects</b> – The data sources were articles about treatment studies selected from 161 journal titles indexed for MEDLINE in the year 2000. <br><b>Setting</b> – MEDLINE database searches performed at the Health Information Research Unit, McMaster University, in Ontario, Canada. <br><b>Methods</b> – Researchers hand searched each issue of 161 journal titles indexed in MEDLINE in the year 2000 to find treatment studies. Journal content included internal medicine, family practice, nursing, and mental health titles. Selected studies met the following criteria: randomisation of subjects, outcome assessment for at least 80% of who entered the study, and an analysis consistent with study design. Of 49,028 potential articles, 6,568 were identified as being treatment or prevention related, and 1,587 met the evaluation criteria. The study authors then created search strategies designed to retrieve articles in MEDLINE that met the same criteria, while excluding articles that did not. They compiled a list of 4,862 unique terms related to study criteria, and tested them using the Ovid Technologies search platform. Overall, 18,404 multiple‐term search strategies were tested. Single terms with specificity greater than 75% and sensitivity greater than 25% were combined into strategies with two or more terms. These multiple term strategies were tested if they yielded sensitivity or accuracy greater than 75% and specificity of at least 50%. <br><b>Main Results</b> – Of the 4,862 unique terms, 3,807 retrieved citations from MEDLINE that researchers used to assess sensitivity, specificity, precision, and accuracy. The single term that yielded the best accuracy while keeping sensitivity greater than 50% was ‘randomized controlled trial.pt.’. The single term that yielded the best precision while keeping sensitivity greater than 50% was also ‘randomized controlled trial.pt.’.This term also gave the greatest balance of sensitivity and specificity. Combination strategies varied. Some two term combinations outperformed single term strategies and three‐term combinations. Tables in the article provide the top three search strategies yielding the highest sensitivity, specificity, accuracy, and balance between sensitivity and specificity. Search strategies with sensitivity greater than 50% and specificity greater than 95% were evaluated further by adding search terms using logistic regression techniques. The best strategies for maximizing sensitivity had sensitivity greater than 99% and specificity higher than 70%. The best strategies for maximizing specificity had sensitivity greater than 93% and specificity more than 97%. <br><b>Conclusion</b> – In addition to providing the best strategies developed in this study, authors compared their results with the results from 19 other published strategies. The published strategies had a sensitivity range of 1.3% to 98.8% on the basis of the hand searched articles. These were all lower than this study’s best sensitivity of 99.3%. Two strategies published by Dumbrigue, with specificities of 98.1% and 97.6%, outperformed this study’s most specific strategy of 97.4%.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.019 | 0.004 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.009 | 0.001 |
| Bibliometrics | 0.001 | 0.002 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".