Google Scholar Retrieves Twice as Many Relevant Citations as PubMed and Provides Greater Full-Text Access for Quick, Clinical Nephrology Searches
Bibliographic record
Abstract
Objective – To compare recall and precision of results retrieved by searches in PubMed and Google Scholar for clinical nephrology literature. 
 
 Design – Survey questionnaire, comparative.
 
 Setting – Canada.
 
 Subjects – Practicing nephrologists with average age of 48 years and who have practiced nephrology for an average of 15 years.
 
 Methods – The researchers identified 100 systematic reviews in renal therapy published between 2001 and 2009. The primary studies cited in the systematic reviews served as the reference standard for relevant articles; 1,574 unique citations were identified and used to measure recall and precision. The researchers created a unique clinical question from each of the objective statements of systematic reviews and sent one question to a random sample of practicing nephrologists to determine the search strings they would use to search for clinical literature; the researchers collected 100 usable responses. Using the search string in both Google Scholar and PubMed, the researchers analyzed the first 40 retrieved results in each for recall of relevant literature and precision. The researchers also analyzed the availability of full-text articles in each database. A pilot study to test the methodology preceded the main study.
 
 Results – Google Scholar’s recall for the first 40 records was 21.9% and PubMed was 10.9%. Each database contained 78% of the relevant literature/reference standard set from the systematic reviews. However, 15% of the articles were in neither database. Precision results were similar (7.6% for Google Scholar and 5.6% for PubMed). Google Scholar had more full-text available at 15% of articles versus 5% for PubMed. Google Scholar and PubMed had similar numbers of relevant articles when all retrieved records were analyzed, but Google Scholar still provided more access to free full-text articles. 
 
 Conclusion – Google Scholar provides better recall and provides more access to full-text than PubMed; however, search strings provided by nephrologists used in both databases failed to retrieve 80% of relevant articles. Therefore improving nephrologists’ ability to effectively search could enhance their ability to implement research in practice helping patients. The researchers suggest future studies should be conducted to determine the generalizability of the findings on recall and precision in other medical disciplines.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.035 | 0.355 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.010 | 0.021 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.011 | 0.230 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; both teacher heads agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".