Summon, EBSCO Discovery Service, and Google Scholar: A Comparison of Search Performance Using User Queries
Bibliographic record
Abstract
Abstract Objectives - To evaluate and compare the results produced by Summon and EBSCO Discovery Service (EDS) for the types of searches typically performed by library users at North Carolina State University. Also, to compare the performance of these products to Google Scholar for the same types of searches. Methods - A study was conducted to compare the search performance of two web-scale discovery services: ProQuest’s Summon and EBSCO Discovery Service (EDS). The performance of these services was also compared to Google Scholar. A sample of 183 actual user searches, randomly selected from the NCSU Libraries’ 2013 Summon search logs, was used for the study. For each query, searches were performed in Summon, EDS, and Google Scholar. The results of known-item searches were compared for retrieval of the known item, and the top ten results of topical searches were compared for the number of relevant results. Results - There was no significant difference in the results between Summon and EDS for either known-item or topical searches. There was also no significant difference between the performance of the two discovery services and Google Scholar for known-item searches. However, Google Scholar outperformed both discovery services for topical searches. Conclusions - There was no significant difference in the relevance of search results between Summon and EDS. Thus, any decision to purchase one of those products over the other should be based upon other considerations (e.g., technical issues, cost, customer service, or user interface).
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.011 | 0.065 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.008 | 0.009 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.004 | 0.004 |
| Open science | 0.001 | 0.002 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.001 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".