Summon, EBSCO Discovery Service, and Google Scholar: A Comparison of Search Performance Using User Queries
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Abstract Objectives - To evaluate and compare the results produced by Summon and EBSCO Discovery Service (EDS) for the types of searches typically performed by library users at North Carolina State University. Also, to compare the performance of these products to Google Scholar for the same types of searches. Methods - A study was conducted to compare the search performance of two web-scale discovery services: ProQuest’s Summon and EBSCO Discovery Service (EDS). The performance of these services was also compared to Google Scholar. A sample of 183 actual user searches, randomly selected from the NCSU Libraries’ 2013 Summon search logs, was used for the study. For each query, searches were performed in Summon, EDS, and Google Scholar. The results of known-item searches were compared for retrieval of the known item, and the top ten results of topical searches were compared for the number of relevant results. Results - There was no significant difference in the results between Summon and EDS for either known-item or topical searches. There was also no significant difference between the performance of the two discovery services and Google Scholar for known-item searches. However, Google Scholar outperformed both discovery services for topical searches. Conclusions - There was no significant difference in the relevance of search results between Summon and EDS. Thus, any decision to purchase one of those products over the other should be based upon other considerations (e.g., technical issues, cost, customer service, or user interface).
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.002 | 0.529 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it