Undergraduates Prefer Federated Searching to Searching Databases Individually
Bibliographic record
Abstract
A Review of: Belliston, C. Jeffrey, Jared L. Howland, & Brian C. Roberts. “Undergraduate Use of Federated Searching: A Survey of Preferences and Perceptions of Value-Added Functionality.” College & Research Libraries 68.6 (Nov. 2007): 472-86. Objective – To determine whether use of federated searching by undergraduates saves time, meets their information needs, is preferred over searching databases individually, and provides results of higher quality. Design – Crossover study. Setting – Three American universities, all members of the Consortium of Church Libraries & Archives (CCLA): BYU (Brigham Young University, a large research university); BYUH (Brigham Young University – Hawaii, a small baccalaureate college); and BYUI (Brigham Young University – Idaho, a large baccalaureate college) Subjects – Ninety-five participants recruited via e-mail invitations sent to a random sample of currently enrolled undergraduates at BYU, BYUH, and BYUI. Methods – Participants were given written directions to complete a literature search for journal articles on two biology-related topics using two search methods: 1. federated searching with WebFeat® (implemented in the same way for this study at the three universities) and 2. a hyperlinked list of databases to search individually. Both methods used the same set of seven databases. Each topic was assigned in random order to one of the two search methods, also assigned in random order, for a total of two searches per participant. The time to complete the searches was recorded. Students compiled their list of citations, which were later normalized and graded. To analyze the quality of the citations, one quantitative rubric was created by librarians and one qualitative rubric was approved by a faculty member at BYU. The librarian-created rubric included the journal impact factor (from ISI’s Journal Citation Reports®), the proportion of citations from peer-reviewed journals (determined from Ulrichsweb.com™) to total citations, and the timeliness of the articles. The faculty-approved rubric included three criteria: relevance to the topic, quality of the individual citations (good quality: primary research results, peer-reviewed sources), and number of citations. Data were then analysed using ANOVA and MANOVA. Finally, librarians at the ACRL 13th National Conference Presentation were polled about their perceptions of the time savings of federated searching, whether the method meets undergraduates’ information needs, undergraduate preference for searching, and the quality of citations found. Main Results – Seventy percent of all participants preferred federated searching. For all schools combined, there was no statistically significant difference between the average time taken using federated searching (20.34 minutes) vs. non-federated searching (22.72 minutes). For all schools combined, there was a statistically significant difference in satisfaction of results favouring federated searching (5.59/7 vs. 4.80/7 for non-federated searching, α = .05). According to the librarian-created rubric, citations retrieved from federated searching were a statistically significant 6% lower in quality than citations retrieved from non-federated searching (α = .05). The faculty-approved rubric did not detect a difference in the quality of the citations retrieved using the 2 methods. Librarians’ perceptions as assessed at the ACRL 13th National Conference Presentation generally matched the authors’ findings. Conclusion – Overall, students in this study preferred federated searching, were more satisfied with the results of federated searching, and saved time (although the savings were not statistically significant). The quality of citations retrieved via both methods was judged to be similar. The study provides useful information for librarians interested in users’ experiences and perceptions of federated searching, and indicates future studies worth conducting.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.003 | 0.019 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.003 | 0.002 |
| Open science | 0.000 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.007 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".