Has the Internet Changed Anything in Reference? The Library Visit Study, Phase 2.
Bibliographic record
Abstract
The Library Visit Study, Phase 2 Phase 2 of the Library Visit study examines 161 user accounts of what happened when users asked a reference question in a public or academic library of their choice between fall 1998 and spring 2000. Although the over-all success rate increased from 60 percent in phase 1 in the early 1990s to 69 percent in the current period, the problems identified in the unsuccessful cases were the same as those identified in phase 1. These problems include failure to conduct a reference interview, unmonitored referral, and failure to ask follow-up questions. A new focus of analysis in phase 2 addresses how the availability of electronic resources, including the Web, may have changed the reference transaction and affected the user's experience. A troubling finding is that reference staff seem to regard the Internet as an external resource that users can search independently--at home or on the library's public access workstations--but not as a full-fledged reference tool for which reference librarians have a responsibility to help users search and evaluate. Like so many research projects, this one started with a puzzle that arose in the course of teaching. The authors teach sections of a basic reference course with some other colleagues at the University of Western Ontario. Over the years, those of us who teach this course have put together a large file of reference questions from which we select ten reference questions for a class assignment for each course offering. Questions that turn out to be too easy or too hard are discarded, so that the question file contains several hundred tried-and-tested reference questions and best answers, classified by type of reference tool needed to answer the question. The faculty members responsible for the course recently noticed a problem: questions that previously had been adequately challenging now were too easy. For example, in the fall 1999 term, one of the authors used the question, Who is child and what is child known for? This question had formerly required students to analyze the question and take a series of logical steps to find the answer, but the 1999 students were able to find an answer in a few minutes. All they had to do was type Dart's Child into Google (www.google.com) and within the top ten hits could find several sources that either provided the answer or provided a shortcut to the answer. We wondered how many other of our reference questions could be answered by authoritative Web sources retrieved within the first ten Google hits. We decided to check systematically and discovered that we had to throw out a third of our tried-and-true questions. By virtue of answers available through just one search engine, a large number of questions--148 out of 442--had become too easy to provide useful learning experiences for students. For us, this was a dramatic illustration of the way in which the World Wide Web has transformed the landscape of reference tools. Despite all the frequently voiced criticisms of the Web--that it contains unfiltered information of dubious authority, that methods of subject control and retrieval are slap-dash, that Web pages are unstable and fugitive--one search engine, Google, proved to be an effective shortcut for finding an answer to one-third of our questions. Literature Review By 1998, evidence was beginning to accumulate in the library literature about the value of answering reference questions with resources in freely available Web sites. For example, Janes and McClure asked whether reference can now be done as well in the Web environment as in the traditional print environment and answered yes. This conclusion was based on search outcomes when twenty-four volunteer searchers tried to find answers to twelve ready reference questions using either Web or non-Web resources.[1] In this study, Web resources were restricted to freely available Web pages and did not include licensed commercial products. …
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.014 | 0.038 |
| Meta-epidemiology (narrow) | 0.000 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.002 | 0.002 |
| Science and technology studies | 0.005 | 0.002 |
| Scholarly communication | 0.005 | 0.005 |
| Open science | 0.002 | 0.004 |
| Research integrity | 0.004 | 0.004 |
| Insufficient payload (model declined to judge) | 0.005 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".