What's in a name? Measuring use and non-use of Digital Resources in the Arts and Humanities through Log Analysis Techniques
Bibliographic record
Abstract
September 3-6, Dartington College of Arts. The LAIRAH (Log Analysis of Internet Resources in the Arts and Humanities) project aims to determine whether, how and why digital resources in the humanities are used, and what factors might make them usable and sustainable. Based at UCL and funded by the UK Arts and Humanities Research Council (AHRC) ICT Strategy scheme, LAIRAH is a year long study which will analyse patterns of usage of online resources through server log analysis (http://www.ucl.ac.uk/slais/LAIRAH/). Log files have been provided by the three online archives supported by AHRC: the AHDS Arts and Humanities Collection (http://www.ahds.ac.uk/); Humbul Humanities Hub (http://www.humbul.ac.uk/) and the Artifact database for the creative and performing arts (http://www.artifact.ac.uk/). The search logs show patterns of which resources are interesting to users, and in the case of the AHDS (which provides links to resources themselves), which ones users visit. Method Despite the creation of over three hundred digital resources funded by the AHRC alone, there is little robust evidence regarding which are used and which neglected.[1] We are therefore analysing a year's worth of transaction log data (a record of web page use automatically collected by servers) from each resource. This provides the words searched on (search logs), the pages viewed (user logs), the web site that the user has come from (referrer logs), and basic, but anonymous, user identification tags, time, and date stamps. We have also designed short online questionnaires, covering user characteristics and perceived outcomes, which will be compared with actual search and usage patterns: research undertaken by UCL's CIBER team has shown a discrepancy between resources actually used, and those that users report having visited. [2] Findings Whether or not they come from an academic domain, most users search for certain resource subjects: war, witchcraft, fashion, suffragettes, war poetry, Shakespeare and census data. More obscure subjects had much lower levels of use, and a worrying number appear never to have been accessed. The title of the resource proved surprisingly important: resources with the words 'census data' in the title are used, whereas one called 'enumerator returns' remained neglected. We found that users from academic domains tend to be more persistent, searching for longer and accessing more resources than those coming from commercial servers. The most heavily used areas of the AHDS central website are those giving information about depositing resources rather than the use of already archived resources. This suggests that digital humanities may still be in a phase of creation rather than re-use of resources. Conclusion Use levels alone cannot be an absolute measure of a resource's quality or usefulness. Some digital humanities resources are created as pure research, or for testing technical solutions, and should not necessarily be measured by their use levels. Equally a resource that is very valuable for a small community may not be widely used. However if a large potential audience ignores a seemingly useful resource, we need to understand why. It is these issues that we will address in the remainder of the project and will report upon in this paper. References [1] Warwick, C., Blandford A. and Buchanan, G. (2005) User Centred Interactive Search: A study of humanities users in a digital library environment. Presented at the Association for Computers and the Humanities- Association for Literary and Linguistic Computing, Conference 2005. University of Victoria, Canada, June 15-18. [2] Huntington P, Nicholas D, Williams, P. (2003) 'Characterising and profiling health web users and site types: going beyond hits'. Aslib Proceedings 55 (5/6): 277-289
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.003 | 0.017 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.004 | 0.007 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.004 | 0.003 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.005 | 0.003 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".