Libraries’ Contributions to the Quality of UK University Research Environments Were Not Acknowledged in REF 2014, but Could Be Made More Visible in REF 2021
Bibliographic record
Abstract
A Review of: Walker, D. (2020). Libraries and the REF: How do librarians contribute to research excellence? Insights, 33(1), 6. https://doi.org/10.1629/uksg.497 Abstract Objective – To measure the extent to which libraries’ contributions to United Kingdom (UK) university research excellence were referenced in the Research Excellence Framework (REF) 2014 unit-level research environment statements, and to make recommendations to libraries for increasing their visibility in the research setting. Design – Content analysis of an existing corpus. Setting – Evaluation of research environments conducted as part of the UK REF 2014 assessment. Subjects – 1,891 unit-level research environment statements submitted for REF 2014. Methods – Each unit-level research environment statement was categorized in terms of how extensively it referenced library or librarian contributions: no mention, brief mention, or substantive mention. The frequency and percentage of each level of mention are reported overall and by disciplinary panel. Main Results – Across all panels, only 25.8% of the statements included substantive references to the library or librarians; most of these were lists of electronic and physical collections, but they also included discussions of the research support services offered by librarians. There were disciplinary differences in the extent of the references to libraries, from 7.2% containing substantive references in a panel examining science, technology, engineering, and mathematics (STEM) units to 44.0% containing substantive references in the panel examining arts and humanities units. Conclusion – In REF 2014, libraries and librarians were rarely discussed in unit-level research environment statements. While this lack of representation may be due to shortcomings of the library’s relationship with the university’s research office, librarians could use a number of approaches to becoming more visible in the REF 2021 research environment statements. Specifically, they could highlight their roles in: ensuring discoverability and accessibility of information resources to researchers; improving research practices through teaching informational and organizational skills, providing direct support to research students and staff, and providing research data management services; managing the research information systems that capture and make discoverable the university’s non-article research outputs; providing support in relation to the responsible use of bibliometrics and other measures of article quality and impact; further developing article impact by training researchers to use social media to their advantage; developing open research initiatives; and assisting with the REF submission process.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.079 | 0.300 |
| Meta-epidemiology (narrow) | 0.000 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.010 | 0.019 |
| Science and technology studies | 0.008 | 0.008 |
| Scholarly communication | 0.017 | 0.014 |
| Open science | 0.002 | 0.016 |
| Research integrity | 0.004 | 0.004 |
| Insufficient payload (model declined to judge) | 0.009 | 0.003 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".