Web-Based Portal for Impact Evaluation Reveals Information Needs for Museums, Libraries and Archives
Bibliographic record
Abstract
Objective – This study reports on research into the information and support needs of practitioners in the museum, archive, and library sectors, who are undergoing an impact evaluation. Design – Qualitative survey. Setting – Web-based questionnaire. Subjects – Twenty-one practitioners in the fields of museums, archives, and libraries. Methods – The study made use of a small-scale web portal that provides impact evaluation research findings, toolkits, and examples of methods. The portal’s intent was to present to the users multiple views of the available information in order to overcome the problem of users not being able to identify their needs. A purposive sample group consisting of 50 practitioners from the museum, library, and archive fields was invited to participate in a questionnaire evaluating the website. Main Results – Despite a fairly low response rate (49%) and poor distribution among the three sectors (museums, libraries, and archives), the results indicated a significant difference in the levels of knowledge and understanding of impact evaluation. Over half of the organizations surveyed had done some assessment of their institution’s economic impact, and there appears to be a rising trend towards doing impact studies for specific projects and developments. Nearly a quarter of the organizations had not undertaken any impact evaluation study previously. Practitioners already familiar with impact evaluation tended to look at broader range of fields for expertise, whereas those with less familiarity remained within their own sector. Practitioners with less experience preferred tools, guidance, and examples of methodologies as opposed to actual evidence of impact. The results also provided the authors with feedback on their web portal and how to organize the information therein. Conclusions – One of the findings of the study was that the overall reaction to impact evaluation support through research evidence, guidance, and other mechanisms was positive. For most practitioners, evaluation itself and the level of understanding of impact evaluation are at early stages. The primary goals for those undertaking impact evaluation were found to be professional and organizational learning, thus there is a need for practical help and guidance in these areas. Time limitation appeared to be a significant factor in the responses – particularly with smaller organizations – suggesting that their portal material would provide much-needed assistance to such organizations. Finally, it was concluded that future emphasis should be placed on developing practical applications rather than pure research.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.012 | 0.028 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.003 | 0.004 |
| Science and technology studies | 0.002 | 0.001 |
| Scholarly communication | 0.004 | 0.006 |
| Open science | 0.001 | 0.003 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.012 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".