PROSPERO at one year: an evaluation of its utility
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
BACKGROUND: PROSPERO, an international prospective register of systematic review protocols in health and social care, was launched in February 2011. After one year of operation we describe access and use, explore user experience and identify areas for future improvement. METHODS: We collated administrative data and web statistics and conducted an online survey of users' experiences. RESULTS: On 21 February 2012, there were 1,076 registered users and 359 registration records published on PROSPERO. The database usage statistics demonstrate the international interest in PROSPERO with high access around the clock and around the world. Based on 232 responses from PROSPERO users (response rate 22%), almost all respondents found joining and navigation was easy or very easy (99%); turn round time was good or excellent (96%); and supporting materials provided were helpful or very helpful (80%). The registration fields were found by 80% to be relevant to their review; 99% rated their overall experience of registering with PROSPERO as good or excellent. Most respondents (81%) had a written protocol before completing the registration form and 19% did not. The majority, 136 (79%), indicated they completed the registration form in 60 minutes or less. Of those who expressed an opinion, 167 (87%) considered the time taken to be about right. CONCLUSIONS: The first year of PROSPERO has shown that registration of systematic review protocols is feasible and not overly burdensome for those registering their reviews. The evaluation has demonstrated that, on the whole, survey respondents are satisfied and the system allows registration of protocol details in a straightforward and acceptable way. The findings have prompted some changes to improve user experience and identified some issues for future consideration.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.464 | 0.166 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.010 | 0.002 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.002 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.077 | 0.064 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it