Bibliographic record
Abstract
Background: Systematic reviews (SRs) can be retrieved in several ways.One approach is a search filter designed to retrieve SRs in bibliographic databases such as PubMed.Another utilizes machine learning (ML) to sort articles into two mutually exclusive classes in online platforms such as DistillerSR.Objective: To test the performance of a binary ML classifier designed to identify SRs in DistillerSR against search filters designed to retrieve SRs in PubMed.Methods: Umbrella reviews will be identified, included SRs will be extracted to create a reference set.Each SR in the reference set will be verified as indexed in PubMed.Two SR search filters will be tested: a narrow SR filter and a broad SR filter.A binary ML classifier developed by DistillerSR to identify SRs will be used, informed by the reference set and a seed of non SR articles.The number of SRs from the reference set retrieved by the two search filters, and the number of SRs from the reference set correctly identified as a SR by the ML classifier will each be recorded discretely.Results: Relative recall, precision and F1 score will be calculated for each set.Recommendations will be made on when to apply a search filter or ML classifier to a search.Conclusions: Different approaches to limiting search results to specific study designs can influence the overall search strategy.The objectives of the project and resource requirements need to be considered when deciding on the approach. PP2. A review of library services and supports for the University of Prince Edward Island's (UPEI) growing One Health programs
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.018 | 0.039 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.003 |
| Bibliometrics | 0.010 | 0.005 |
| Science and technology studies | 0.002 | 0.001 |
| Scholarly communication | 0.010 | 0.005 |
| Open science | 0.004 | 0.007 |
| Research integrity | 0.006 | 0.004 |
| Insufficient payload (model declined to judge) | 0.540 | 0.326 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".