073WS Collaborative Guideline Implementability Tool Development and Evaluation
Bibliographic record
Abstract
Background Research shows that guidelines are more easily translated to practice when accompanied by information that helps users accommodate, implement and evaluate use of the recommendations. We refer to this information in guidelines or other resources as guideline implementability tools (GItools). We identified exemplar GItools, generated criteria by which to assess and develop GItools through consultation with G-I-N members, and created an online directory for sharing of GItools. Discussion with G-I-N members would help us improve these resources and identify partners for collaborative GItool development. This work is a core activity of GIRAnet: the Guideline Implementability Research and Application Network ( http://www.g-i-n.net/activities/implementation/giranet ). Objectives To share information about GItool resources; gather feedback on how to improve GItool resources; learn about other GItool initiatives; and establish partners for joint development or evaluation of GItools. AUDIENCE Guideline developers, implementers, users (clinicians, managers, policy-makers) and researchers. Description A brief presentation will define GItools and describe their potential purpose and impact; review methods used to identify, describe and evaluate exemplar GItools; and demonstrate the GItool Directory (20 minutes). Participants will be asked to individually rate, then discuss enhancements for GItool assessment criteria (20 min). Open discussion will enable sharing and mutual learning about other GItool initiatives, and explore partnerships for future GItool initiatives (40 min). Remaining time will be used to identify priorities for ongoing research (10 min). Participant feedback will be used to improve GItool resources, and guide ongoing GIRAnet projects.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.149 | 0.214 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.003 |
| Bibliometrics | 0.005 | 0.004 |
| Science and technology studies | 0.002 | 0.002 |
| Scholarly communication | 0.007 | 0.003 |
| Open science | 0.003 | 0.009 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.085 | 0.026 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".