A multidimensional approach to assessing intervention fidelity in a process evaluation of audit and feedback interventions to reduce unnecessary blood transfusions: a study protocol
Bibliographic record
Abstract
BACKGROUND: In England, NHS Blood and Transplant conducts national audits of transfusion and provides feedback to hospitals to promote evidence-based practice. Audits demonstrate 20% of transfusions fall outside guidelines. The AFFINITIE programme (Development & Evaluation of Audit and Feedback INterventions to Increase evidence-based Transfusion practIcE) involves two linked, 2×2 factorial, cluster-randomised trials, each evaluating two theoretically-enhanced audit and feedback interventions to reduce unnecessary blood transfusions in UK hospitals. The first intervention concerns the content/format of feedback reports. The second aims to support hospital transfusion staff to plan their response to feedback and includes a web-based toolkit and telephone support. Interpretation of trials is enhanced by comprehensively assessing intervention fidelity. However, reviews demonstrate fidelity evaluations are often limited, typically only assessing whether interventions were delivered as intended. This protocol presents methods for assessing fidelity across five dimensions proposed by the Behaviour Change Consortium fidelity framework, including intervention designer-, provider- and recipient-levels. METHODS: (1) Design: Intervention content will be specified in intervention manuals in terms of component behaviour change techniques (BCTs). Treatment differentiation will be examined by comparing BCTs across intervention/standard practice, noting the proportion of unique/convergent BCTs. (2) Training: draft feedback reports and audio-recorded role-play telephone support scenarios will be content analysed to assess intervention providers' competence to deliver manual-specified BCTs. (3) Delivery: intervention materials (feedback reports, toolkit) and audio-recorded telephone support session transcripts will be content analysed to assess actual delivery of manual-specified BCTs during the intervention period. (4) Receipt and (5) enactment: questionnaires, semi-structured interviews based on the Theoretical Domains Framework, and objective web-analytics data (report downloads, toolkit usage patterns) will be analysed to assess hospital transfusion staff exposure to, understanding and enactment of the interventions, and to identify contextual barriers/enablers to implementation. Associations between observed fidelity and trial outcomes (% unnecessary transfusions) will be examined using mediation analyses. DISCUSSION: If the interventions have acceptable fidelity, then results of the AFFINITIE trials can be attributed to effectiveness, or lack of effectiveness, of the interventions. Hence, this comprehensive assessment of fidelity will be used to interpret trial findings. These methods may inform fidelity assessments in future trials. TRIAL REGISTRATION: ISRCTN 15490813 . Registered 11/03/2015.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.332 | 0.250 |
| Meta-epidemiology (narrow) | 0.006 | 0.005 |
| Meta-epidemiology (broad) | 0.006 | 0.009 |
| Bibliometrics | 0.010 | 0.009 |
| Science and technology studies | 0.008 | 0.007 |
| Scholarly communication | 0.007 | 0.006 |
| Open science | 0.007 | 0.009 |
| Research integrity | 0.007 | 0.011 |
| Insufficient payload (model declined to judge) | 0.033 | 0.008 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".