Extending the Calgary Audit and Feedback Framework into the virtual environment: a process evaluation and empiric evidence
Bibliographic record
Abstract
Abstract Background The Calgary Audit and Feedback Framework (CAFF) is a pragmatic, evidence-based approach for the design and implementation of in-person social learning interventions using Audit and Group Feedback (AGF). This report describes extension of CAFF into the virtual environment as part of a multifaceted intervention bundle to reduce redundant daily laboratory testing in hospitals. We evaluate the process of extending CAFF in the virtual environment and share resulting evidence of participant engagement with planning for practice change. Methods We describe an innovative virtually facilitated AGF intervention based on the CAFF. The AGF intervention was part of an intervention bundle which included individual physician laboratory test utilization reports and educational tools to reduce redundant daily laboratory testing in hospitals. We used data from recorded and transcribed virtual AGF sessions, post AGF session surveys and detailed field notes maintained by project team members. We used simple descriptive statistics for quantitative data and analyzed qualitative data according to the elements of CAFF. Results Eighty-three physicians participated over twelve virtual AGF sessions conducted across four tertiary care hospitals during the study period. We demonstrate that all prerequisite activities for CAFF (relationship building, question choice and data representation) were present in every virtual AGF session. Virtual facilitation was effective in supporting the transition of participants through different steps of CAFF in each session to lead to change talk and planning. All participants contributed to discussion during the AGF sessions. The post AGF session surveys were filled by 66% of participants (55/83), with over 90% of respondents reporting that the session helped them improve practice. 46% of participants (38/83) completed personal commitment to change forms at the end of the sessions. Conclusions Virtual AGF sessions, developed and implemented with fidelity to the CAFF approach, successfully engaged physicians in a group learning environment that led to change planning. Further studies are needed to determine the generalizability of our findings and to add to the literature on evidence-based virtual facilitation techniques.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.396 | 0.331 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.002 |
| Bibliometrics | 0.006 | 0.006 |
| Science and technology studies | 0.003 | 0.005 |
| Scholarly communication | 0.005 | 0.003 |
| Open science | 0.003 | 0.005 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".