Practice Doesn’t Always Make Perfect: A Qualitative Study Explaining Why a Trial of an Educational Toolkit Did Not Improve Quality of Care
Bibliographic record
Abstract
BACKGROUND: Diabetes is a chronic disease commonly managed by family physicians, with the most prevalent complication being cardiovascular disease (CVD). Clinical practice guidelines have been developed to support clinicians in the care of diabetic patients. We conducted a pragmatic cluster randomized controlled trial (RCT) of a printed educational toolkit aimed at improving CVD management in diabetes in primary care, and found no effect, and indeed, the possibility of some harm. We conducted a qualitative evaluation to study the strategy for guideline implementation employed in this trial, and to understand its effects. This paper focuses solely on the qualitative findings, as the RCT's quantitative results have already been reported elsewhere. METHODS AND FINDINGS: All family practices in the province of Ontario had been randomized to receive the educational toolkit by mail, in either the summer of 2009 (intervention arm) or the spring of 2010 (control arm).A subset of 80 family physicians (representing approximately 10% of the practices randomized and approached, with records on 1,592 randomly selected patients with diabetes at high risk for CVD) then took part in a chart audit and reflective feedback exercise related to their own practice in comparison to the guideline recommendations. They were asked to complete two forms (one pre- and one post-audit) in order to understand their awareness of the guidelines pre-trial, their expectations regarding their individual performance pre-audit, and their reflections on their audit results. In addition, individual interviews with thirteen other family physicians were conducted. Textual data from interview transcripts and written commentary from the pre- and post-audit forms underwent qualitative descriptive analysis to identify common themes and patterns. Analysis revealed four main themes: impressions of the toolkit, awareness was not the issue, 'it's not me it's my patients', and chart audit as a more effective intervention than the toolkit. Participants saw neither the toolkit content nor its dissemination strategy to be effective, indicating they perceived themselves to be aware of the guidelines pre-trial. However, their accounts also indicated that they may be struggling to prioritize CVD management in the midst of competing demands for their attention. Upon receiving their chart audit results, many participants expressed surprise that they had not performed better. They reported that the audit results would be an important motivator for behaviour change. CONCLUSIONS: The qualitative findings outlined in this paper offer important insights into why the intervention was not effective. They also demonstrate that physicians have unperceived needs relative to CVD management and that the chart audit served to identify shortcomings in their practice of which they had been hitherto unaware. The findings also indicate that new methods of intervention development and implementation should be explored. This is important given the high prevalence of diabetes worldwide; appropriate CVD management is critical to addressing the morbidity and mortality associated with the disease.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.005 | 0.068 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".