Variation in responsiveness to warranted behaviour change among NHS clinicians: novel implementation of change detection methods in longitudinal prescribing data
Bibliographic record
Abstract
OBJECTIVES: To determine how clinicians vary in their response to new guidance on existing or new interventions, by measuring the timing and magnitude of change at healthcare institutions. DESIGN: Automated change detection in longitudinal prescribing data. SETTING: Prescribing data in English primary care. PARTICIPANTS: English general practices. MAIN OUTCOME MEASURES: In each practice the following were measured: the timing of the largest changes, steepness of the change slope (change in proportion per month), and magnitude of the change for two example time series (expiry of the Cerazette patent in 2012, leading to cheaper generic desogestrel alternatives becoming available; and a change in antibiotic prescribing guidelines after 2014, favouring nitrofurantoin over trimethoprim for uncomplicated urinary tract infection (UTI)). RESULTS: Substantial heterogeneity was found between institutions in both timing and steepness of change. The range of time delay before a change was implemented was large (interquartile range 2-14 months (median 8) for Cerazette, and 5-29 months (18) for UTI). Substantial heterogeneity was also seen in slope following a detected change (interquartile range 2-28% absolute reduction per month (median 9%) for Cerazette, and 1-8% (2%) for UTI). When changes were implemented, the magnitude of change showed substantially less heterogeneity (interquartile range 44-85% (median 66%) for Cerazette and 28-47% (38%) for UTI). CONCLUSIONS: Substantial variation was observed in the speed with which individual NHS general practices responded to warranted changes in clinical practice. Changes in prescribing behaviour were detected automatically and robustly. Detection of structural breaks using indicator saturation methods opens up new opportunities to improve patient care through audit and feedback by moving away from cross sectional analyses, and automatically identifying institutions that respond rapidly, or slowly, to warranted changes in clinical practice.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.105 | 0.327 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.002 | 0.003 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.003 | 0.003 |
| Open science | 0.002 | 0.002 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".