Identification and operationalisation of indicators to monitor successful uptake of advance care planning policies: a modified Delphi study
Bibliographic record
Abstract
BACKGROUND: In 2014, the province of Alberta, Canada implemented a province-wide policy and procedures for advance care planning (ACP) and goals of care designation (GCD) across its complex, integrated public healthcare system. This study was conducted to identify and operationalise performance indicators for ACP/GCD to monitor policy implementation success and sustainment of ACP/GCD practice change. METHODS: A systematic review and environmental scan was conducted to identify potential indicators of ACP/GCD uptake (n=132). A purposive sample of ACP/GCD stakeholders was invited to participate in a modified Delphi study to evaluate, reduce and refine these indicators through a combination of face-to-face meetings and online surveys. RESULTS: An evidence-informed Donabedian by Institute of Medicine (IOM) framework was adopted as an organising matrix for the indicators in an initial face-to-face meeting. Three online survey rounds reduced and refined the 132 indicators to 18. A final face-to-face meeting operationalised the indicators into a measurable format. Nine indicators, covering 11 of the 18 Donabedian×IOM domains, were operationalised. CONCLUSIONS: Nine ACP/GCD evidence-informed indicators mapping to 11 of 18 Donabedian×IOM domains were endorsed, and have been operationalised into an online ACP/GCD dashboard. The indicators provide a characterisation of ACP/GCD uptake that could be generalised to other healthcare settings, measuring aspects related to ACP/GCD documentation, patient satisfaction and agreement between medical orders and care received. The final nine indicators reflect the stakeholders' expressed intent to strike a balance between comprehensiveness and feasibility within a large provincial healthcare system.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.169 | 0.118 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.005 | 0.004 |
| Science and technology studies | 0.005 | 0.005 |
| Scholarly communication | 0.004 | 0.004 |
| Open science | 0.002 | 0.009 |
| Research integrity | 0.002 | 0.003 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".