Monitoring intensive care unit performance—impact of a novel individualised performance scorecard in critical care medicine: a mixed-methods study protocol
Bibliographic record
Abstract
INTRODUCTION: Patients admitted to a critical care medicine (CCM) environment, including an intensive care unit (ICU), are susceptible to harm and significant resource utilisation. Therefore, a strategy to optimise provider performance is required. Performance scorecards are used by institutions for the purposes of driving quality improvement. There is no widely accepted or standardised scorecard that has been used for overall CCM performance. We aim to improve quality of care, patient safety and patient/family experience in CCM practice through the utilisation of a standardised, repeatable and multidimensional performance scorecard, designed to provide a continuous review of ICU physician and nurse practice, as well as departmental metrics. METHODS AND ANALYSIS: This will be a mixed-methods, controlled before and after study to assess the impact of a CCM-specific quality scorecard. Scorecard metrics were developed through expert consensus and existing literature. The study will include 19 attending CCM physicians and approximately 300 CCM nurses. Patient data for scorecard compilation are collected daily from bedside flow sheets. Preintervention baseline data will be collected for 6 months for each participant. After this, each participant will receive their scorecard measures. Following a 3-month washout period, postintervention data will be collected for 6 months. The primary outcome will be change in performance metrics following the provision of scorecard feedback to subjects. A cost analysis will also be performed, with the purpose of comparing total ICU costs prior to implementation of the scorecard with total ICU costs following implementation of the scorecard. The qualitative portion will include interviews with participants following the intervention phase. Interviews will be analysed in order to identify recurrent themes and subthemes, for the purposes of driving scorecard improvement. ETHICS AND DISSEMINATION: This protocol has been approved by the local research ethics board. Publication of results is anticipated in 2019. If this intervention is found to improve patient- and unit-directed outcomes, with evidence of cost-effectiveness, it would support the utilisation of such a scorecard as a quality standard in CCM.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.073 | 0.042 |
| Meta-epidemiology (narrow) | 0.006 | 0.003 |
| Meta-epidemiology (broad) | 0.006 | 0.006 |
| Bibliometrics | 0.004 | 0.003 |
| Science and technology studies | 0.004 | 0.003 |
| Scholarly communication | 0.004 | 0.004 |
| Open science | 0.005 | 0.003 |
| Research integrity | 0.007 | 0.005 |
| Insufficient payload (model declined to judge) | 0.026 | 0.007 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".