Developing a customised set of evidence-based quality indicators for measuring workplace violence towards healthcare workers: a modified Delphi method
Bibliographic record
Abstract
BACKGROUND: Workplace violence (WPV) is a complex global challenge in healthcare that can only be addressed through a quality improvement initiative composed of a complex intervention. However, multiple WPV-specific quality indicators are required to effectively monitor WPV and demonstrate an intervention's impact. This study aims to determine a set of quality indicators capable of effectively monitoring WPV in healthcare. METHODS: This study used a modified Delphi process to systematically arrive at an expert consensus on relevant WPV quality indicators at a large, multisite academic health science centre in Toronto, Canada. The expert panel consisted of 30 stakeholders from the University Health Network (UHN) and its affiliates. Relevant literature-based quality indicators which had been identified through a rapid review were categorised according to the Donabedian model and presented to experts for two consecutive Delphi rounds. RESULTS: 87 distinct quality indicators identified through the rapid review process were assessed by our expert panel. The surveys received an average response rate of 83.1% in the first round and 96.7% in the second round. From the initial set of 87 quality indicators, our expert panel arrived at a consensus on 17 indicators including 7 structure, 6 process and 4 outcome indicators. A WPV dashboard was created to provide real-time data on each of these indicators. CONCLUSIONS: Using a modified Delphi methodology, a set of quality indicators validated by expert opinion was identified measuring WPV specific to UHN. The indicators identified in this study were found to be operationalisable at UHN and will provide longitudinal quality monitoring. They will inform data visualisation and dissemination tools which will impact organisational decision-making in real time.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.183 | 0.185 |
| Meta-epidemiology (narrow) | 0.002 | 0.002 |
| Meta-epidemiology (broad) | 0.002 | 0.004 |
| Bibliometrics | 0.011 | 0.005 |
| Science and technology studies | 0.003 | 0.004 |
| Scholarly communication | 0.004 | 0.004 |
| Open science | 0.003 | 0.012 |
| Research integrity | 0.003 | 0.003 |
| Insufficient payload (model declined to judge) | 0.005 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".