System-wide analysis of qualitative hospital incident data: Feasibility of semi-automated content analysis to uncover insights
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
BACKGROUND: Advances in technology have increased the ease of reporting hospital incidents, resulting in large amounts of qualitative descriptive data. Health services have little experience analysing these data at scale to incorporate into routine reporting. OBJECTIVE: We aimed to explore the feasibility of applying a semi-automated content analysis (SACA) tool (Leximancer™) to qualitative descriptions of system-wide hospital incidents to provide insights into safety issues at all health service levels. METHOD: Data from 1245 incidents reported across a network of hospitals in Australia were analysed using the SACA tool. Summaries were generated using a variety of techniques, including inductive and deductive approaches to extract key concepts in the data. RESULTS: The analysis was feasible and provided an actionable summary of the types of incidents reported in the data; the visual interface allowed users to explore the underlying text for a deeper understanding. Deductive analysis was utilised to explore specific areas of interest, and stratified analysis revealed more detailed concepts. The SACA tool was more efficient than manual processes; however, due to the context present in the incident descriptions, significant time, reading and subject matter expertise is still required to refine the analysis. CONCLUSION: Semi-automated tools provide an opportunity for improving patient safety culture and practices by providing rapid content analysis of vast datasets that can be customised for specific organisational contexts and deployed at scale. Further research is required to assess usefulness with system users. IMPLICATIONS: Qualitative data abound and system-wide analysis is essential to creating actionable insights.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.007 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.003 | 0.005 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it