System-wide analysis of qualitative hospital incident data: Feasibility of semi-automated content analysis to uncover insights
Bibliographic record
Abstract
BACKGROUND: Advances in technology have increased the ease of reporting hospital incidents, resulting in large amounts of qualitative descriptive data. Health services have little experience analysing these data at scale to incorporate into routine reporting. OBJECTIVE: We aimed to explore the feasibility of applying a semi-automated content analysis (SACA) tool (Leximancer™) to qualitative descriptions of system-wide hospital incidents to provide insights into safety issues at all health service levels. METHOD: Data from 1245 incidents reported across a network of hospitals in Australia were analysed using the SACA tool. Summaries were generated using a variety of techniques, including inductive and deductive approaches to extract key concepts in the data. RESULTS: The analysis was feasible and provided an actionable summary of the types of incidents reported in the data; the visual interface allowed users to explore the underlying text for a deeper understanding. Deductive analysis was utilised to explore specific areas of interest, and stratified analysis revealed more detailed concepts. The SACA tool was more efficient than manual processes; however, due to the context present in the incident descriptions, significant time, reading and subject matter expertise is still required to refine the analysis. CONCLUSION: Semi-automated tools provide an opportunity for improving patient safety culture and practices by providing rapid content analysis of vast datasets that can be customised for specific organisational contexts and deployed at scale. Further research is required to assess usefulness with system users. IMPLICATIONS: Qualitative data abound and system-wide analysis is essential to creating actionable insights.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.184 | 0.293 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.006 | 0.005 |
| Science and technology studies | 0.003 | 0.003 |
| Scholarly communication | 0.006 | 0.006 |
| Open science | 0.003 | 0.007 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.005 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".