Evaluation of the implementation of an electronic occurrence reporting system at Eastern Health, Newfoundland and Labrador (Phase One)
Bibliographic record
Abstract
In June 2008, Eastern Health completed the implementation of an electronic occurrence reporting system (Phase One). Phase One included a pre-go-live site (an integrated services site in a rural setting) and three go-live sites (acute care, long term care and community health in an urban setting). The evaluation study had a dual purpose: (a) to assess and report on the impact of the implementation of the electronic occurrence reporting system on achieving its stated objectives, particularly those that could be measured within the timelines of the project and (b) to analyze findings to identify contributions to the literature in the recently developing field of implementations of electronic occurrence reporting systems in health care. -- The evaluation was guided by the framework outlined in the report, "Towards an Evaluation Framework for Electronic Health Records Initiatives" (Neville et al., 2004), which emphasizes stakeholder involvement in evaluation studies, pre/post comparative study design, and triangulation of data where possible. Data were collected from several sources such as project documentation, administrative occurrence reporting records, surveys, focus groups and key informant interviews. -- The findings of this study provide evidence that frontline staff and managers support the implementation of the electronic occurrence reporting system, that there is little difference in results between the various sectors of the continuum of health services and the new system had both positive and negative impacts on the role of frontline managers. There were limitations related to some of the findings due to the small sample size, particularly the long term care sector. -- Many benefits were realized such as: (a) an increase in the number of occurrences reported, (b) increase in the number of occurrences reported within 48 hours, (c) increase in the number of occurrences reported by staff other than registered nurses, (d) increase in the number of close calls reported, (e) positive changes in the patient safety culture, (f) improved timelines for notification of high alert occurrences to managers, and (g) satisfaction with the electronic tool related to ease of use, accessibility, and consistency. -- The implementation process also encountered challenges, such as issues related to customizing the software and development of the classification system for coding occurrences. These issues impacted on the ability of the managers to obtain timely customized reports and to close out files. These challenges are currently being addressed by the Project Implementation Team. Participants noted that resolving these issues will enhance the many positive impacts of the system already realized. Lessons learned during the Phase One implementation process (including the identification of facilitators and barriers) resulted in recommendations that can assist with future implementations.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.044 | 0.033 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.003 | 0.002 |
| Scholarly communication | 0.003 | 0.002 |
| Open science | 0.003 | 0.004 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".