Measuring and Tracking the Progress of Implementing a Comprehensive Electronic Health Record: A Mixed-Methods Approach
Bibliographic record
Abstract
Background: India is entering an era of technology revolution in health care. The current literature and experiences in global settings reveals that implementation of electronic health records (EHR) is often challenging. Although, data on EHR implementation is available from several countries including UK, USA and Canada, there is little data about EHR implementation experiences in India. This body of knowledge will be useful to guide future implementations of EHRs in the Indian context. In this paper, we discuss our early experiences of implementing a comprehensive EHR at four facilities of a single private health care system in India. We use a mixed-methods approach (both quantitative and qualitative) in our assessment. Rationale and Preparation for EHR Implementation: In 2009, the leadership of Max Health Care Institute Ltd. launched their efforts to implement open source Vista. Dell Services (Perot Systems then) were brought in a technology partners and the WorldVista flavor of VistA was chosen to be integrated to Max-HIS (Max's home ground HIS). The main reasons for EHR implementation were potential reduction of medical errors, improved medication management, rapid access to vital and accurate information, reduced duplication of services and cost, access to a more comprehensive picture of health for promoting advances in the diagnosis and treatment of illnesses, improved and informed decision making, and providing continuity of care to patients. To help prepare for this transition, the leadership also recruited external consultants to facilitate implementation processes and procedures. We used WorldVistA that included the following applications: progress notes, templates, computerized provider order entry (CPOE) for medications, procedures, lab and imaging tests, Bar Code Medication Administration system (BCMA), pharmacy dispensing linked to drug orders and was capable of interfacing with Patient information management, lab, Radiology. Major organizational and workflow changes in preparation for implementation included identifying and using superusers, redefining workflows and training staff. Teams were setup to come and discuss as-is processes with the demo of the system. This facilitated the mindset change for the transition where new processes and procedures were written out of each section of the hospital. This was continuously done over workshops running daily for over 2 months. This prepared the future state. Three months before go-live classroom training sessions were setup with mock-live hardware and a curriculum to educate all users who would be affected by the system. Besides Doctors, nurses and health technicians, other operational staff, service and medical quality teams and medical records department were also trained. Superusers were chosen from the end users choosing the people who took to the system easily and also understood the processes. Settings: Max Super Specialty Hospital, Saket and 3 other new hospitals of Max Healthcare Methods: We used a mixed-method approach involving three methods in measuring and tracking the progress of EHR implementation. First, we used a quantitative approach involving the use of six “automated” outcome metrics that were easily extracted from Mumps database which is the backbone of the EHR. These metrics included: 1) Use of Progress Notes; 2) Use of CPOE for medications, procedures, lab and imaging tests; 3) Documentation of two daily inpatient progress notes (morning and evening) by a consultant on all inpatients; 4) Generating Problem lists involving selection from ICD-9 coded problems; 5) Documentation of Input and Output logs by nurses on inpatient wards; and 6) Use of BCMA by nurses. Second, we interviewed four groups of representative users: senior consultants including head of units and head of departments, junior consultants (but not resident doctors), “Floor Mentor” who is a senior person team member who can be contacted by the patient or their attendants’ in case of any problem during the stay of their stay in the hospital and the Nursing Supervisors. A total of 50 participants were interviewed. Content analysis of the interviews was conducted to identify major themes. Third, we used fact-finding questionnaires that obtained data from the same group that was interviewed. These provided objective assessment of ease of use, system functionality, and change management. Data is now becoming available 5 months after the EHR launch date (24th July 2011) and is presented briefly here. Additional data will be available in the next 3 weeks and is currently being analysed. Results: The Max Health Care Institute Ltd. implemented a comprehensive open source EHR in both inpatient and outpatient settings and underwent major organizational and workflow transformation changes to adapt. Table 1 summarizes the automated quantitative metrics, some of which show high usage patterns whereas other applications were slower to adopt. Use of CPOE is a stand-out because the organization mandated this as the only way to order in the hospital. Table 1: Automated measures to track implementation progress Implementation Metric % Use 1) Use of Progress Notes 76% 2) Use of CPOE for medications, procedures, lab and imaging tests 100% 3) Documentation of 2 daily inpatient progress notes (morning and evening) by a consultant on all inpatients 65% 4) Use of Problem lists involving selection from ICD-9 coded problems 15% 5) Documentation of Input and Output logs by nurses 82% in IPD, Critical care on parallel paper process 6) Use of BCMA by nurses Real Time MAR 41%, however including after the fact goes upt 78% Qualitative analysis showed that the younger generation of doctors was more comfortable working on the system, whereas the senior consultants were mostly hesitant to use this new technology and had low degree of adoption. Typing issues and EHR system “complexity” were raised as main concerns by several participants, but mostly from senior consultants. Because of time investment, many perceived reduction of efficiency. However, most users who had 2-3 days of hands-on EHR use immediately perceived its benefits and reported high degree of comfort in its use. Unfortunately, there was lack of support from the Super Users because they were many instances when they were off, post-duty. Some of them were genuinely busy to not be able to help others. Differences between doctors and nurses also emerged; nursing staff were responsible for the maximum use of the system. Many facilitators of the process included support received from the top management, user friendly clinical templates and easy accessibility of all Max enrolled patients’ records from any of the Max facilities located in various parts in the Delhi/NCR region and also from other locations such as Bhatinda and Mohali. The system quick response time and stability while using can also be credited with aiding its acceptance. Additional preliminary measures on tracking effectiveness of EHR implementation were instituted that showed room for improvement. For instance for CPOE, we started measuring proportion of stat medication orders that were filled appropriately and found only a 90% compliance. We also found an increase in the overall time in the discharge process when the EHR was only used partially, i.e. when both paper based and electronic records were available. This phase however is expected to be transitional. Nevertheless, EHR implementation led to the creation of certain new quality measures and quality expectations, a benefit that was well received among facility leadership. Conclusions and Implications: We report on an early experience of implementing a comprehensive EHR at four facilities of a large health care system in India. We believe that this is the largest implementation of its kind in India. Several lessons we learned about measuring and tracking the progress of implementing a comprehensive EHR could also be useful for other facilities in India considering a transition to improving care through electronic health records.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.084 | 0.005 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.002 | 0.000 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.000 | 0.002 |
| Open science | 0.002 | 0.000 |
| Research integrity | 0.001 | 0.010 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".