Visualizing Benefits: Evaluating Healthcare Information System Using IS-Impact Model
Bibliographic record
Abstract
Reducing costs and optimizing operations are major challenges in many large-scale organizations including healthcare authorities. Research shows that despite ongoing investments in healthcare information systems (IS), the promised benefits often are partially realized or not at all. Part of the solution is to ensure that evaluation methodologies are available to clearly identify the success of these initiatives and from here articulate and mitigate the deficiencies or move to an alternative technology. The literature asserts that few practitioners have implemented a standardized evaluation approach. Using an established model, namely Information System Impact (IS-impact) model, we propose a modified evaluation model to assess and a visualization tool to visualize the success of an information systems from a healthcare perspective. The modified IS-Impact model includes six constructs - - individual impact, organization impact, provincial alignment impact, system quality, information quality, and service quality. We applied the modified IS-impact model and the proposed visualization tool against an existing healthcare software solution. An empirical study was conducted at a healthcare authority, with responses from 150 participants who use the healthcare IS, which confirmed that the proposed model and the visualization tool are valid and reliable to measure healthcare systems success. The evaluation model and the visualization tool are found to be efficient to narrow down the scope of inquiry from the general to the specific and quickly identifying the gaps and successes within the established software solution for the healthcare authority. Healthcare or clinical informatics researchers will be benefited from this research in evaluating the ongoing or nearly established healthcare information systems.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.007 | 0.025 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.001 |
| Bibliometrics | 0.006 | 0.004 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.004 | 0.005 |
| Open science | 0.001 | 0.002 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.005 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".