Healthcare-Associated Infection Decisions of Antibiotic-Resistant Organisms: A Data Quality Review
Bibliographic record
Abstract
Background: Infection Prevention and Control (IPC) for Alberta Health Services and Covenant Health in the province of Alberta, Canada conducts surveillance for methicillin-resistant Staphylococcus aureus (MRSA) and vancomycin-resistant enterococcus (VRE) on all individuals admitted to acute-care and acute tertiary-care rehabilitation care facilities. Objective: The objective of this study was to determine the consistency and accuracy of infection decisions for MRSA and VRE. Methods: Surveillance cases of antibiotic-resistant organisms (AROs) collected using the provincial data entry surveillance platform between April 1, 2015, and March 31, 2017, across the province were reabstracted by infection control professionals and physicians using the NHSN infection definitions and compared to the original case severity decisions. Interrater agreement (Cohen’s ) and validity (sensitivity, specificity and predictive values) were calculated to compare the original and reabstracted infection decisions. Results: Collectively, 97% (87 of 90) of the IPC program staff and physicians who were initially invited re-abstracted 264 MRSA cases and 103 VRE cases within the review period. Provincially, 20% of the ARO cases reviewed (74 of 367) differed from the original infection decision. Among these 74 cases, 46 cases (34 MRSA and 12 VRE cases) changed from infection (original decision) to colonization (reabstracted decision) and 28 cases (21 MRSA and 7 VRE cases) changed from colonization to infection. The Cohen values for MRSA and VRE were 0.55 and 0.56, respectively, suggesting a moderate level of agreement for decisions made among IPC program staff. The sensitivity of the infection decision was higher with MRSA (86.5%) than for VRE (74.1%), meaning that there were more MRSA cases than VRE cases classified as infection in the original decision that remained infection following the review. Conclusions: Observed discordances on infection decisions were identified and may be attributed (1) to variations in the interpretation of the NHSN definitions, (2) to additional information that may have been available in the re-abstracted review compared to the original review, or (3) a difference in the information that was accessed to perform the original review compared to the reabstraction. This data-quality review provided an opportunity for IPC staff and physicians to become more familiar with infection definitions and such reviews will continue to be a regular process used to assess data quality within the IPC department. Funding: None Disclosures: None
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.193 | 0.537 |
| Meta-epidemiology (narrow) | 0.001 | 0.002 |
| Meta-epidemiology (broad) | 0.006 | 0.006 |
| Bibliometrics | 0.021 | 0.027 |
| Science and technology studies | 0.002 | 0.003 |
| Scholarly communication | 0.005 | 0.004 |
| Open science | 0.005 | 0.004 |
| Research integrity | 0.003 | 0.003 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".