Clinical Informatics Consult Service Positively Affects Some Clinical Decisions in the ICU
Bibliographic record
Abstract
A Review of: Mulvaney, Shelagh A., Leonard Bickman, Nunzia B. Giuse, Warren E. Lambert, Nila A. Sathe, and Rebecca N. Jerome." A Randomized Effectiveness Trial of a Clinical Informatics Consult Service: Impact on Evidence-based Decision-making and Knowledge Implementation." Journal of the American Medical Informatics Association 15.2 (2008): 203-11. Objective – To determine whether the provision of synthesized research evidence provided by the Clinical Informatics Consult Service (CICS) affects the clinical decision-making of clinicians working in intensive care units (ICUs). Design – Non-blinded randomized control effectiveness trial. Setting – ICUs in United States-based 658 bed university hospital providing tertiary care for adults and children. Subjects – Clinical staff working within one of four ICUs who submitted a request for clinical information during the study period. Methods – Valid requests submitted by clinical staff from the four clinical ICUs (medical, paediatric, trauma, or neonatal) were randomly allocated to receive information from the CICS (CICS provided) or no information (no CICS provided). Pre-consult forms, completed at the time of the request, examined reasons for the request and the clinical actions clinicians thought might be influenced by the search results. Requestors could opt out of the no CICS provided group either before or after the randomization of their request. Responses to requests, supplied within 0.5 to 7 days as agreed with the requestor, included a search strategy and bibliographic references, a targeted list of full-text articles, and a written synthesis and critique of the relevant research. Clinicians within both groups were free to conduct their own searches and reviews. An online evaluation form, emailed to recipients, was used to assess the impact of the information supplied. The evaluation form asked clinicians to record the time spent on their own searches, sources of information consulted including colleagues, the immediate and future impact of the information provided (either from the CICS or their own searches), what influence the information had on their clinical actions, whether there were any barriers to using the information, and quality and overall satisfaction with the results provided by the CICS. Data was analyzed according to the randomized group assignment using standard intention-to-treat analysis for the main outcomes between the two groups. Statistical adjustments were made to control for possible clustering of responses or multiple ratings from individual clinicians. The data was also analyzed on an efficacy basis depending on who provided the search results. The groups were Clinician only, CICS librarian-only, or Both Clinician and CICS librarian. Results from the Clinician only search group were used as a comparison to the remaining groups. This assessment did not take account of the randomization and therefore constitutes a cohort analysis. Results were analysed by one of two methods using statistical software SAS Proc Mixed (v9) for multi-level quantitative data analysis, e.g., analysis of variance, and SPSS (v14) for all other quantitative data analysis, including descriptive statistics. Main Results – The study period was conducted over 19 months: August 2004 to March 2006. During this time, 299 valid requests were received and 226 post consult evaluation forms were returned giving a response rate of 76%. Post consult forms were returned for 108/146 of the CICS provided group and 118/153 of the no CICS provided group. The 24% of requests that had no post consult evaluation were excluded from further analysis. Statistical tests, conducted to check for potential bias relating to missing data, suggested that the missing data had little impact on the findings. Medical and neonatal ICUs accounted for the majority of completed forms (40.3% and 38.1% respectively). The majority of opt-outs (10.2% overall) were from the medical ICU. No significant difference in outcome variables was found between opt-out and other requests when tested using analysis of variance (ANOVA). Evaluation forms were completed by 89 unique clinicians and over half (49) submitted more than one request. The average requests per clinician was 2.96, SD 3.17, range 1-15, and the average number of requests per clinician who submitted more than one request was 4.57, SD 3.55, range 2-15. Total number of responses, mean, standard deviation, and Cohen’s d effect size were reported for the outcome variables based on intention to treat analysis. Results showed no significant difference between the groups on the immediate impact of the information provided, the number of articles read or the frequency with which clinicians consulted colleagues, either formally or informally. The potential future impact of the information was rated higher in the CICS group (p=
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.017 | 0.188 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.004 | 0.002 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.008 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".