Improving Triage Accuracy of Unclear Rheumatology Referrals: A Quality Improvement Study
Bibliographic record
Abstract
Objectives Patients with early inflammatory arthritis (EIA) need to be seen urgently to initiate treatment. Our community rheumatology clinic in Ontario, Canada was concerned that EIA cases may be delayed unnecessarily if referrals lacked sufficient detail to triage accurately. In a prior quality improvement project, we redesigned our triage process to include a patient survey (the “EIA Tool”), which was validated to identify referrals with EIA. In this study, we aimed to evaluate the sensitivity and specificity of the new triage process for referrals with unclear urgency after 12 months of use. Methods All referrals accepted by 1 rheumatologist were included from April 2020-July 2022. During the intervention period, we implemented the new triage process. The rheumatologist triaged all referrals as urgent, non-urgent, or unclear. Patients with unclear urgency were asked to complete the online EIA Tool prior to scheduling (Figure). Their survey result determined a triage score of urgent or non-urgent, and consultations were scheduled accordingly. Post-consultation, the rheumatologist determined the ‘true’ urgency score, while blinded to the pre-consultation score. Data were collected prospectively on all incoming referrals. We analyzed the data using descriptive statistics and calculated the sensitivity and specificity of the baseline and new triage processes. Results The 16-month baseline period (April 2020 to July 2021) included 1296 referrals; 647 (50%) were triaged as urgent. The 12-month intervention period (August 2021-July 2022), included 888 referrals; 508 (57%) were triaged as urgent, and 97 (11%) were triaged as unclear. The EIA Tool was completed in all unclear cases; 93 patients submitted the survey online, and 4 patients without email completed the survey by phone. Most patients (86%) completed the survey within 1 day of receiving it. Unclear cases had a cycle time from referral to scheduling of 5 days, compared to 3 days for those who were not sent the EIA Tool. The sensitivity to identify urgent cases was 97% during the intervention versus 85% at baseline. The specificity during the intervention was 59% versus 70% at baseline. Conclusion The EIA Tool helped us detect 97% of truly urgent cases, thereby reducing the risk of delayed treatment caused by triage error. We have since spread this process to 4 rheumatologists in our clinic. Our next step is to analyze urgent referral volume using statistical process control charts, in order to modify our scheduling algorithm. Supported by a CIORA grant
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.005 | 0.003 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".