Analytical validation of Exandra: a clinical decision support system for promoting guideline-directed therapy of type-2 diabetes in primary care – a collaborative study with experts from Diabetes Canada
Bibliographic record
Abstract
BACKGROUND: Individuals with type 2 diabetes (T2D) have a high prevalence of cardiovascular and renal comorbidities. Despite clinical practice guidelines recommending the use of cardiorenal protective medications, many people with T2D are not prescribed these medications. A clinical decision support system called Exandra was developed to provide treatment recommendations for individuals with T2D based on current clinical practice guidelines from Diabetes Canada. The current study aimed to medically validate Exandra via review by external medical experts in T2D. METHODS: Validation of Exandra took place in two phases. Test cases using simulated clinical scenarios and recommendations were generated by Exandra. In Phase 1 of the validation, reviewers evaluated whether they agreed with Exandra's recommendations with a "yes," "no," or "not sure" response. In Phase 2, reviewers were interviewed about their "no" and "not sure" responses to determine possible reasons and potential fixes to the Exandra system. The primary outcome was the precision rate of Exandra following the interviews and final adjudication of the cases. The target precision rate was 90%. RESULTS: Exandra displayed an overall precision rate of 95.5%. A large proportion of cases that were initially labeled "no" or "not sure" by reviewers were changed to "yes" following the interview phase. This was largely due to the validation using a simplified user interface compared with the complexity of the actual Exandra system, and reviewers needing clarification of how the outputs would be displayed on the Exandra platform. CONCLUSION: Exandra displayed a high level of accuracy and precision in providing guideline-directed recommendations for managing T2D and its common comorbidities. The results of this study indicate that Exandra is a promising tool for improving the management of T2D and its comorbidities.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".