Clinical Performance Evaluation of an Artificial Intelligence–Based Tool for Predicting the Presence of Obstructive Coronary Artery Disease: Protocol for a Cohort Observational Study
Notice bibliographique
Résumé
Background A significant number of individuals undergoing coronary computed tomography angiography (CCTA) for suspected (CAD) have nonobstructive or no CAD. There is a need for clinically proven models that can predict the pretest probability of stable CAD and help to identify low-risk individuals. Optimizing patient stratification is of paramount importance to improve diagnostic yield and cost-effectiveness. Objective We aimed to determine whether each patient needs to undergo CCTA because of suspected CAD. The main objective of this study is to evaluate the clinical performance of an artificial intelligence (AI)-based tool in predicting significant coronary artery stenosis (>50%), as well as its utility by medical professionals. Methods Data for this study have been acquired from 750 participants as part of routine clinical practice in AHEPA (American Hellenic Educational Progressive Association) General Hospital of Thessaloniki. The dataset has several features, including demographics (eg, age, gender), medical history (eg, diabetes mellitus, arterial hypertension), and clinical variables (eg, creatinine, epicardial fat volume). At least 2 expert cardiologists and 2 expert radiologists are involved in this study, who provide the ground truth. A trained AI-based model embedded in an easy-to-use and user-friendly web application is implemented in practice. Several AI algorithms are being examined, and the model found to perform best so far is the Optimized Voting model, which is a combination of the best performing iterations of random forest and extreme gradient boosting. The performance metrics that are being used are accuracy, precision, recall, F1-score, area under the receiver operating characteristic curve, and area under the precision-recall curve. Results Recruitment for this study began in July 2023. Data collection, development, training, and deployment of the AI web tool were completed by May 2024. In total, data from 500 individuals were collected for training and internal validation, while the best performing model was validated externally in another 250 individuals. For training and internal validation, the dataset was split into 70% for training and 20% for validation and 10% for testing. Currently, the best performing model achieves an accuracy of approximately 82% in successfully predicting stenosis greater than 50%. Additionally, an explainable AI algorithm is used to provide explanations in relation to the decisions made aiming to increase the trust of the clinicians in the tool. Conclusions The proposed study represents a novel approach of a web-based AI-driven solution with explainability features for optimizing patient stratification with the goal of improving diagnostic yield and cost-effectiveness of CCTA utilization within the context of cardiology clinical practice. International Registered Report Identifier (IRRID) DERR1-10.2196/67697
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Comment cette classification a été obtenuedéplier
Prédiction machine sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Le volet Gemma est une étiquette directe du modèle pour chaque travail de la base, lue sur la notice réduite au titre. Le volet Codex est un classifieur appris des 10 348 étiquettes directes de Codex et calibré sur les taux pondérés de l'échantillon; les champs sans appui suffisant ne portent aucun appel Codex. Le mode candidate est l'union des deux volets; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont pas des étiquettes humaines.
Scores du classifieur distillé par catégorie (deux têtes)
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,045 | 0,040 |
| Méta-épidémiologie (sens strict) | 0,002 | 0,001 |
| Méta-épidémiologie (sens large) | 0,004 | 0,005 |
| Bibliométrie | 0,002 | 0,002 |
| Études des sciences et des technologies | 0,002 | 0,002 |
| Communication savante | 0,002 | 0,001 |
| Science ouverte | 0,002 | 0,002 |
| Intégrité de la recherche | 0,002 | 0,003 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,010 | 0,003 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découleClassification
machine, non validéePrédiction automatique; un appel candidat d’une seule source (Gemma direct ou Codex distillé), pas un consensus.
Le détail, modèle par modèle et score par score, se trouve en fin de page sous « Comment cette classification a été obtenue ».