VALIDATION OF AMERICAN COLLEGE OF RHEUMATOLOGY DIGITAL CLINICAL QUALITY MEASURES FOR LUPUS CARE IN THE RHEUMATOLOGY INFORMATICS SYSTEM FOR EFFECTIVENESS (RISE) REGISTRY
Notice bibliographique
Résumé
PV086 / #369 Poster Topic: AS11 - Epidemiology and Public Health Background/Purpose In collaboration with the Centers for Disease Control and Prevention (CDC), the American College of Rheumatology (ACR) recently developed digital clinical quality measures for lupus clinical care. These measures included 1) hydroxychloroquine use (yes/no), 2) limiting glucocorticoid use of doses above 7.5 mg/day of prednisone to fewer than 6 months (yes/no), and 3) kidney function and urine protein laboratory testing for monitoring/screening for lupus nephritis (yes/no). We aimed to assess the accuracy of calculating these quality measures in the ACR’s Rheumatology Informatics System for Effectiveness (RISE) registry compared to manual medical record review. Methods Five practices that participate in the RISE registry were recruited to participate in this study. All eligible patients with systemic lupus erythematosus (SLE) (defined as ≥ 2 SLE codes ≥ 30 days apart in 2022) who were at least 18 years of age and had at least 1 visit with a participating practice in 2022 were identified from the RISE registry. Among those patients, stratified random sampling was used to select 35 patients from each practice based on patients’ glucocorticoid and hydroxychloroquine use in 2022. Practice staff were asked to complete a structured medical record review for at least 25 of the 35 patients at each site. Data related to the SLE quality measures were abstracted for the measurement year 2022, including hydroxychloroquine use and its contraindications, glucocorticoid use, including dose and duration of use, end-stage kidney disease (ESKD), and laboratory testing for kidney function and urine protein excretion. Corresponding electronic health record data on these patients were extracted from the RISE registry. Cohen’s Kappa statistic and percent agreement were calculated for each measure component. Results We included 121 patients with SLE, of whom 92% were female, 21% were Black or African American, and 13% had lupus nephritis (Table 1). Agreement between the practice medical record review and RISE data varied across the measure components, with higher Kappa statistics and percent agreement for medication use than for laboratory testing (Table 2). The Kappa for hydroxychloroquine use and glucocorticoid use were 0.70 (95% CI 0.60- 0.90) and 0.90 (95% CI 0.82-0.98), respectively. Glucocorticoid use over 7.5 mg/day for longer than 6 months was infrequent, with 97% agreement, but lower kappa 0.48 (95% CI (0.05-0.92). One patient was identified with ESKD, with 100% agreement between data sources, and was excluded from the kidney monitoring measure. Kidney function measurement was reported at least once in 2022 for 93% per medical record review and 83% of patients per RISE data, with Kappa 0.27 (95% CI 0.04-0.50). Urine protein measurement was reported at least once in 2022 for 72% per medical record review and 49% of patients per RISE data, with Kappa 0.35 (95% CI 0.21-0.50). Table 1. Characteristics of Patients with Systemic Lupus Erythematosus Table 2. Agreement between RISE data and Medical Record Review Conclusions In this initial validation study of digital clinical quality measures for lupus applied to 5 rheumatology practices in the RISE registry, we found good agreement for hydroxychloroquine and glucocorticoid use between medical record review and RISE data. Discrepancies in glucocorticoid dosing rarely led to misclassification for the glucocorticoid measure. Agreement was lower for the capture of kidney monitoring tests. Further work will include additional measure validity testing between RISE and Medicare data and will examine strategies to more accurately capture kidney monitoring for patients with lupus.
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Comment cette classification a été obtenuedéplier
Prédiction machine sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Le volet Gemma est une étiquette directe du modèle pour chaque travail de la base, lue sur la notice réduite au titre. Le volet Codex est un classifieur appris des 10 348 étiquettes directes de Codex et calibré sur les taux pondérés de l'échantillon; les champs sans appui suffisant ne portent aucun appel Codex. Le mode candidate est l'union des deux volets; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont pas des étiquettes humaines.
Scores du classifieur distillé par catégorie (deux têtes)
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,136 | 0,249 |
| Méta-épidémiologie (sens strict) | 0,001 | 0,001 |
| Méta-épidémiologie (sens large) | 0,001 | 0,001 |
| Bibliométrie | 0,004 | 0,005 |
| Études des sciences et des technologies | 0,001 | 0,001 |
| Communication savante | 0,003 | 0,002 |
| Science ouverte | 0,002 | 0,003 |
| Intégrité de la recherche | 0,001 | 0,001 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,002 | 0,001 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découleClassification
machine, non validéePrédiction automatique; un appel candidat d’une seule source (Gemma direct ou Codex distillé), pas un consensus.
Le détail, modèle par modèle et score par score, se trouve en fin de page sous « Comment cette classification a été obtenue ».