MétaCan
Menu
← Back to cohort

605 The Systemic Lupus Erythematosus International Collaborating Clinics (SLICC), American College of Rheumatology (ACR), and Lupus Foundation of America (LFA) Damage Index Revision – Item Generation Phase

2022· article· en· W4313532424 on OpenAlexaffabout
Burak Kundakci, Ann E. Clarke, Sindhu R. Johnson, Hermine I. Brunner, Jiacai Cho, N. Costedoat‐Chalumeau, Ellen M. Ginzler, John G. Hanly, Abida Hasan, Murat İnanç, Naureen Kabani, Kaitlin Lima, Lívia Lindoso, Anselm Mak, Rosalind Ramsey- Goldman, Guillermo Ruiz‐Irastorza, Clóvis A. Silva, Farah Tamirou, Vitor Trindade, Évelyne Vinet, Ian N Bruce, Megan R.W. Barber

Bibliographic record

Venuenot available
Typearticle
Languageen
FieldMedicine
TopicSystemic Lupus Erythematosus Research
Canadian institutionsQueen Elizabeth II Health Sciences CentreUniversity of TorontoDalhousie UniversityToronto Western HospitalMount Sinai HospitalMcGill UniversityUniversity of Calgary
FundersNational Institutes of Health
KeywordsSystemic lupus erythematosusMedicineConstruct (python library)DelphiDelphi methodConstruct validityInternal medicineComputer scienceClinical psychologyArtificial intelligenceDiseasePsychometrics

Abstract

fetched live from OpenAlex

Background The SLICC, ACR and LFA embarked on a data- and expert-driven project to develop a revised systemic lupus erythematosus (SLE) organ damage index (SDI). The methodological approach includes 5 phases: updating the construct of damage (I), item generation (II), item reduction (III), item weighting and threshold determination (IV), and the assessment of validation and reliability (V). In phase I, a consensus statement was developed to define the construct of damage in SLE1. In the Item Generation phase, we aimed to develop and agree on a candidate list of items that reflect the construct of damage in SLE and are appropriate to be included in a new damage index including consideration of relevant items from adult, paediatric and young adult SLE. In this analysis, we compare the two approaches to initial item generation that were employed in a parallel process, namely a literature review and a Delphi exercise. Methods Item generation included a literature review and 3-part Delphi exercise. A group of lupus experts conducted a literature review to identify items that reflect the construct of damage in SLE and grouped the items into organ domains. Each domain was reviewed by paediatric rheumatologists. Snow-ball sampling was used among SLICC members, asking them to nominate 3-4 SLE experts considering a range of clinical expertise, equality, diversity and inclusiveness factors, and the global nature of SLE research. The LFA, Lupus UK, Lupus Europe and Lupus Canada were also asked to nominate 4-6 patient/carer representatives to participate in the Delphi exercise. Participants were asked to nominate items that should be included in a revised damage index based on the updated construct definition1 using a free-text option in Delphi exercise. Results We established a group of 146 individuals (mean age 50.6 ranging from 28 to 79 years; 60.3% females; 58.9% white; clinical experience from 1 to 51 years) from 35 countries, broadly representative of the lupus research and patient community. There were 135 medical doctors, 2 allied health professionals and 9 patients. Of 135 medical doctors, 120 were rheumatologists, 7 internists, 5 nephrologists, 2 dermatologists, and 1 immunologist. The response rate after the first round Delphi exercise was 97.9%. All items in the original SDI were nominated in both processes. Item generation yielded approximately 2,600 items. After rationalising for repetition, redundancy, and harmonisation of synonyms, 220 unique items were identified across 14 organ systems. The literature review proposed 4 (1.8%) unique items, 103 (46.8%) unique items were from the Delphi only and 113 (51.4%) items appeared in both exercises (figure 1). Conclusion Using a combined data-driven and expert/patient-based approach, items and domains that comprise damage in SLE have been expanded. Just over half of all items were nominated by both approaches. However, the Delphi exercise which included a wide and diverse group of contributors, provided a large number of unique items for further consideration. Our data confirms the value of large group exercises early in such a process to maximise the scope of new items to consider for a revised index. Reference Johnson, S. R et al. Evaluating the construct of damage in SLE. Arthritis Care Res. 2021. Lay Summary The SLICC/ACR Damage Index (SDI) (published in 1996) is widely used in clinical studies and trials to measure the long-term complications that can occur in lupus patients, such as cataracts, fractures, and kidney failure. Higher scores are associated with poorer quality of life, as reported by patients. A number of drawbacks have also been found with the SDI. We need to better understand and measure the impact of these complications from a patient and doctor’s perspective to get a much deeper understanding of how SLE affects people. We used two methods to generate new items to include in an updated SDI. First, we used the medical literature to identify possible complications of lupus. Then, we asked a large group of lupus experts and patients to nominate complications. The process generated approximately 2,600 items. After removing redundant suggestions, 220 unique items were identified. The literature review proposed 4 (1.8%) unique items, 103 (46.8%) unique items were from the large group only and 113 new (51.4%) items appeared in both exercises. Our data shows the value of large group exercises that include patient representatives, to maximise the scope of new items to consider for a revised index.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.

How this classification was reachedexpand

Full frame machine prediction

Teacher imitation

Not calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.

metaresearch head score (Codex)0.048
metaresearch head score (Gemma)0.071
Version: metacan-v3-hybrid-931329e0061cValidation status: machine_predicted_unvalidated
Candidate categoriesnone
Consensus categoriesnone
DomainCandidate signal: none · Consensus signal: none
Study designCandidate signal: Qualitative · Consensus signal: none
GenreCandidate signal: Methods · Consensus signal: none
Teacher disagreement score0.048
Threshold uncertainty score0.255

Distilled classifier scores by category (both heads)

CategoryCodexGemma
Metaresearch0.0480.071
Meta-epidemiology (narrow)0.0010.001
Meta-epidemiology (broad)0.0010.001
Bibliometrics0.0040.004
Science and technology studies0.0010.001
Scholarly communication0.0020.001
Open science0.0010.003
Research integrity0.0010.001
Insufficient payload (model declined to judge)0.0330.006

Machine scores (provisional)

The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.

Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.

Opus teacher head0.025
GPT teacher head0.341
Teacher spread0.315 · how far apart the two teachers sit on this one work
Validation statusscore_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it

Classification

machine, unvalidated

Machine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.

The models applied no category: nothing in the taxonomy fit this work.
Study designQualitative
Domainnot available
GenreMethods

How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".

Quick stats

Citations0
Published2022
Admission routes2
Has abstractyes

Explore more

Same topicSystemic Lupus Erythematosus Research→French-language works237,207→