Evaluating the Quality of Systematic Reviews & Meta-Analyses Published on Direct Oral Anticoagulants in the Past 5 Years
Bibliographic record
Abstract
Introduction In the past decades evidence-based medicine has begun to drive clinical decision making. With direct oral anti-coagulants (DOACs) emerging as alternatives to warfarin for the treatment and prevention of thromboembolic disorders, it has become crucial that clinicians utilize unbiased and robust evidence to inform their decisions about their use. Systematic reviews (SRs) sit at the "top" of the hierarchy of research evidence - the goal of this study was to evaluate the quality of SRs published on DOACs using AMSTAR criteria. Methods A comprehensive search of Medline, EMBASE, and the Cochrane Database of Systematic Reviews from Jan 2013 to February 2019 was performed. Screening was done across two stages with title and abstract followed by full-text analysis. Any study that was a SR (with or without a meta-analysis) published on DOACs was included. Data extracted included AMSTAR rating, journal of publication, year of publication, number of studies included, reporting adherence to PRISMA guidelines, number of citations, and journal of publication impact factor. Screening and data extraction were both done by two reviewers independently in duplicate. AMSTAR evaluation was done by three reviewers, one of which was a senior author. Statistical analyses comparing AMSTAR scores in relation to the above factors were done. Results A total of 3729 articles were found with 249 being included for analysis. Quality of SRs was highly variable across years with the mean (SD) being 5.68 (2.21). [Figure 1]. There were no significant relationships between quality vs citation rate (r=-0.04; 95% CI -0.17, 0.09; p=0.26) and impact factor (r=-0.05; 95% CI -0.18, 0.08; p=0.219). One-way ANOVA revealed no significant difference of AMSTAR scores between years (F6,242 = 1.85 p=0.09) [Figure 1.]. Reporting adherence to PRISMA guidelines increased the likelihood of being moderate (AMSTAR Score = 5-8) or high-quality evidence (AMSTAR Score = 9-11) (OR = 4.159; 95% CI 2.32, 7.46, p<0.01). Studies included/excluded in reviews (17, 7%) and conflicts of interests in both the review and included studies (21, 8%) were the least reported AMSTAR criteria while characteristics of included studies (226, 90%) and appropriate use of combining findings (217, 87) were the most. [Figure 2.] Conclusions The overall quality of SRs published on DOACs was moderately low and there was no relationship between journal impact factor and quality of the reviews that journals published. Our findings highlight specific areas within which authors can improve their reporting. Reviewers and editors of journals should familiarize themselves with AMSTAR criteria to ensure robust and transparent quality reporting in an effort to increase the quality of evidence being used to guide clinical decision making. Disclosures Crowther: Diagnostica Stago: Other: preparing educational material and/or providing educational presentations, Research Funding; Bayer: Other: Data and Safety Monitoring Board, Research Funding, Speakers Bureau; BMS Canada: Membership on an entity's Board of Directors or advisory committees, Research Funding; Servier Canada: Membership on an entity's Board of Directors or advisory committees, Research Funding, Speakers Bureau; Pfizer: Other: preparing educational material and/or providing educational presentations; CSL Behring: Other: preparing educational material and/or providing educational presentations; Asahi Kasei: Membership on an entity's Board of Directors or advisory committees; Octapharma: Membership on an entity's Board of Directors or advisory committees; Shionogi: Membership on an entity's Board of Directors or advisory committees; Alexion: Speakers Bureau; Alnylam: Equity Ownership.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.234 | 0.494 |
| Meta-epidemiology (narrow) | 0.003 | 0.003 |
| Meta-epidemiology (broad) | 0.019 | 0.039 |
| Bibliometrics | 0.049 | 0.043 |
| Science and technology studies | 0.002 | 0.003 |
| Scholarly communication | 0.009 | 0.007 |
| Open science | 0.004 | 0.005 |
| Research integrity | 0.004 | 0.002 |
| Insufficient payload (model declined to judge) | 0.005 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".