P5707An evaluation of the use of propensity scores in cardiovascular literature: a systematic review and recommendations
Bibliographic record
Abstract
Abstract Background As randomized trials cannot always be conducted, propensity score (PS) methods are frequently used in cardiovascular observational studies. Previous evaluations found insufficient and inaccurate reporting of PS methods in the clinical literature, however; a comprehensive and current evaluation of their use and reporting is thus warranted. Purpose 1) To conduct a systematic review of cardiovascular articles published in high-impact medical and cardiovascular journals to evaluate the use and reporting of PS methods, as well as interpretation of results. 2) To identify key reporting and analysis elements when using PS methods. Methods Cardiovascular articles using PS published between 2010 to 2017 in high-impact medical (5) and cardiovascular (3) journals were reviewed. Included studies used a PS-based method, and focused on cardiovascular diseases, outcomes, interventions, or techniques. Information was extracted on: (1) PS assumptions, (2) variable selection and assessment of model success, (3) specifics of each PS method, and (4) consistency of written interpretation with the causal effect estimated. Each article was evaluated by 2 reviewers. Recommendations for key elements to be included in publications were based on findings and PS methodological literature. Results Of the 296 included articles, most were published in Journal of the American College of Cardiology (83 articles; 28%), followed by Circulation (79 articles; 27%), European Heart Journal (46 articles; 14%), Journal of the American Medical Association (35 articles; 12%), British Medical Journal (30 articles; 10%), New England Journal of Medicine (10 articles; 3%), Annals of Internal Medicine (10 articles; 3%), and Lancet (3 articles; <1%). The most commonly used PS method was matching (53% of studies), followed by a combination of methods (19%), direct adjustment (13%), inverse probability weighting (IPW, 12%), and stratification (3%). Variables in the PS model were predefined in 77% of articles, selected with statistical testing in 17% of articles, or both in 5% of articles (no details for 1% of articles). Balance was not assessed in 16% of papers and 38% of articles that evaluated balance did not report standardized differences, the recommended measure to assess balance. For matching, most studies (85%) conducted a 1:1 match; however, 17% of studies had >50% of unmatched treated (or untreated) patients (based on targeted parameter). Overall, heterogeneity of effects was assessed in 89% of all articles. In 48% of the reviewed articles, however, interpretations of the effect estimates did not correspond to the PS method conducted or described. Conclusions Detailed reporting of PS methods is important to maintain transparency and reproducibility, evaluate the appropriateness of the method, and correctly interpret the results and the population to which they apply. This systematic review describes areas for improvement.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.152 | 0.427 |
| Meta-epidemiology (narrow) | 0.002 | 0.002 |
| Meta-epidemiology (broad) | 0.007 | 0.011 |
| Bibliometrics | 0.023 | 0.015 |
| Science and technology studies | 0.002 | 0.003 |
| Scholarly communication | 0.007 | 0.011 |
| Open science | 0.006 | 0.004 |
| Research integrity | 0.006 | 0.004 |
| Insufficient payload (model declined to judge) | 0.010 | 0.003 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".