Propensity score weighting with survey weighted data when outcomes are binary: a simulation study
Bibliographic record
Abstract
Abstract Background Despite that propensity score methods have been widely adopted in observational studies, few studies have focused on how to properly estimate and incorporate propensity scores in complex survey data settings. Particularly, it is lacking that the research on propensity score-based weighting methods when estimating the treatment effect using complex survey data with binary outcomes. The objective of this study is therefore to compare three propensity score weighting methods for complex survey data when outcomes are binary. MethodsA simulation study was used to compare three propensity score weighting approaches for estimating treatment effects using survey data: 1) no survey weights in the propensity score model or the outcome model; 2) survey weights in the outcome model only; 3) survey weights in both models. Each of the three methods is applied in the context of four measures of the treatment effect: the sample average treatment effect (SATE), the population average treatment effect (PATE), the sample average treatment effect on the treated (SATT), and the population average treatment effect on the treated (PATT). The methods are compared in terms of mean relative bias and coverage probability under different scenarios by varying combinations of sample size and treatment effect, degrees of model misspecification and levels of overlap in propensity score. In addition, using the 2015 National Health Interview Survey (NHIS) data as a real data example, each method is employed to estimate the effect of provider-patient discussion about smoking on smoking cessation. ResultsThe methods which account for the survey weight outperform the unweighted method as the degree of misspecification increases, regardless of the sample size, treatment effect and level of overlap in propensity score. The performance of the two weighted methods, where the survey weight is incorporated in the outcome model or both the propensity score model and the outcome model, are similar. Conclusions The propensity score weighting methods accounting for the survey weight are necessary for estimating the population-level treatment effects. This paper provides guidance for selecting an appropriate propensity score weighting method based on the estimand of interest when using complex survey data with binary outcomes.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.071 | 0.239 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.003 | 0.004 |
| Bibliometrics | 0.002 | 0.004 |
| Science and technology studies | 0.001 | 0.002 |
| Scholarly communication | 0.003 | 0.005 |
| Open science | 0.004 | 0.003 |
| Research integrity | 0.004 | 0.005 |
| Insufficient payload (model declined to judge) | 0.011 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".