Variable Selection for Propensity Score Estimation via Balancing Covariates

To the Editor: Recently, several new approaches have been proposed for estimating propensity scores by achieving balance in the covariates. The philosophy is that by achieving balance, the bias in the estimated causal treatment effect due to measured covariates can be reduced.1 In this study, we focus on two approaches in this class: the generalized boosted model2 and the covariate balancing propensity score.3 For both approaches, the estimation depends on the covariates that we aim to balance. The traditional belief is that we should obtain balance on all the available covariates in a study.4 However, will including covariates that are not real confounders increase the variance of the causal estimator? Should we also include covariates that are related only to the treatment assignment? To investigate which set of covariates should be included in the balancing condition, we conduct a simulation study following Brookhart et al.5 We first generate three covariates, (X1, X2, X3), from a standard normal distribution. Then, the treatment indicator T is generated from a Bernoulli distribution and the outcome variable Y is generated from a Poisson model with the true treatment effect α = 0.5 (details in the eAppendix, https://links.lww.com/EDE/A868). Based on the simulation setup, X1 is the real confounder that is jointly related to the treatment and the outcome variable; X3 is related only to the treatment variable and X2 is related only to the outcome variable. We employ two approaches to estimate the treatment effect: inverse probability weighting and matching (details in the eAppendix, https://links.lww.com/EDE/A868). We generate 1,000 datasets with n = 500 and n = 2,500. We record the bias, variance, and mean squared error of the estimated treatment effect, . The results for covariate balancing propensity scores are displayed in the Table and the results for generalized boosted model are displayed in eTable 1 in the eAppendix (https://links.lww.com/EDE/A868). In both the tables, the reference model for estimating the propensity scores is the probit model with X1 and X2 as the covariates. We choose this model because Brookhart et al.5 found that this model leads to the smallest variances and mean squared errors among all possible probit models for estimating the propensity scores.TABLE: Simulation Results for the Estimated Treatment Effect; Propensity Scores Estimated by Covariate Balancing Propensity ScoresFrom the simulation results, we find that the best propensity score is the model with X1 and X2 in the balancing conditions. Placing an additional balancing condition on X3 leads to increased variance and mean squared error. For inverse probability weighting and matching estimators, it also increases the bias of the causal estimates in most cases. Compared with covariate balancing propensity scores, generalized boosted model has larger biases but smaller variances and smaller mean squared errors in general. This set of simulations has certain limitations because there are only three covariates in the setup. In practice, to make sure there are no unmeasured confounders, researchers usually collect information on a large number of covariates. Generalized boosted model tends to have superior performance when there are a large number of covariates because it can automatically perform variable selection without specifying a parametric model.6 In summary, the simulation results indicate that for both approaches, we should aim to achieve balance on real confounders, as well as covariates that are related to the outcome variable. Finally, this study is also in line with Austin et al.4 and Stuart et al.7 The former compares several propensity score models by evaluating the models’ ability to balance all available covariates in the study. The latter compares balance statistics in terms of removing bias. However, the focus of this study was to investigate which set of covariates should be included in the above-mentioned evaluation procedures. Yeying Zhu Maya Schonbach Department of Statistics and Actuarial Science University of Waterloo Waterloo, ON, Canada [email protected] Donna L. Coffman The Methodology Center Pennsylvania State University University Park, PA Jennifer S. Williams The Center for Childhood Obesity Research Pennsylvania State University University Park, PA

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.