Causal Inference with Neural Network Predictions

The estimation of Average Treatment Effect (ATE) as a causal parameter involves modeling treatment and outcome, incorporating potential confounders, and inserting the resulting predictions into ATE estimators such as the Inverse Probability Weighting (IPW) estimator or Augmented Inverse Probability Weighting (AIPW) estimator. Due to the concerns regarding the nonlinear or unknown relationships between confounders and the treatment and outcome, there has been an interest in applying non-parametric methods such as Machine Learning (ML) algorithms instead. Neural Networks (NNs) are a class of complex ML algorithms that can be applied in almost all scenarios even if the confounders are of text or image forms. NNs converge at a slower rate than the $\sqrt{n}$ of the parametric models. In addition, in the scenarios where we have an empirical violation of the positivity assumption, that is propensity scores are too close to zero or one, the ATE estimators including IPW and AIPW will have high variance. As the first proposed remedy for such situations, we introduce a normalized version of AIPW (nAIPW) which is a consistent estimator of ATE and is asymptotically normal. We perform scenario analysis and simulations to illustrate the superiority of nAIPW over AIPW in scenarios with empirical violation of the positivity assumption. However, the outcome and treatment predictions inserted in these estimators are from 2 separate NNs for the outcome and treatment, referred to as the double NN (dNN). The NN architectures do not specifically target the confounders nor can they dampen strong effects to avoid the empirical violation of positivity. For this purpose, we propose a joint NN (jNN) architecture in which the output layer has both the treatment and outcome. Additionally, we introduce an extra ``targeted" $L_1$ regularization for dampening extreme propensity values. Simulations demonstrate the superiority of jNN and dNN with the targeted regularization over dNN without the targeted regularization. Having the option of inserting the predictions of two sets of NN architectures with many hyperparameter settings comes with a disadvantage. We must use cross-validation to either select one scenario and one set of predictions or perform super-learning. We explore an alternative approach of using a recently introduced estimator called the Multiple Robust (MR) estimator. In MR, we do not need to select only one set of predictions for the outcome and treatment, and we can use all the predictions from the trained models. We prove the consistency of MR and numerically explore its performance when we change the number of outcomes and treatment predictions. We propose a general estimating equation framework that generates the MR estimator as a particular case. We show that MR is consistent if either of the treatment or outcome models is consistent. In addition, unlike the previously introduced estimating equation in the literature, we can derive an asymptotic variance estimator which does not need to select a single set of first-step predictions.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.