Rethinking Residual Confounding Bias Reduction: Why Vanilla hdPS Alone is No Longer Enough
Bibliographic record
Abstract
Health studies that use administrative databases often lack complete information on confounders. On the other hand, a large number of additional diagnoses, procedures, and medication codes that are regularly recorded in healthcare encounters, are not used in epidemiological studies, due to their perceived lack of relevance to the study question. To address this residual confounding problem, researchers have developed the high-dimensional propensity score (hdPS) algorithm. This algorithm allows researchers to leverage this additional information as proxies for unmeasured and mis-measured covariates, which can help reduce residual confounding bias in the estimation of treatment effects. Since the hdPS algorithm deals with massive amounts of information, machine learning variable selection methods are proposed as an alternative. These methods have been shown to be effective in reducing bias, but it remains a challenge to estimate variance correctly in this context. Even doubly robust or targeted maximum likelihood estimators (TMLE) can struggle with this issue. To address this problem, we designed a simulation study to compare the performance of methods of the following categories: (1) vanilla hdPS, (2) machine learning and hybrid alternatives proposed in the literature, and (3) TMLE versions with two sets of candidate learners for super learning. We will evaluate these methods in terms of bias, variance (both model-based and empirical), and coverage. We will present a nationally representative analysis as a motivating example, explain how this study fits into the literature so far, and provide practical recommendations for practitioners based on our findings.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.078 | 0.202 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.002 |
| Bibliometrics | 0.001 | 0.002 |
| Science and technology studies | 0.001 | 0.003 |
| Scholarly communication | 0.003 | 0.005 |
| Open science | 0.003 | 0.004 |
| Research integrity | 0.002 | 0.004 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".