Assessing the prior event rate ratio method via probabilistic bias analysis on a Bayesian network
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
BACKGROUND: Unmeasured confounders are commonplace in observational studies conducted using real-world data. Prior event rate ratio (PERR) adjustment is a technique shown to perform well in addressing such confounding. However, it has been demonstrated that, in some circumstances, the PERR method actually increases rather than decreases bias. In this work, we seek to better understand the robustness of PERR adjustment. METHODS: We begin with a Bayesian network representation of a generalized observational study, which is subject to unmeasured confounding. Previous work evaluating PERR performance used Monte Carlo simulation to calculate joint probabilities of interest within the study population. Here, we instead use a Bayesian networks framework. RESULTS: Using this streamlined analytic approach, we are able to conduct probabilistic bias analysis (PBA) using large numbers of combinations of parameters and thus obtain a comprehensive picture of PERR performance. We apply our methodology to a recent study that used the PERR in evaluating elderly-specific high-dose (HD) influenza vaccine in the US Veterans Affairs population. That study obtained an HD relative effectiveness of 25% (95% CI: 2%-43%) against influenza- and pneumonia-associated hospitalization, relative to standard-dose influenza vaccine. In this instance, we find that the PERR-adjusted result is more like to underestimate rather than to overestimate the relative effectiveness of the intervention. CONCLUSIONS: Although the PERR is a powerful tool for mitigating the effects of unmeasured confounders, it is not infallible. Here, we develop some general guidance for when a PERR approach is appropriate and when PBA is a safer option.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.007 | 0.013 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it