Commentary

Johnson et al1 use probabilistic bias analysis to funnel uncertainty about exposure misclassification onto the odds ratio. They do this to quantify the effects of bias on the analysis results. But it may be a useful reminder that the full power of probabilistic bias analysis is unleashed when uncertainty about bias parameters is combined with uncertainty due to random error. In the NHANES study, and indeed throughout their paper, the data are analyzed as though random error does not exist. Certainly there is a logic to this approach because it simplifies the investigation of bias. Furthermore, the sample sizes are large and random error is perhaps less important. However, obesity and diabetes are strongly correlated. In practice, we encounter smaller associations and low statistical power, and quantifying random error is essential. It is also important to note that systematic and random error do not act independently on the log odds ratio (OR). When the measurement error is large, the bias-corrected standard error can explode. For example, Greenland and Gustafson2 showed that adjusting for nondifferential misclassification can increase the width of the 95% confidence interval for the log odds ratio. Thus it is difficult to understand the impact of random error without incorporating it into a full probabilistic bias analysis. A full probabilistic bias analysis that incorporates random error and systematic error proceeds as follows: First we sample bias parameters from their assigned probability distribution and calculate the bias-corrected log OR and standard error SE(log OR). Lash, Fox, and Fink3(Ch. 6) give formulas for the bias-corrected standard error when there is exposure measurement error. Second, we sample a single log odds ratio from a normal distribution with mean equal to logOR and standard deviation SE(log OR). This process is iterated repeatedly to obtain a large sample of log odds ratios. We can summarize the results using the sample median, with 95% interval based on the 2.5% and 97.5% percentiles of the distribution. The beauty of this procedure is that the interval estimate combines uncertainty from bias and random error simultaneously. Our view is that such intervals represent a new and exciting frontier for epidemiologic research using large datasets. Gustafson and Greenland4 demonstrate that these interval estimates will have good frequentist coverage of the true parameter value provided that the probabilistic bias analysis assigns relatively high prior probability to the actual values of the bias parameters. In contrast, the 95% simulation intervals described by Johnson et al1 are not confidence intervals because they do not incorporate random error. Of course specifying a probability distribution for bias parameters is critical to probabilistic bias analysis. Our view is that this is best conceptualized as a prior distribution in a Bayesian analysis. The prior distribution is a model for the investigators’ beliefs about bias before having analyzed the data. When formulating a prior, it is important to specify both the location of the distribution, and also the width and shape. In particular, the width conveys the magnitude of uncertainty about bias. In their NHANES data example, Johnson et al1 identified 5 published studies containing estimates of the sensitivity (Se) and specificity (Sp). However, they ignored the published standard errors in favor of a trapezoidal distribution with mean 90% and limits 85% to 95%. The Table gives sensitivity estimates, with standard errors, as abstracted from the 5 studies. A DerSimonian-Laird5 random effects meta-analysis indicates heterogeneity, with an estimated pooled sensitivity of 85%. In formulating our beliefs about the sensitivity of self-reported obesity in NHANES, then, we could place 95% of the probability mass on logit(Se) straddled around logit(0.85) plus/minus twice the between-study standard deviation. This translates to the interval from 66% to 94% for sensitivity. Alternatively, the evidence from the literature could be synthesized in other ways, but we simply point out that the priors for sensitivity and specificity are perhaps too narrow in light of the heterogeneity among the 5 studies.TABLE: Estimated Sensitivities of Self-reported Obesity, with Uncertainty Assessments, in the 5 Studies Identified by Johnson et al1Building on the theme of choosing prior distributions, Johnson et al1 illustrate adjustment for exposure misclassification under several assumptions, including 2 they refer to as the “exactly nondifferential” assumption and the “approximately nondifferential” assumption. Both invoke bivariate prior distributions with identical marginal distributions. For instance, consider the sensitivity of the exposure assessment. In their NHANES example, Johnson et al1 assign the symmetric triangular distribution running from 0.85 to 0.95 to both control sensitivity and case sensitivity. But the exactly nondifferential assumption constrains the 2 sensitivities to be equal, whereas the approximately nondifferential assumption takes them to be a priori independent of one another. Viewed this way, there is clearly a gap between the exactly and approximately nondifferential assumption—the prior correlation between the 2 sensitivities (and between the 2 specificities) could be set between 0 and 1 instead. In fact, bivariate priors with high correlations have been used in such settings.10,11 Chu et al12 refer to this application of bivariate priors with identical marginals and high positive correlations as invoking a nearly nondifferential assumption. Clearly then, the nearly nondifferential assumption sits between the exactly and approximately nondifferential assumption.13 Arguably the nearly nondifferential assumption would be appropriate in many applications. Context-specific clues to the direction of nondifferentiality (does classification tend to be better for cases than controls, or vice-versa) will often be absent. This speaks to being somewhere on the exactly-nondifferential assumption to approximately-nondifferential assumption spectrum (technically speaking, wanting the same marginal distributions for case probabilities as for controls, or even more technically speaking wanting exchangeability). However, the exactly-nondifferential assumption is a very strong assumption, ruling out subtle and unanticipated influences of disease status on the exposure assessment, whereas the “approximately nondifferential” assumption is quite weak, literally saying that investigators would not revise their beliefs about the case exposure-classification probabilities if the corresponding probabilities for controls were revealed to them. This seems inappropriate if there is no understood mechanism impelling the 2 sets of probabilities to differ. The nearly nondifferential assumption feels about right for many problems, then. It directly asserts that any deviations of the case exposure-classification probabilities from their control counterparts are likely to be small. In implementing their probabilistic bias analysis, Johnson et al1 remark that if a randomly drawn (Se, Sp) pair lead to a negative exposure-disease odds ratio, then this pair is discarded. Lurking here is an issue that has been discussed in the literature.14,15 In fact, bumping into a nonsensical OR can be more than a fluke. For instance, mimicking the context of the simulation work by Johnson et al,1 say the true exposure prevalence in the control population is 10%, and say that uncertainty about both the control sensitivity and the control specificity is encapsulated by the symmetric triangular distribution running from 0.85 to 0.95. Further, for sake of illustration, say the actual control sensitivity and specificity happen to be 0.87 and 0.93, respectively, in line with the asserted prior distribution. The prevalence of apparent exposure in the control population then transpires to be 15% [computed as (0.1) (0.87) + (0.9) (1−0.93)]. And, due to sampling variation, the sample prevalence could easily be a bit lower. For instance, the proportion classified as exposed among 400 controls could easily be 13% (slightly more than 1 standard deviation below the population proportion). Such data would then induce a 95% confidence interval of 13.0% ± 3.4% for the prevalence of apparent exposure in the control population. Here we pause to consider the compatibility of the data and the prior distribution. The prevalence of apparent exposure is inherently constrained to exceed 1-Sp, with violations of this constraint triggering a “negative” (or, more properly, undefined) odds ratio. So, the message from the data (prevalence of apparent exposure is 13.0% ± 3.4%) does partially conflict with the prior distribution (1-Sp lies lies between 0.05 and 0.15). To be more explicit, consider jointly simulating a value of apparent exposure prevalence from a normal distribution with a mean of 0.130 and standard deviation of 0.017, along with a value of specificity from its prior distribution. The chance of getting a pair of values violating the constraint is computed to be 0.13. What should we make of this? First, the good news. In such a scenario we gain extra information about specificity. A Bayesian analysis would yield a posterior distribution on specificity that is a bit more concentrated than the prior, with downweighting of smaller values. The bad news is that applying probabilistic bias analysis is more dubious in this circumstance. The strategy of drawing specificity values from the prior distribution and treating them all equally unless the OR is undefined is not likely to mimic the more principled Bayesian synthesis of evidence. For many specific problems, datasets and prior distributions, probabilistic bias analysis and Bayesian analysis will indeed agree closely, with the former being easier to implement. However, when an appreciable proportion of sampled bias parameters are “invalid,” then there is a greater imperative to go the Bayesian route. ABOUT THE AUTHORS PAUL GUSTAFSON is a Professor in the Department of Statistics at the University of British Columbia. LAWRENCE MCCANDLESS is an Associate Professor in the Faculty of Health Sciences at Simon Fraser University. Both authors have longstanding interests in sensitivity analysis for epidemiologic applications.

Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.