Dealing with missing data using the Heckman selection model: methods primer for epidemiologists

Missing data are a recurrent problem in all types of medical and epidemiological studies, regardless of the rigour with which they are designed and executed. Although causes of missing data can greatly vary, their presence is often addressed by omitting study participants with incomplete records from the analysis or by adopting multiple imputation methods.1 These strategies generally assume that missing values can be recovered from the observed data without introducing bias in study results (that is, data is missing at random; MAR). Unfortunately, problems arise when the missing data process is informative, i.e. both the probability of data availability and the value of missing observations are affected by unobservable or unmeasured factors. In this situation, data are missing not at random (MNAR).2 For instance, consider a health survey in which human immunodeficiency virus (HIV) testing is conducted to determine HIV prevalence in a population. The willingness to participate in a survey is rarely completely random, and often depends on observed (e.g. age, gender) or even unobserved (e.g. level of trust in the health care system, historical information on HIV status) factors. Missing values can therefore arise for key survey variables (e.g. HIV test result) when individuals refuse to participate or do not report some information. One option to address these missing values is to adopt traditional imputation methods and to recover incomplete variables (e.g. HIV status) from other observed survey variables (e.g. age, gender, sexual orientation). Although this approach is reasonable when the measured variables are strongly related to the missing data, validity is questionable when variables that affect the survey participation and the incomplete variables are not measured. In such a situation, it becomes important to adopt more advanced imputation methods that explicitly account for the causes of missing data.3,4 One of the most common approaches to deal with MNAR data is the so-called selection model, which involves two main equations to describe the outcome and the missingness process. The literature on selection models is extensive and different selection models have been proposed.5,6 We focus on the selection model originally proposed by Heckman (henceforth ‘Heckman model’)7 to impute missing variables. We first provide an overview of the use of the Heckman model with a focus on epidemiological studies and, in addition, propose a guide that may be useful when selecting exclusion restriction variables. The Heckman selection model was originally developed to address situations in which individuals are selectively missing from an observational survey or study.8 This model corrects for the sample selection bias that occurs when the selection of individuals or units in a sample is driven by observable and unobservable variables. The unobservable variables are correlated with other unobservable variables that also affect the outcome variable of interest. The Heckman model involves two main equations to derive unbiased estimates of study associations in a non-representative sample. The first equation is called the outcome equation and describes the relationship between the covariates (e.g. age, sex, sexual orientation) and the outcome of interest (e.g. HIV status). In contrast, the second (selection) equation specifies the probability that an individual will be included in the study sample. Estimation of the selection equation sometimes requires the inclusion of one or more instrumental (exclusion restriction) variables that are not included in the outcome equation. The outcome and selection equations are usually combined into a single (multivariate) model by assuming that their respective error terms follow a bivariate normal distribution (BVN) (Supplementary Material, available as Supplementary data at IJE online). The correlation parameter between the error terms of both equations has a straightforward interpretation. If the estimates for the correlation differ from zero, there is an indication that the missing observations (e.g. for the HIV test value) are MNAR. Conversely, if the estimates for the correlation are close to zero, the Heckman model collapses to the outcome equation, suggesting that the missing data follow an MAR mechanism. The Heckman model has recently been extended to allow multiple imputation of variables that are MNAR.9,10 The outcome equation is typically used as imputation model to describe the (conditional) distribution of the missing data given the observable data, from which the imputed values of missing completely at random (MCAR) or missing at random (MAR) variables are drawn. To accommodate variables that are MNAR, the imputation model is specified from the joint estimates of the outcome and selection equations. The degree of correlation between both equations allows Heckman-based imputation methods to switch between different degrees of MCAR, MAR and MNAR conditions without having to pre-specify the ‘correct’ missing data mechanism. Theoretically, the Heckman selection model is identifiable when the same set of variables is included in the outcome and in the selection equation. Such identification is possible under the normality assumption of the residuals, more specifically, due to the non-linearity of the selection equation, through a correction function known as the inverse Mills ratio (Supplementary Material). However, in practice, this correction is nearly linear over most of its range, which often leads to (quasi-) collinearity and numerical convergence problems.11 For this reason, it is often helpful to facilitate the estimation of the Heckman model by including an ERV in the selection equation.12 An ERV is an instrument that provides independent information about the selection into the sample, and by definition it meets two conditions: relevance and exclusion.13 The relevance condition implies that the ERV should have a causal effect on the presence of missingness. In other words, the ERV should (partially) explain why an individual is not included in the sample (in case of missing study participants) or why a variable has not been recorded (in case of missing data). Conversely, the exclusion condition states that the ERV has no direct or indirect effect on the outcome other than through the selection variable and that there are no common causes between the ERV and outcome variable. For example, in HIV prevalence studies, the identity of the interviewer is often used as an ERV when interviewers are randomly assigned to survey participants and since certain interviewer characteristics (e.g. social skills) may affect a subject’s willingness to cooperate in data collection (and for example agree to undergo HIV testing) but not affect the data being collected (e.g. HIV test results). Ideally, an ERV should be unrelated to the outcome variable, not weakly related to the selection variable, random and well supported. We discuss these properties in more detail below. First of all, it is important that the ERV is not affected by other variables that also affect the outcome variable (i.e. common causal factors). For example, consider that patients with more severe illnesses are treated by more experienced physicians than patients with less severe illness. When the identity of the treating physician is used as an ERV to describe the missingness of HIV test results, HIV prevalence estimates will be biased. This is because the (causal) effect of the ERV on study participation is confounded by the experience level of the treating physician. Second, the ERV should be a strong instrument, i.e. it should have a moderate to high correlation with the selection variable even after controlling for other predictors. A weak ERV could lead to multicollinearity problems, when the BVN error assumption is not met,14 and it has been shown empirically that a moderate to strong ERV can lead to more stable model estimates.13 Third, the ERV should randomly differ between study participants. A common example in surveys are characteristics of the interview, such as time (e.g. day of the week) and location. However, it is possible that some of these survey characteristics (e.g. interview ID) are affected by other survey characteristics (e.g. location or language of the study participants). When the ERV does not vary completely at random between study participants, confounding bias may arise between the ERV and the selection variable, if no adjustments are made in the selection equation. Caution is warranted when adjusting for (potential) confounders of imperfect ERV variables, as the inclusion of variables that do not affect the allocation of the ERV and the presence of missingness may deteriorate the estimation of the selection equation. Additional problems arise when ERVs do not have monotonic effects on study participants. For example, an interviewer may encourage participation in a specific group of subjects (e.g. men, young people) but discourage participation in another group (e.g. women, elderly). In this case, the relevance condition of the ERV is weak or even invalid. Importantly, the validity of an ERV cannot be tested from the observed data. However, the choice of the instrument could be motivated by expert opinion or by previous studies where the instrument was associated with study participation or completeness of the outcome variable. Although ERV are often identified at the analysis phase of a study, they can also explicitly be specified during their design phase (Figure 1). Below, we describe both situations in more detail. Recommended actions for defining an exclusion restriction variable (ERV) Efforts to identify ERVs are often made after data collection, which is not a very straightforward process because, in general, the exclusion condition cannot formally be verified from the observable data. For this reason, researchers sometimes prefer to remove one variable from the outcome equation or to include variables that do not affect study participation in the selection equation. Both approaches tend to alleviate multicollinearity problems but may add noise and even increase bias in estimated study associations.15 Careful thought is thus needed when designing the selection equation. In practice, ERVs can often be derived from survey characteristics or demographic variables that affect the selection process (Table 1). But many potential ERVs can also be selected from metadata and linked data (e.g. social data or clinical workflow data on electronic health records) describing the data collection process. For instance, presence of missingness sometimes varies across interviewers or treating physicians,17–20 by calendar time of the interview (e.g. hour of the day, day of the week, or even by season), by order of the scheduled interview or by small variations in the study protocol. Examples of exclusion restriction variables HIV, human immunodeficiency virus; MNAR, missing not at random; ART, antiretroviral therapy, ID, identification; ERV, exclusion restriction variable; MDA, mass drug administration. The selection of ERVs is limited to the available information, from which it is not uncommon that instrumental variables are not available or that the measured instrumental variables are incomplete or do not meet the criteria for defining an ERV. In such situations, the adoption of Heckman models can be problematic. When data are collected explicitly for research purposes, it may be useful to consider the creation of an ERV prior to data collection, that is during the design phase of a study. This ensures that ERVs will be measured in an appropriate manner and be available upon entry into the analysis phase to impute key study variables that are informatively missing. To this end, researchers could make an explicit effort to collect as much metadata as possible. The survey protocol could encourage the interviewer to gather information about valid ERVs pertaining to the data collection process, such as the duration of the survey and the survey environment (e.g. the presence of other people at the time of the survey). Such information can often be retrieved automatically, without explicit need for human interaction. To avoid confounding factors, it is recommended to randomize the allocation of relevant instrumental variables during the study design phase. In practice, many survey characteristics such as interviewer assignment, the dates of visits, the order of visits or even the locations of visits are chosen semi-arbitrarily and could explicitly be randomized. Such randomization efforts would not affect the observational nature of the study and help to ensure the validity of ERVs ahead of time. Finally, a specific type of ERVs are micro-incentives (Table 2). These incentives are designed to influence (usually increase) survey participation and to alter (usually reduce) the probability of missing data. To this purpose, a reward is offered to individuals who agree to participate in the survey or who agree to share certain information. Micro-incentives do not affect study outcomes and therefore do not affect the observational nature of surveys. Again, to avoid confounding of micro-incentive effects on the presence of missing data, randomization of micro-incentives is recommended. Examples of standard tools to induce behaviour changes Monetary compensation Tangible item (pen, keychain) Food voucher Option to participate in a raffle Free admission to an event Discounts on future purchases Visual card showing people affected or at risk Graph showing how previous interventions have increased the quality of life in the community Reward a charity instead of the individual Briefing letter about the objectives of the study before data collection Explain the benefits of the survey and implications for future research Show the history and reputation of the organization conducting the study The creation of ERVs could also be facilitated in registries by randomly affecting their data recording process. For example, many registries are populated by health care staff through a software interface. This interface could randomly incentivize the completion of certain variables (e.g. smoking status, disease severity) by altering the visualization of incomplete fields (e.g. missing entries are displayed in red or accompanied by a warning message) or even by generating a pop-up window requesting the missing information be filled out. We consider a hypothetical study aimed at estimating the prevalence of HIV in a specific population. We anticipate that HIV test results will be MNAR, because some patients may have been recently or repeatedly tested or because some believe they are at low risk of infection. We therefore consider measuring an ERV to enable the adoption of Heckman-based imputation methods during the study analysis phase. We compare three different survey protocols to illustrate the effect of ERV on the estimation of the HIV prevalence in the population. The test results are given either in a continuous forms Y*, e.g. the cut-off index or as well as a binary outcome (Y*

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.