Daring to draw causal claims from non-randomized studies of primary care interventions

Primary care interventions, including new primary care policies or quality improvement programs, are often evaluated without the use of randomized controlled trials (RCTs), as randomizing who receives the intervention can be infeasible for many practical, ethical and political reasons (1). In these cases, evidence on the effect of the intervention must stem from non-randomized studies (e.g. quasi-experimental studies, natural experiments or observational studies) which presents many complexities to isolating the causal effect from the many sources of bias and threats to validity, including concurrent events, lack of comparability across groups, selection bias, etc. Faced with these barriers, researchers often conservatively accept that determining causal effects in such non-randomized settings is unattainable and have become complacent with claims of ‘association’ rather than ‘causation.’ Recent methodological developments in the causal inference literature, however, have shown that, if specific conditions hold, the causal effect of non-randomized interventions can still be reliably estimated (2,3). These advancements represent a paradigm shift in how we approach omnipresent causal questions, opening up the possibility of making causal claims even with non-randomized data. Methods developed under this causal framework are becoming increasingly used in many other fields, including epidemiology (4), pharmacosurveillance (5,6) and health economics (7), but have yet to permeate into mainstream primary care research. Given that many primary care studies are conducted outside the randomized setting, causal inference methods offer enormous potential to this field including applications in practice-based research, health services research, pragmatic trials and quality improvement initiatives. This methods brief provides (i) an overview of the causal inference framework and its underlying conditions and (ii) practical examples of how its analytical methods can be applied to reduce bias in the estimation of the effect of common primary care interventions. For more in-depth readings, seminal references are provided throughout the text. Let us consider an example where we want to evaluate the effect of a new primary care intervention, say the introduction of interdisciplinary primary care teams, on an outcome, such as chronic disease management. Suppose enrollment into this intervention was voluntary and we wish to compare disease management for patients in these team-based practices (intervention group) to patients in solo practices (comparison group). To attribute changes in disease management to the intervention and claim a ‘causal effect’, we ideally wish to know for every patient: ‘Would their disease management be the same whether they received interdisciplinary care or not?’. From Figure 1, this question equates to the theoretical situation on the bottom, where we would expose everyone to the intervention and then, in a counterfactual world, withhold the intervention from everyone and compare their outcome on disease management. In reality, of course, we can only observe the situation at the top, where each patient either receives the intervention or not, and so, we can only observe the outcome for the exposure actually received. The fact that we can only ever observe one of the two potential outcomes for each person is what is called the ‘fundamental problem of causal inference’ (2). Illustration of causation versus association. The causal inference framework (also known as the potential outcomes framework) formalizes the once vague concept of causality, using explicit mathematical notation to define and address these causal concepts. It maps out the conditions needed to use the observed data at the top of Figure 1 to infer to the theoretical situation on the bottom of Figure 1, essentially allowing us to extend from ‘association’ to ‘causation’. In the case of non-randomized studies, this framework shows that the key is to consider these studies as if they were pseudo-randomized (2). While this may seem out-of-reach, this requirement simply relies on three conditions being met: (i) consistency, (ii) positivity and (iii) exchangeability (2). Consistency refers to the condition that the intervention to be evaluated be well defined and specific enough to warrant an unambiguous and meaningful estimate of the causal effect. In other words, the intervention should be the same for all study subjects and be implemented in the same way. For example, the intervention on interdisciplinary primary care teams should specify the exact team composition (doctor, nurse, social worker, pharmacist, etc.), and clinics applying the intervention should adhere to the same intervention guidelines on roles and responsibilities of the team members, schedule for team meetings, etc. Otherwise, variations on the intervention would make it difficult to attribute a single causal effect. Positivity requires that all persons in the study population be ‘potentially exposable’ to the intervention and comparison group. In our example, this would imply that any patient from the target population could, in theory, have received the interdisciplinary care intervention. One scenario where this condition might be violated would be in the case of regional barriers, for example, where only patients with primary care providers in urban areas could access this new intervention. The exchangeability condition, also known as ‘no unmeasured confounding’, refers to the interchangeability of patients between the intervention and comparison group. This means that if we swapped the patients in intervention group and those in the comparison group, the expected difference in the outcome would remain unchanged (2). While this is theoretically guaranteed under randomization, in the case of non-randomized allocation of interventions, this is often not justifiable as there are usually imbalances or systematic differences in the characteristics of the patients in each group. For example, patients receiving the interdisciplinary care intervention may be older, less educated, have more comorbidities, etc. When systematic imbalances of covariates across intervention groups are causally linked to the outcome of interest, we call them ‘confounders’. A key mathematical result within the causal inference framework is that if we can control for all existing confounders, then receiving the intervention or not becomes independent of any variables that may cause the outcome, as is the case in an RCT, allowing for the estimation of a causal effect. There are other scenarios when the exchangeability condition may be violated. Adjusting for variables that are on the causal pathway between the intervention and outcome (mediators) or variables that are affected by both the intervention and an unmeasured covariate of the outcome (colliders) can actually induce rather than reduce bias in the estimate of the effect (8). A diagram of the causal relationships between variables, known as a directed acyclic graph (DAG; Fig. 2), can help to distinguish between these types of variables and determine which analytical approach is needed to address the different sources of bias. Practical examples of DAGs and these analytical approaches are presented in the next section. Causal relationships between variables in a DAG. Now that we have reviewed the conditions (consistency, positivity and exchangeability) for the estimation of causal effects of non-randomized interventions, we now describe three causal inference methods that can be used to answer relevant primary care research questions that are implicitly or explicitly causal in nature. These methods address various threats to the exchangeability condition in ways that conventional regression techniques cannot. Marginal structural models (MSMs) were primarily developed to overcome the limitations of conventional confounder adjustment methods with respect to biases arising from so-called time-dependent confounding (2,3). A recently published article by Héroux et al. (9) aimed to assess the impact of patient enrolment into an integrated primary care delivery model (Family Medicine Groups or FMGs) on emergency department (ED) visits in Québec over a 3-year follow-up period. The presence or absence of chronic illnesses was believed to be confounders, affecting both patient enrolment into an FMG and the likelihood of ED visits. In addition, patient enrolment into an FMG was, in turn, also thought to influence chronic illness. When a confounder, like chronic illness, changes over time because it is influenced by prior exposure to an intervention (FMG), it also acts as a mediator in the causal pathway (Fig. 3). When unmeasured covariates (underlying health status) are present (Fig. 3), this creates a phenomenon known as ‘time-dependent confounding’ (10). Conventional regression adjustment for time-dependent confounders would induce a biased estimation of the intervention effect. An example of time-dependent confounding when assessing the impact of FMGs on ED visits. To address this issue, Héroux et al. (9) analysed their data with a MSM and compared their results to a conventional regression approach. The MSM uses a weighting approach to emulate the theoretical population shown on the bottom of Figure 1. This weighting, which is often derived from propensity scores, balances exposed and unexposed patients across all measured confounders, thus ensuring that the exchangeability condition holds (4). In the study, the conventional regression model estimated a biased risk ratio of 0.979 (95% CI 0.963–0.995), while the MSM produced an unbiased risk ratio of 0.933 (95% CI 0.909–0.958). This example demonstrates the advantage of using MSMs in longitudinal studies where the exposure to the intervention and the confounders can vary over time. Instrumental variable (IV) analysis represents another important tool for causal inference in primary care research. Recall that the exchangeability condition requires that we know and measure all confounders of the relationship between an intervention and outcome. What happens when we know there are important confounders we cannot measure? IV analysis provides a ‘work-around’ to estimate the causal effect of interventions, even in the presence of unmeasured confounding (11). It does this by finding an external variable, the IV, that satisfies the following assumptions: (i) it is strongly predictive of who receives the intervention; (ii) it causes the outcome only through its relationship with the intervention and (iii) it cannot be influenced by other unmeasured predictors of the outcome (Fig. 4) (12). Since the IV allows for the estimation of an intention to treat effect (blue arrow), it circumvents the bias introduced by unmeasured confounding. Causal diagram illustrating assumptions for IV analysis. A commonly used IV in pharmacoepidemiology is physician prescribing preference (13). In a database study assessing the short-term effects of COX-2 inhibitors versus other non-steroidal anti-inflammatory drugs (NSAIDs) on gastrointestinal toxicity, Brookhart et al. (14) identified unmeasured confounding as a major threat to the validity of their findings. To address this concern, they analysed data using IV analysis in addition to conventional regression and compared their results to published results from a previous RCT. Because prescribing different types of NSAIDs is thought to significantly vary between physicians, and the preference for NSAIDs is assumed not to be associated with any confounders, physician prescribing preference was selected as the IV. The IV analysis found a protective effect attributed to COX-2 inhibitors when compared with NSAIDS, which was in agreement with the RCT. The conventional regression approach, on the other hand, found no statistically significant difference. Good IVs can be hard to find and come with some additional assumptions, but when applicable, IV analysis provides an ingenious solution to dealing with unmeasured confounders, an all-too-common scenario in non-randomized studies. Identifying mediators of a causal pathway between intervention and health outcome is important for population health, as it allows for health policy experts to develop targeted solutions that can intervene at the level of the mediator. However, mediation analyses are susceptible to the same type of bias as time-dependent confounding and conventional adjustment methods fail to produce effects that satisfy the exchangeability condition for causal inference. With the advent of causal inference methods such as MSMs and IV analyses, we now have the tools to decompose the total causal effect of an intervention into direct (blue arrow) and indirect or ‘mediated’ (red arrows) causal effects (Fig. 5) (15). For example, in a recent French study investigating the mediators of lung cancer risk in men with varying degrees of education, the total causal effect of education level on lung cancer incidence was decomposed into direct and indirect effects mediated by smoking and other occupational exposures using MSMs (16). Menvielle et al. (16) found that 31% of the total effect was mediated by cumulative lifelong smoking among men with a high-school degree. Based on their results, the authors recommended health policies targeting tobacco control to reduce socioeconomic disparities in lung cancer. Direct and indirect causal effect of education level on lung cancer. In conclusion, the causal inference framework states that, when causal conditions hold (consistency, positivity, exchangeability), causal effects can still be estimated for non-randomized primary care interventions (Supplementary Table S1). If one or more conditions are violated, the impact of these violations must be further investigated (for instance, through applying sensitivity analyses for unmeasured confounders). Causal inference methods provide analytical tools to deal many sources of bias that cannot be dealt with using conventional regression methods: MSMs may be applied to overcome adjustment problems arising from time-dependent confounding, IV analyses can be used to address unmeasured confounding and mediation analyses can elucidate causal pathways of an intervention effect (Supplementary Table S2). New advances in causal inference offer promising ways to conduct our primary care studies, improve the quality of evidence that we produce and ensure that changes to our practices and health systems are based on sound, robust evidence of the causal effects of the interventions studied. Causal methods are the future and should be at the forefront of the quantitative armamentarium for primary care researchers. Supplementary material is available at Family Practice online. We would like to thank Dr Geneviève Arsenault-Lapierre for her input on the manuscript and Ms Marine Hardouin for assistance with the figures. Funding: This methods brief was funded by the Canadian Institutes of Health Research–Vanier Canada Graduate Scholarship. Conflicts of interest: The authors have no conflicts of interest to declare.

Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.