A Call for Caution in Using Information Criteria to Select the Working Correlation Structure in Generalized Estimating Equations
Bibliographic record
Abstract
To the Editor: Generalized estimating equations (GEEs) are popular tools for estimating associations in clustered data settings. The semiparametric nature of this approach makes it highly appealing because unbiased effect estimators can be obtained without knowing the true distribution of the data being modeled. For example, it is unnecessary to specify a specific parametric distribution or even the correct correlation structure within the data – the mean model parameter estimators are unbiased if the mean model is correctly specified. However, a working correlation that is close to the structure of the true data-generating mechanism provides greater efficiency than a poorly specified working correlation.1 Thus, it is tempting to employ some method of choosing the working correlation structure – potentially reducing standard errors and improving the power to detect an association between a covariate and the outcome. To this end, several criteria for specifically selecting the working correlation structure (as opposed to selecting covariates in the mean model) have been proposed, including Pan’s seminal quasi-likelihood information criterion2 and variations thereof.3,4 Many of these criteria have been implemented in commonly used software such as SAS and Stata, which facilitates their use by data analysts, some of whom may not be fully aware of the drawbacks of the criteria. While such information criteria have sound theoretical bases, their use can have unintended consequences if their application leads the analyst to choose an inappropriate working correlation structure for the chosen mean model. For instance, GEEs yield biased estimators of cross-sectional model parameters when the true data-generating mechanism relies on covariate history5 (such as when a “cross-sectional” model is being fit to data and the true underlying data-generating mechanism is not cross-sectional) unless an independence correlation structure is assumed. For example, one may wish to understand the predictive value of current covariate measurements on current health status to understand what can be learned from the information available in a given visit without relying on historical measurements. Current health is highly likely to be predicted by additional antecedent factors, e.g., previous health status. In this setting, data analysts must use an independence working correlation when regressing health status on covariates using GEEs. We have previously demonstrated6 that type I error is distorted because of postselection inference, i.e., the use of confidence intervals or significance tests following model selection. Moreover, in the eAppendix; https://links.lww.com/EDE/B384, we demonstrate via brief simulations that bias can arise due to using information criteria in settings where an independence working correlation is required. While these limitations of model selection in the GEE context are well-known to statisticians, this message appears to be insufficiently disseminated to other fields. For instance, more than 80% of the citations of Pan’s quasi-likelihood information criterion are in nonstatistical journals,6 suggesting that the criterion is being used in routine data analysis, in, for example, epidemiology and cancer biology. Even in our institution, the routine use of these criteria is encouraged, without mentioning the potential perils discussed above. Moreover, new criteria continue to be developed7,8 despite these potential perils. We urge data analysts to consider selection of the working correlation structure based on the data-generating mechanism and not solely on information criteria. The development or extensions of ever more methods for choosing among different correlation structures is of little use and may even be counterproductive if used in the same manner as the previously developed criteria already in use. Thus, while GEEs offer consistency without perfect knowledge of the correlation structure, reliance on this known and proven property may be the most prudent and fruitful analysis approach. Wilhemina Adoma PelsAfrican Institute for Mathematical SciencesSenegal Mbour, Senegal Shomoita AlamMcGill UniversityCanada Montreal, Canada Lindsay N. CarppVaccine and Infectious Disease DivisionFred Hutchinson Cancer Research CenterSeattle, USA Erica E. M. MoodieMcGill UniversityCanada[email protected]
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.097 | 0.468 |
| Meta-epidemiology (narrow) | 0.003 | 0.002 |
| Meta-epidemiology (broad) | 0.005 | 0.004 |
| Bibliometrics | 0.004 | 0.005 |
| Science and technology studies | 0.003 | 0.012 |
| Scholarly communication | 0.009 | 0.009 |
| Open science | 0.014 | 0.004 |
| Research integrity | 0.023 | 0.061 |
| Insufficient payload (model declined to judge) | 0.010 | 0.009 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".