Re-analysis of data from cluster randomised trials to explore the impact of model choice on estimates of odds ratios: study protocol
Bibliographic record
Abstract
BACKGROUND: There are numerous approaches available to analyse data from cluster randomised trials. These include cluster-level summary methods and individual-level methods accounting for clustering, such as generalised estimating equations and generalised linear mixed models. There has been much methodological work showing that estimates of treatment effects can vary depending on the choice of approach, particularly when estimating odds ratios, essentially because the different approaches target different estimands. METHODS: In this manuscript, we describe the protocol for a planned re-analysis of data from a large number of cluster randomised trials. Our main objective is to examine empirically whether and how odds ratios estimated using different approaches (for both primary and secondary binary outcomes) vary in cluster randomised trials. We describe the methods that will be used to identify the datasets for inclusion and how they will be analysed and reported. DISCUSSION: There have been a number of small comparisons of empirical differences between the different approaches to analysis for CRTs. The systematic approach outlined in this protocol will allow a much deeper understanding of when there are important choices around the model approach and in which settings. This will be of importance given the heightened awareness of the importance of estimands and the specification of statistical analysis plans.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.312 | 0.565 |
| Meta-epidemiology (narrow) | 0.007 | 0.005 |
| Meta-epidemiology (broad) | 0.011 | 0.015 |
| Bibliometrics | 0.007 | 0.009 |
| Science and technology studies | 0.003 | 0.005 |
| Scholarly communication | 0.006 | 0.007 |
| Open science | 0.006 | 0.005 |
| Research integrity | 0.014 | 0.015 |
| Insufficient payload (model declined to judge) | 0.054 | 0.017 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".