The influence of study characteristics on coordinate-based fMRI meta-analyses
Bibliographic record
Abstract
Abstract Given the increasing amount of neuroimaging studies, there is a growing need to summarize published results. Coordinate-based meta-analyses use the locations of statistically significant local maxima with possibly the associated effect sizes to aggregate studies. In this paper, we investigate the influence of key characteristics of a coordinate-based meta-analysis on (1) the balance between false and true positives and (2) the reliability of the outcome from a coordinate-based meta-analysis. More particularly, we consider the influence of the chosen group level model at the study level (fixed effects, ordinary least squares or mixed effects models), the type of coordinate-based meta-analysis (Activation Likelihood Estimation, fixed effects and random effects meta-analysis) and the amount of studies included in the analysis (10, 20 or 35). To do this, we apply a resampling scheme on a large dataset ( N = 1400) to create a test condition and compare this with an independent evaluation condition. The test condition corresponds to subsampling participants into studies and combine these using meta-analyses. The evaluation condition corresponds to a high-powered group analysis. We observe the best performance when using mixed effects models in individual studies combined with a random effects meta-analysis. This effect increases with the number of studies included in the meta-analysis. We also show that the popular Activation Likelihood Estimation procedure is a valid alternative, though the results depend on the chosen threshold for significance. Furthermore, this method requires at least 20 to 35 studies. Finally, we discuss the differences, interpretations and limitations of our results.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.445 | 0.697 |
| Meta-epidemiology (narrow) | 0.003 | 0.002 |
| Meta-epidemiology (broad) | 0.008 | 0.026 |
| Bibliometrics | 0.006 | 0.009 |
| Science and technology studies | 0.002 | 0.004 |
| Scholarly communication | 0.006 | 0.006 |
| Open science | 0.005 | 0.003 |
| Research integrity | 0.004 | 0.006 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".