Bibliographic record
Abstract
Professor Mudge and colleagues have presented a strong case for the use of meta-analysis in providing the best evidence on which to base health care decisions. They have proposed several benefits and highlighted the limitations of meta-analysis. While we concur with all the listed limitations, we feel that the following benefits they proposed need to be presented with caution. We agree that this is one of the key potential benefits of meta-analysis; however, caution needs to be exercised on two points regarding power. First, large does not always imply better. A very large sample size can lead to a detection of small and non-clinically important effect sizes as statistically significant. The size of the summary effect should be interpreted clinically before it is interpreted statistically. Second, while there is a general perception that meta-analyses always have high power to detect main effects, this is not necessarily true. Borenstein et al. [1] makes the point that in the Cochrane Database of Systematic Reviews, the median number of trials included in a review is six. When a review includes a small number of trials, the power to detect a moderate effect size may be low, especially if the random effects rather than the fixed effects model is used to compute the summary effect. This is because in the random effects model, power depends on both the within-studies error and between-studies variation. A random effects meta-analysis will only have more power than the individual studies that comprise it if the effect sizes are consistent across studies and there are a substantial number of studies. It is possible to have low power even if there are thousands of participants included in the meta-analysis [1]. This is good as a global intent of meta-analysis, but without author and journal adherence to reporting guidelines, it is doubtful whether such transparency can be achieved. The real question here is how do we ensure that the primary studies are transparent, even if there is transparency in the reporting of meta-analysis? While it is true that meta-analyses can provide greater generalizability, there are cases where the evidence has been clearly concentrated in certain regions of the world and yet generalized to others. For example, paediatric clinical trials are well known to be lacking in countries with the greatest burden of disease, and a low correlation between the number of trial participants and disease burden has been reported, especially in low-income countries [2]. Meta-analysis refers to the statistical tools to summarize evidence. Currently the methods do not include ways to disseminate such evidence. Greenhalgh et al. [3] point out many problems that hinder knowledge translation, including that the evidence must be presented in a form that is interpretable to those who need it, including the public and clinicians of limited statistical literacy. Moreover, evidence-based medicine's greatest challenge has been in getting evidence into policy and practice, leading to ‘widespread scepticism concerning its impact on practice’ [4].
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.009 | 0.067 |
| Meta-epidemiology (narrow) | 0.002 | 0.002 |
| Meta-epidemiology (broad) | 0.003 | 0.004 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.012 | 0.006 |
| Scholarly communication | 0.013 | 0.006 |
| Open science | 0.006 | 0.006 |
| Research integrity | 0.174 | 0.104 |
| Insufficient payload (model declined to judge) | 0.024 | 0.017 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".