Bibliographic record
Abstract
# Journal reviewers are even more baffled by sample size issues than grant proposal reviewers {#article-title-2} EDITOR—With reference to the article by Bacchetti,1 the confusion surrounding sample size estimates in research protocols elicits quite strange responses from reviewers when they are faced with the completed research in a report submitted to a journal for publication. One of our submissions was rejected because the planned sample size was not attained. But the effect size was greater in the study than we had anticipated, and thus the difference was of clinical and statistical significance. Another submission met the same fate for a similar reason—it was an equivalence trial—and even though the difference in effect between intervention and control arms (and both sides of the confidence interval around this difference) lay completely within the equivalence interval, the fact that the planned sample size was not attained in some way invalidated the result in the mind of the reviewer. Although sample size estimation is useful in considering the feasibility of conducting a study (and protocol reviewers should discourage funding for studies that are plainly too small to be meaningful) attainment of the planned sample size does not seem to me to be a useful indicator by which journal reviewers should assess the validity of a completed research report in which clinically and statistically meaningful results have been obtained. 1. 1.↵1. Bacchetti P .Peer review of statistics in medical research: the other problem.BMJ2002; 324:1271–1273. (25 May.) [OpenUrl][1][FREE Full Text][2] # Rationale for requiring power calculations is needed {#article-title-4} EDITOR—The article by Bacchetti with its comments about uncertainties surrounding power calculations prompted me to seek advice about an issue that has implications for clinical research.1 The company I work for, Laxdale Limited, often conducts pilot studies on new entities. Our usual practice is to state in the protocol that there is no reasonable basis for a power calculation. In order … [1]: {openurl}?query=rft.jtitle%253DBMJ%26rft.stitle%253DBMJ%26rft.issn%253D0007-1447%26rft.aulast%253DBacchetti%26rft.auinit1%253DP.%26rft.volume%253D324%26rft.issue%253D7348%26rft.spage%253D1271%26rft.epage%253D1273%26rft.atitle%253DPeer%2Breview%2Bof%2Bstatistics%2Bin%2Bmedical%2Bresearch%253A%2Bthe%2Bother%2Bproblem%26rft_id%253Dinfo%253Adoi%252F10.1136%252Fbmj.324.7348.1271%26rft_id%253Dinfo%253Apmid%252F12028986%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [2]: /lookup/ijlink?linkType=FULL&journalCode=bmj&resid=324/7348/1271&atom=%2Fbmj%2F325%2F7362%2F491.2.atom
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Direct model labels (unvalidated)
Per-model category and study-design labels from the labeling rounds. They are machine output, unvalidated, and the disagreement between models ships as data. No study design here is MEDLINE-validated yet.
| Model arm | Categories | Study design | Confidence |
|---|---|---|---|
| gemma | Metaresearch Domain: Evaluation · Genre: Commentary About the Canadian research system: no · About a Canadian topic: no | Not applicable | high |
| gpt | Metaresearch Domain: Evaluation · Genre: Commentary About the Canadian research system: no · About a Canadian topic: no | Not applicable | high |
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.574 | 0.617 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.006 | 0.001 |
| Bibliometrics | 0.001 | 0.002 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.004 | 0.000 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.255 | 0.016 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedLabeled directly by 2 models reading the full record.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".