Methodological standards in the design and reporting of pilot and feasibility studies in emergency medicine literature: a systematic review
Bibliographic record
Abstract
OBJECTIVE: Pilot and feasibility studies are intended to ensure that subsequent randomised controlled trials (RCTs) are feasible, economical and rigorous, especially in a challenging research environment such as emergency medicine (EM). We aimed to evaluate the methodological quality in conducting and reporting randomised pilot and feasibility studies in the EM literature and propose recommendations to improve their quality. DESIGN: Methodological systematic review. DATA SOURCES AND ELIGIBILITY: We searched MEDLINE and Embase (2018-29 September 2023) for pilot or feasibility RCTs published as full texts in the five top-ranked and other first-quartile EM journals according to Scimago. DATA EXTRACTION AND ANALYSIS: We assessed their methodological features and reporting quality primarily based on the Consolidated Standards of Reporting Trials (CONSORT) extension. RESULTS: A total of 24 randomised trials identified as pilot (n=13), feasibility (n=3) or both (n=8) were included. At least one feasibility outcome was assessed in 9 trials (feasibility trials), while 15 others only focused on treatment efficacy (efficacy trials). Only three (12.5%) studies progressed to the main trials. Among 12 feasibility trials, 55.6% reported their outcomes with uncertainty estimates, and 33.3% had clear progression criteria. Efficacy trials tended to draw clinical implications on their results. Studies from the five top-ranked journals had better methodological and reporting quality than those from other first-quartile journals. CONCLUSION: Main methodological concerns for pilot and feasibility studies in first-quartile EM literature include misconceptions, misuses and suboptimal design and reporting quality. These issues were more prominent in lower-ranked first-quartile journals. Our findings highlight the need for resources and training for researchers, journal editors and peer reviewers on the value, objectives and appropriate conduct of pilot and feasibility studies. The conceptual framework and standardised methodological components should be emphasised. EM journals should reinforce the reporting standards and support their publication. These actions can lead to more methodologically rigorous pilot and feasibility studies in EM. PROSPERO REGISTRATION NUMBER: CRD42023468437.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.580 | 0.767 |
| Meta-epidemiology (narrow) | 0.003 | 0.005 |
| Meta-epidemiology (broad) | 0.015 | 0.017 |
| Bibliometrics | 0.030 | 0.027 |
| Science and technology studies | 0.005 | 0.010 |
| Scholarly communication | 0.013 | 0.014 |
| Open science | 0.008 | 0.007 |
| Research integrity | 0.011 | 0.006 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".