Campbell Standards: Modernizing Campbell's Methodologic Expectations for Campbell Collaboration Intervention Reviews (MECCIR)
Bibliographic record
Abstract
Introduction: The authors formed a small working group to modernize the Methodological Expectations for Campbell Collaboration Intervention Reviews (MECCIR). We reviewed comments and feedback from editors, peer reviewers of Campbell submissions, and authors; for example, that the Campbell MECCIR was long and some of the items in the reporting and conduct checklists were difficult to cross-reference. We also wanted to make the checklist more relevant for reviews of associations or risk factors and other quantitative non-intervention review types, which we welcome in Campbell. Thus, our aim was to develop a shorter, more holistic guidance and checklist of Campbell Standards, encompassing both conduct and reporting of these standards within the same checklist. Methods: Our updated Campbell Standards will be a living document. To develop this first iteration, we invited Campbell members to join a virtual working group; we sought experience in conducting Campbell systematic reviews and in conducting methods editor reviews for Campbell. We aligned the items from the MECCIR for conduct and reporting, then compared the principles of conduct that apply across review types to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)-literature search extension (S) and PRISMA-2020 reporting standards. We discussed each section with the aim of developing a parsimonious checklist with explanatory guidance while avoiding losing important concepts that are relevant to all types of reviews. We held nine meetings to discuss each section in detail between September 2022 and March 2023. We circulated this initial checklist and guidance to all Campbell editors, methods editors, information specialists and co-chairs to seek their feedback. All feedback was discussed by the working group and incorporated to the Standards or, if not incorporated, a formal response was returned about the rationale for why the feedback was not incorporated. Campbell Policy: The guidance includes seven main sections with 35 items multifaceted but distinct concepts that authors must adhere to when conducting Campbell reviews. Authors and reviewers must be mindful that multiple factors need to be assessed for each item. According to the Campbell Standards, the reporting of Campbell reviews must adhere to appropriate PRISMA reporting guidelines(s) such as PRISMA-2020. How to Use: The editorial board recommends authors use the checklist during their work in formulating their protocol, carrying out their review, and reporting it. Authors will be asked to submit a completed checklist with their submission. We plan to develop an online tool to facilitate use of the form by author teams and those reviewing submissions. Providing Feedback: . Plan for Updating: We will update the Campbell Standards periodically in light of new evidence.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.785 | 0.924 |
| Meta-epidemiology (narrow) | 0.003 | 0.009 |
| Meta-epidemiology (broad) | 0.008 | 0.011 |
| Bibliometrics | 0.033 | 0.031 |
| Science and technology studies | 0.009 | 0.019 |
| Scholarly communication | 0.032 | 0.020 |
| Open science | 0.015 | 0.028 |
| Research integrity | 0.016 | 0.028 |
| Insufficient payload (model declined to judge) | 0.011 | 0.009 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".