Campbell Standards: Modernizing Campbell's Methodologic Expectations for Campbell Collaboration Intervention Reviews (MECCIR)
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Introduction: The authors formed a small working group to modernize the Methodological Expectations for Campbell Collaboration Intervention Reviews (MECCIR). We reviewed comments and feedback from editors, peer reviewers of Campbell submissions, and authors; for example, that the Campbell MECCIR was long and some of the items in the reporting and conduct checklists were difficult to cross-reference. We also wanted to make the checklist more relevant for reviews of associations or risk factors and other quantitative non-intervention review types, which we welcome in Campbell. Thus, our aim was to develop a shorter, more holistic guidance and checklist of Campbell Standards, encompassing both conduct and reporting of these standards within the same checklist. Methods: Our updated Campbell Standards will be a living document. To develop this first iteration, we invited Campbell members to join a virtual working group; we sought experience in conducting Campbell systematic reviews and in conducting methods editor reviews for Campbell. We aligned the items from the MECCIR for conduct and reporting, then compared the principles of conduct that apply across review types to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)-literature search extension (S) and PRISMA-2020 reporting standards. We discussed each section with the aim of developing a parsimonious checklist with explanatory guidance while avoiding losing important concepts that are relevant to all types of reviews. We held nine meetings to discuss each section in detail between September 2022 and March 2023. We circulated this initial checklist and guidance to all Campbell editors, methods editors, information specialists and co-chairs to seek their feedback. All feedback was discussed by the working group and incorporated to the Standards or, if not incorporated, a formal response was returned about the rationale for why the feedback was not incorporated. Campbell Policy: The guidance includes seven main sections with 35 items multifaceted but distinct concepts that authors must adhere to when conducting Campbell reviews. Authors and reviewers must be mindful that multiple factors need to be assessed for each item. According to the Campbell Standards, the reporting of Campbell reviews must adhere to appropriate PRISMA reporting guidelines(s) such as PRISMA-2020. How to Use: The editorial board recommends authors use the checklist during their work in formulating their protocol, carrying out their review, and reporting it. Authors will be asked to submit a completed checklist with their submission. We plan to develop an online tool to facilitate use of the form by author teams and those reviewing submissions. Providing Feedback: . Plan for Updating: We will update the Campbell Standards periodically in light of new evidence.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.353 | 0.217 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.017 | 0.009 |
| Bibliometrics | 0.002 | 0.006 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.005 | 0.002 |
| Open science | 0.004 | 0.000 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.005 | 0.011 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it