The methodological and reporting characteristics of Campbell reviews: A systematic review
Notice bibliographique
Résumé
Background The Campbell Collaboration undertakes systematic reviews of the effects of social and economic policies (interventions) to help policymakers, practitioners, and the public to make well-informed decisions about policy interventions. In 2010, the Cochrane Collaboration and the Campbell Collaboration developed a voluntary co-registration policy under the rationale to make full use of the shared interests and diverse expertise from different review groups within these two organizations. In order to promote the methodological quality and transparency of Campbell intervention reviews, the Methodological Expectations of Campbell Collaboration Intervention Reviews (MECCIR) were introduced in 2014 to guide Campbell reviewers. However, there has not been a comprehensive review of the methodological quality and reporting characteristics of Campbell reviews. Objectives This review aimed to assess the methodological and reporting characteristics of Campbell intervention reviews and to compare the methodological quality and reporting completeness of Campbell reviews published before and after the implementation of MECCIR. A secondary aim was to compare the methodological quality and reporting completeness of reviews registered with Campbell only versus those co-registered with Cochrane and Campbell. Search Methods We searched the Campbell Library to identify all the completed intervention reviews published between 1 January 2011 to 31 January 2018. Selection Criteria One researcher downloaded and screened all the records to exclude non-intervention reviews based on reviews’ title and abstract. A second researcher checked the full text of all the excluded records to confirm the exclusion. In case of discrepancies, the two researchers jointly agreed on the final decision. Data Collection and Analysis We developed the abstraction form based on mandatory reporting items for methods, results, and discussion from the MECCIR reporting standards Version 1.1; and additional epidemiological characteristics identified in a similar study of systematic reviews in health. Additionally, we judged the methodological quality and completeness of reporting of each included review. For methodological quality, we used the AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews 2) instrument; for reporting completeness we used the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist. We rated reporting as either complete/partial or not reported. We described characteristics of the included reviews with frequencies and percentages, and median with interquartile ranges (IQRs). We used Stata version 12.0 to conduct multiple linear regressions for continuous data and the ordered logistic regressions for ordered data to investigate associations between prespecified factors and both methodological quality and completeness of reporting. Main Results We included 96 Campbell reviews, 46 were published between January 2011 and September 2014 (pre-MECCIR) and 50 between October 2014 and January 2018 (post-MECCIR). Twenty-two of 96 (23%) reviews were co-registered with Cochrane. For overall methodological quality, 16 (17%) reviews were rated as high, 40 (42%) as moderate, 24 (25%) as low and 16 (17%) as critical low using AMSTAR 2. Reviews published after the release of MECCIR had better methodological quality ratings than those published before MECCIR (odds ratio [OR] =6.61, 95% confidence interval [CI] [2.86, 15.27], p < .001). The percentages of reviews of high or moderate quality were 76% (post-MECCIR) and 39% (pre-MECCIR). Reviews co-registered with Cochrane were rated as having better methodological quality than those registered only with Campbell (OR = 5.57, 95% CI [2.13, 14.58], p < .001). The percentages of reviews of high or moderate quality were 77% versus 53% between co-registered and Campbell registered only reviews. Twenty-five of 96 reviews (26%) completely or partially reported all 27 PRISMA checklist items. The median number of items reported across reviews was 25 (IQR, 22–26). Reviews published after the release of MECCIR reported 2.80 more items than those published before MECCIR (95% CI [1.74, 3.88], p < .001); reviews co-registered on Campbell and Cochrane reported 1.98 more items than reviews only registered in Campbell (95% CI [0.72, 3.24], p = .003). An increasing trend over time was observed for both the percentage of high and moderate methodological quality of reviews and the median number of PRISMA items reported. Authors' Conclusions Many features expected in systematic reviews were present in Campbell reviews most of the time. Methodological quality and reporting completeness were both significantly higher in reviews published after the introduction of MECCIR in 2014 compared with those published before. However, this may also reflect general improvement in the reporting the methodology of systematic reviews over time or associations with other characteristics which were not assessed such as funding or experience of teams. Reviews co-registered with Cochrane were of higher methodological quality and more complete reporting than reviews only registered in Campbell.
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Comment cette classification a été obtenuedéplier
Prédiction machine sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Le volet Gemma est une étiquette directe du modèle pour chaque travail de la base, lue sur la notice réduite au titre. Le volet Codex est un classifieur appris des 10 348 étiquettes directes de Codex et calibré sur les taux pondérés de l'échantillon; les champs sans appui suffisant ne portent aucun appel Codex. Le mode candidate est l'union des deux volets; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont pas des étiquettes humaines.
Scores du classifieur distillé par catégorie (deux têtes)
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,608 | 0,884 |
| Méta-épidémiologie (sens strict) | 0,003 | 0,006 |
| Méta-épidémiologie (sens large) | 0,015 | 0,018 |
| Bibliométrie | 0,035 | 0,058 |
| Études des sciences et des technologies | 0,005 | 0,008 |
| Communication savante | 0,014 | 0,011 |
| Science ouverte | 0,006 | 0,008 |
| Intégrité de la recherche | 0,007 | 0,005 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,007 | 0,002 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découleClassification
machine, non validéePrédiction automatique; l’étiquette directe de Gemma et le classifieur distillé Codex s’accordent sur ce qui est montré ici.
Le détail, modèle par modèle et score par score, se trouve en fin de page sous « Comment cette classification a été obtenue ».