The Quality of Systematic Reviews in Hand Surgery

Sir: The strength of systematic reviews relies inherently on the methodology of the included studies, which inevitably to attain a high level of evidence must be reliable and transparent. We read with great interest a study conducted by Momeni et al.,1 in which the authors evaluated the methodologic quality of systematic reviews in the hand surgery literature using the validated critical appraisal tool AMSTAR.2 The authors concluded that the median AMSTAR overall score of systematic reviews in the hand surgery literature was 7 of 11, which corresponds to a fair to good score. However, we noted that the study has two major underlying issues. First, the authors stated that the majority of the evaluated systematic reviews provided only a list of included studies, but none provided a list of excluded studies. The fifth criterion in the AMSTAR tool states that fulfillment of this criterion requires “a list of included and excluded studies should be provided.”2 However, the authors stated that providing solely a list of included studies was sufficient to satisfy this AMSTAR criterion. Modifying the original criterion of AMSTAR can potentially negatively influence the validity of the AMSTAR tool and thus lead to misleading results. Second, the last criterion of the AMSTAR tool, which consists of evaluating conflict of interest, was reported in 81 percent of the included systematic reviews. This criterion states that “potential sources of support should be clearly acknowledged in both the systematic review and the included studies.”2 However, the authors consider acknowledgment of source of support in systematic reviews only and not the included studies. Popovich et al. found that more than half of Cochrane reviews did not assess conflicts of interest in their included studies, which resulted in a downshift of final score.3 Although we acknowledge this is a difficult criterion to achieve, we feel that this criterion should not be counted as “yes” if not fully met, as it could result in a misleadingly high-quality score. We conducted a pilot systematic search, which involved a MEDLINE search of all hand surgery systematic reviews from 2003 to 2013 published in Plastic and Reconstructive Surgery. The search identified a total of 90 articles. After screening titles and abstracts, 76 articles were excluded. Eventually, 14 hand surgery systematic reviews were identified. We applied the same quality assessment tool, AMSTAR, on these studies, focusing on the two aforementioned criteria discussed previously. We found that only 21.4 percent (versus 88.1 percent) of studies provided a list of included and excluded studies and none of the studies (versus 81 percent) provided a clear acknowledgment of potential source of support in the systematic review and the included studies. Researchers and clinicians have the responsibility of providing the highest possible level of evidence, given the significant implications on clinical and policy decision-making. When we assess the literature, the aim is to ensure that the best quality and highest level of evidence is being used. We should avoid overstating any results, as this could affect the progress of the improvement of quality in our surgical specialty by giving a false encouraging image. We find that this unjustified modification of the two AMSTAR criteria has led to inflated results that may have influenced the final conclusion of the study in a misleading manner. Although we feel that these modifications will not help in the improvement of the quality of systematic reviews in the hand surgery literature, the authors may have a reasonable justification for modifying these two AMSTAR criteria. DISCLOSURE The authors have no financial interest to declare in relation to the content of this communication. This letter did not require any funding. Osama A. Samargandi, M.D., M.H.Sc. School of Population and Public Health Faculty of Medicine University of British Columbia Vancouver, British Columbia, Canada Haroon Hasan, B.Sc., M.P.H. Department of Radiation Oncology British Columbia Cancer Agency Vancouver, British Columbia, Canada and Pediatric Oncology Group of Ontario Toronto, Ontario, Canada

Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.