The benefits and challenges of conducting an overview of systematic reviews in public health: a focus on physical activity

The ultimate aim of Cochrane systematic reviews is to inform policy and practice decisions for better health outcomes. However, due to the increasing numbers of scientific publications, wading through the available evidence of both individual studies and systematic reviews can be challenging and overwhelming even for avid authors and readers. This paper briefly describes the first overview (a systematic review of reviews) of the Cochrane Public Health Group (CPHG) in development and proposes a way forward for the methodologies under consideration. Vast is the number of single studies being published every year relevant to public health. Collectively this presents an overwhelming challenge for those engaged in public health decision making both at a practice and policy level. Over the past 21 years, The Cochrane Collaboration1 has continuously improved the methodology for conducting rigorous systematic reviews, as well as compiling and publishing nearly 6000 systematic reviews that span the health care continuum in The Cochrane Database of Systematic Reviews. Systematic reviews are now a standard and well-recognized research tool, employing robust scientific methods in answering a focused question through comprehensively searching and identifying relevant studies, assessing risk of bias, summarizing and then interpreting all the available research.2 It is now well recognized that systematic reviews play an important role in identifying the interventions that are potentially effective, those potentially ineffective, those that are harmful, and those where current evidence is insufficient. Whilst the recognition and role of systematic reviews within clinical disciplines such as pregnancy and childbirth3 is widespread and integrated in clinical care, systematic reviews addressing preventative, upstream and population-level interventions are increasing in number, and influence,4 but arguably have not yet reached a state of integration in public health policy and practice. The numbers and span of relevant reviews is increasing: 800 systematic reviews relevant to public health decision making are on The Cochrane Database of Systematic Reviews. Public health professionals, however, have reported challenges in identifying public health relevant Cochrane reviews both when searching The Cochrane Library or other electronic databases, such as PubMed.5 In addressing this challenge, Health Evidence™ was developed (http://www.healthevidence.org). Health Evidence™ is a freely accessible, online registry of published systematic reviews published since 1995, which evaluate the effectiveness of public health interventions.6 All reviews included in Health Evidence™ have been assessed for methodological quality by two independent raters and this rating is provided in the site for each review. It includes all Cochrane reviews relevant to public health as well as systematic reviews which are published outside The Cochrane Library. While a resource such as Health Evidence™ assists public health decision makers to quickly find evidence related to the effectiveness of public health interventions, new challenges with respect to the evidence are emerging. For example, in the past 15 years, the magnitude of research and systematic reviews being published related to public health has increased dramatically. Some public health topics, chronic disease prevention for example, now have hundreds of systematic reviews with more being published every month. Recent searches of Health Evidence™ for high-quality reviews published since 2008 on interventions targeted at physical activity and nutrition, yielded 272 and 268 reviews, respectively. The challenge now for public health decision makers is to determine which of these reviews is best to answer their question. The issue becomes even more challenging if the results of reviews conflict, as is often the case when inclusion criteria and methods vary despite common overall topics and titles, leading to quality heterogeneity, with 582 of the 3761 reviews rated in Health Evidence™ as being of weak methodological quality. Health Evidence™ does facilitate search refinement with features such as topic focus, and the filtering of results for quality and year of publication. However, should multiple reviews for the same intervention be identified, users are then presented with the challenge of deciding which to use, or developing strategies for summarizing results across reviews. The Cochrane Collaboration, through regular updating and the use of rigorous methods in systematic reviews, aims to continue to strive for leadership and rigorous methodological inquiry.7 High-quality, efficient approaches to summarize results across reviews to support evidence informed public health decision making is now urgently called for, with a particular lens for public health that accounts for the complexity and contextual challenges presented, as a result of the amalgam of content that falls under the public health umbrella, scope and paradigms. Overviews of systematic reviews, also known as umbrella reviews, are systematic reviews of reviews and seek to provide an overview on a topic, rather than focus on a single intervention.2 Overviews summarize the evidence across the plethora of intervention providers, settings, type, quality and time. The primary differences between reviews on the same topic can also be explained within an overview. Generally the scope of systematic review topics can be described as lumped—being broad, or spilt—being narrow. Lumped reviews aim to identify the common generalizable features within similar interventions. During lumping, minor differences in study characteristics become less important.8 For an overview of systematic reviews, this principle applies as the overview question is ‘lumped’ and thus broad, allowing for generalization. In our experience, overviews can influence a decision maker's conceptions about the quantity of evidence that is available to inform decision making. For example, in early 2013, the National Collaborating Centre for Methods and Tools conducted a workshop in which nearly 90 public health decision makers (practitioners and policy makers) came together from across Canada to learn about current evidence on the effectiveness of various childhood obesity prevention interventions, with the intent of using this evidence to inform policies and programmes.10 Using audience response technology, we assessed the extent to which participant's perceptions changed regarding evidence related to childhood obesity prevention over the course of this 1 day workshop. Participants were asked to rate on a 4-point scale at the beginning of the day and again at the end of the day, their confidence in identifying evidence-based obesity prevention programmes. Overall, participants were significantly (P < 0.00) more confident in identifying evidence-based programmes at the end of the day in comparison to the beginning of the day (67% had ‘some confidence’ or were ‘fully confident’ at beginning of the day versus 94% having ‘some confidence’ or being ‘fully confident’ at the end of the day). Furthermore, perceptions also changed significantly with respect to the amount of useful evidence in systematic reviews to inform decision making. Participants reported less useful evidence from systematic reviews was available following the workshop than prior (59% of the audience changed their perception, 47% less evidence than thought, 12% more, Z = −3.835, P < 0.001). In total, 94% of the participants agreed with the findings of the presentation and only 6% disagreed. These data show that, generally, the public health decision makers in attendance were willing to accept evidence on childhood obesity prevention that had been systematically and transparently summarized. However, what is unclear is whether changes in perception of the findings of this overview resulted in significant changes in public health policies and programmes. It is generally recognized that the goal of producing an overview is to aid decision makers.11 Lavis12 reports that overviews are valuable in outlining the range of policy and programme options which are available. Health departments and agencies commission rapid overviews on highly ‘lumped’ topics, asking questions like ‘What physical activity and nutrition programs have the greatest impact on health outcomes?’ Being able to answer a ‘lumped question’ with a list of potentially effective interventions is an achievement, but translation to the community setting for population impact is imperative. To inform policy, the interest in an overview is likely to be greatest at a state, provincial or national level. Thus, applicability and transferability of the potential interventions are an important step in the assessment of usefulness,13 taking into consideration an organization's capacity to undertake an intervention, community preferences and determines whether an effect similar to the studies is achievable. Evidence from overviews can also be translated to challenging contexts,14 such as disadvantaged populations. Overviews also have limitations.11 Like systematic reviews, overviews are only as good as the studies which they contain. Accordingly, overview reviewers need to consider the limitations not only of the systematic reviews, but also have an understanding of the limitations of the primary studies contained therein. Clearly there is a need for The Cochrane Collaboration to commence the production of overviews of public health interventions. As was the case earlier with systematic reviews, overviews require the development of new methods for their production9 which have a significant variability in the methods of clinical overviews in the literature.14 Cochrane reviews are well known for having a core set of methods, to minimize variability, which are described and regularly updated in the Cochrane Handbook.2 This core set is then carefully adapted by some review groups presented as guidelines to fit the research methods generally observed for studies in that field and/or topic area.15 With the emergence of overviews within The Cochrane Database of Systematic Reviews, Cochrane has sought to provide guidance to the production of overviews,2,16 following similar steps and methods to systematic reviews of interventions; but with broader scope and wider ranges of included study designs. Public Health Overviews will follow this pathway, adopting recommendations that exist for other content areas, but responding to the emergence of the need to pay keen attention to complexity, systems, frameworks for levels of intervention implementation and rigour relevant to best available evidence. In the following we made recommendations on the methods for conducting rigorous overviews of public health relevant reviews. Statement of the question: in developing the statement, systematic reviews of public health intervention evidence are developed with PICO-T (Population, Intervention, Comparison, Outcome, Type of study). Similar to systematic reviews, overviews begin with a focused question, but where a review would include a paired comparison of an intervention, the overview question is likely to encompass several interventions. Overviews in public health are likely to be conducted on topics relevant to potentially modifiable population-level burden of disease. Selection criteria: the selection criteria of a review define the study designs for both inclusion and exclusion. Public health requires the usage of best available study designs (e.g., controlled before–after, cluster trials, controlled post-intervention and uncontrolled before–after).4 The basis of systematic reviews is individual studies, whereas the basis of overviews is systematic reviews.9 Sources and searching: in preparing systematic reviews, locating and retrieving all the relevant primary studies is extensive with a high volume of screening. To date, public health systematic reviews have required a broader and sophisticated search strategy covering a greater number of bibliography resources and databases reflecting the additional sectors and settings within which relevant research may be conducted and published than that for clinical interventions.17 When conducting an overview relevant for public health, we propose that Health Evidence™, as a current comprehensive database of reviews, be searched for systematic reviews as this will identify relevant published reviews as well as provide an assessment of the methodological rigour of each review. Overview protocols should specify dates for inclusion and we propose that those undertaking overviews relevant to public health may be justified in searching for overviews no further back than 10 years, or appropriate, relevant to societal and technological changes. Debate has emerged as to the need to search and complement the evidence in current systematic reviews with additional primary studies. Although this appears to be useful, it does add a complexity to the search and synthesis process of the overview. It may ensure that users and stakeholders are confident that all evidence has been identified and, simultaneously, may render the user more confused.16 This is definitely an area in which understanding of the context for the content will be essential—for some areas, the existing reviews may be comprehensive—in others the exclusion of best available study designs beyond RCT's in earlier reviews may render earlier reviews incomplete. Furthermore, the methodological scope and inclusion of data that helps public health users with questions regarding equity, differential effectiveness or economics has only been embraced more widely more recently. Quality assessment of evidence: systematic reviews require assessment of the risks of bias in duplicate. Likewise, overviews require assessment of the quality of evidence;9 however, the tools used in the overviews must be compatible with the study design used in the primary studies, which in this case may differ from clinical interventions. For example, AMSTAR, a tool with good validity and reliability in clinical settings,18 appears to have limited application to public health interventions. Analysis: the Cochrane Handbook indicates overviews should provide their synthesis at the review level.2 Although systematic reviews of primary studies may undertake a meta-analysis where data are combinable, the emphasis of an overview is the presentation of the reader with the major conclusions of the reviews in accordance with the question.9 In public health, the emphasis should be to present evidence from reviews with the intent of informing public health policy. The CPHG recognizes the demand for a coordinated approach to providing reviews that enable those needing evidence to inform responses to the growing burden of disease prevented reducing consumption of unhealthy dietary patterns, improved nutrition and related environmental contexts and lifestyle behaviours such as increasing physical activity and is working with authors to increase useful, relevant, priority reviews. It is well established that physical activity is associated with enhanced health and reduced risk of all-cause mortality19–22 and physical inactivity has been estimated to account for 6% of global deaths.21 Due to the socioeconomic costs, in particular the financial burden on healthcare systems associated with physical inactivity and obesity, hundreds of original studies and systematic reviews examining the effectiveness of interventions to increase physical activity are published annually. This volume of research has made decisions regarding investment in the most efficient or beneficial interventions difficult and time consuming for policy makers. Synthesizing the current evidence base regarding physical activity is essential in a public health setting. As a result, ‘Public health interventions for increasing physical activity in adults, adolescents and children’, soon to be published, will examine interventions which are used to increase physical activity rather than treat a disease or health condition. This overview will include systematic reviews where strategies are employed for the stated purpose to increase physical activity, time spent exercising or decreasing sedentary behaviour to improve health and well-being of children, adolescents and adults. The overview will include all interventions for physical activity within the public health and health promotion context intending either, where the focus is to directly or indirectly improve physical activity at a population level, rather than those targeted solely at individuals with particular disease conditions, with the exception of where the population is described as obese. These interventions may operate at the level of community, systems, policy and legislation. This overview will be the first of a succession of overviews by the CPHG and thus we would urge the public health community to engage with the CPHG as we develop the methods, hear the needs from users, and inform the development of new overviews to ensure the process for using the investments by authors of reviews is efficient, relevant and reliable. The explosion in systematic reviews of interventions has responded to the need to efficiently synthesize and summarize high-quality evidence for decision makers, but the sheer abundance has now rendered the context a challenging playing field for users of evidence who would like to efficiently engage with a wide range of topics relevant to a complex public health problem—whether that be mental health and parenting or physical activity and noncommunicable diseases. Overviews of systematic reviews serve a systematic approach to providing a higher level synthesis of evidence to broad public health issues. Methods for overviews need the same vigilance and scrutiny for bias and relevance that were first developed for reviews themselves. This paper provides a brief summary of the issues associated with the first Overview of the CPHG's, and provides an invitation for public health users of evidence to engage with us to ensure the practical complexities for users and readers are addressed in the processes and products. New Cochrane protocols and reviews of interest to health promotion and public health stakeholders in May/June 2014 of The Cochrane Library (*denotes CPHG review/protocol). Zinc supplementation for preventing mortality, morbidity, and growth failure in children aged 6 months to 12 years of age. Vitamin D supplementation for prevention of cancer in adults. Yoga for the primary prevention of cardiovascular disease. Extra fluids for breastfeeding mothers for increasing milk production. Hormonal contraception for women exposed to HIV infection. *Unconditional cash transfers for reducing poverty and vulnerabilities: effect on use of health services and health outcomes in low- and middle-income countries. Delivery arrangements for health systems in low-income countries: an overview of systematic reviews. Implementation strategies for health systems in low-income countries: an overview of systematic reviews. Omega-6 fatty acids for the primary prevention of cardiovascular disease. Mobile phone-based interventions for improving contraception use. Sun protection for preventing basal cell and squamous cell skin cancers. Interventions for tobacco use cessation in people living with HIV and AIDS. Vitamin C supplementation for the primary prevention of cardiovascular disease. Multiple micronutrient powders for home (point-of-use) fortification of foods in pregnant women. Multiple risk factor interventions for primary prevention of cardiovascular disease in low- and middle-income countries. The CPHG acknowledges the financial support of the National Health & Medical Research Council of Australia (NHMRC), The Jack Brockhoff Foundation and the Victorian Health Promotion Foundation (VicHealth). Heath Evidence is funded in part by the National Collaborating Centre for Methods and Tools.

Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.