Evidence‐based medicine ‐ the promise, the reality

In 1993 I attended the RCOG's Annual Scientific Meeting in Hong Kong, where I listened to a presentation by Dr Richard Johansen, an obstetrician and gynaecologist from Stoke on Trent. Richard presented a free communication about a randomised controlled trial (RCT) of vacuum extraction that he had led, followed by a systematic review of trials of forceps and vacuum extraction, which included a meta-analysis with the new study incorporated. The systematic review was published on an electronic database called the Oxford Database of Perinatal Trials (ODPOT). I was impressed not only with the primary study (a randomised trial of an intervention in second stage is no mean feat) but also with the concept of the original research being considered alongside all the other research on this topic. And I was doubly impressed with the idea of an electronic database summarising all the studies. At the time this was a new approach. I spoke to Richard after his presentation and he explained that Dr Iain Chalmers, of the UK National Perinatal Epidemiological Unit was starting an even more interesting project and planned to publish systematic reviews of all health care. I was very interested and wrote to Iain asking if there was anyone interested in systematic reviews on treatments of endometriosis. This was just when electronic communication was beginning and so all of this was by old-fashioned airmail letters in stamped envelopes! I was very pleased when Iain promptly wrote back. He told me that no one had shown any interest in any of the gynaecology topics and suggested that we meet. I was able to arrange some funding to visit the UK in 1994 where I met up with Iain. Somewhat surprisingly he asked if I would like to lead the gynaecology topics for this new venture which was to be known as the Cochrane Collaboration. In order to prepare for starting a Cochrane group I arranged to do a sabbatical in Oxford in 1995. I spent some time at the UK Cochrane Centre and also at the Health Services Research Unit in Oxford. During this time I prepared an application for registering the Cochrane Menstrual Disorders Group, which involved two exploratory meetings of about 30 interested individuals. We were finally registered in late 1995 after I returned to New Zealand. Registration was only the start. We needed funding. All my initial applications were declined. The idea of funding work leading to systematic reviews was new and not particularly well understood. Eventually we started the project with grants of $40 000 from the Northern Regional Health Authority and $5000 from the Royal NZ College of Obstetricians and Gynaecologists. The Cochrane Menstrual Disorders Group had an official opening in 1996 with Professor Liggins of National Women's Hospital (NWH) and the University of Auckland, Professor Chris Silagy from the Australasian Cochrane Centre, Gary Henry (General Manager of NWH) and Dr Tony Baird (NWH Medical Director) all of whom were very supportive at the time. Our Head of Department, Professor Gillian Turner was also very supportive. In 1998 we added subfertility topics to our remit, as the Subfertility Group in Leeds was closing. In 2016 we changed our name and also celebrated 20 years of what is now the Cochrane Gynaecology and Fertility Group.1 Currently we are working with nearly 1000 authors and 34 editors and have nearly 300 published reviews and 40 protocols. Our current impact factor is 6.44. We also measure our impact according to use by guideline developers and we are pleased to say that the WHO, RCOG, ASRM, ESHRE, RANZCOG and NICE guidelines all cite our reviews. This term is attributed to Professor Gordon Guyatt, a professor at McMaster University in Hamilton, Ontario who suggested the term in 1992 as an alternative to the use of ‘intuition, unsystematic clinical experience and pathophysiologic rationale as sufficient grounds for clinical decision-making’.2 Professor David Sackett, also from McMaster University, went on, with others, to define evidence-based medicine (EBM) in the British Medical Journal as ‘the conscientious explicit and judicious use of current best evidence in making decisions about the care of the individual patient’.3 Guyatt and Sackett, again with others, at McMaster jointly wrote a series in JAMA ‘The User Guides to Medical Literature’, which were very valuable and instructive articles for the EBM movement.4 So where does Archie Cochrane fit into this story? He was a Scottish epidemiologist who led clinical trials on the health of Welsh mining communities and in 1960 became Professor of Tuberculosis and Chest Diseases at what is now Cardiff University. He was subsequently appointed Director of the Medical Research Council's Epidemiology Research Unit in Cardiff. Archie argued that health care should only be used if it had been shown to be effective in properly designed evaluations. He wrote in 1979 ‘it is surely a great criticism of our [medical] profession that we have not organised a critical summary by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials’.5 These words include some ground-breaking concepts on the need for synthesis, rigorous appraisal, systematic organisation, and keeping current – and all from an age before electronic databases, electronic mail, or even fax machines. In the same article Archie Cochrane awarded obstetrics and gynaecology (O&G) ‘the wooden spoon’, which is traditionally the prize awarded to whoever comes last in a race.5 Thus O&G was designated the least evidence-based of all medical specialties. This dubious accolade most likely resulted from instances in the field of O&G where practices were introduced without evidence and with potential harm. To name but a few: thalidomide for nausea in early pregnancy, estriol as a biomarker for fetal growth, antenatal stress tests, diethylstilbestrol for recurrent miscarriage, the widespread use of hormone therapy for prevention of long-term conditions such as heart disease, and removing ovaries at the time of hysterectomy. In 1985 Iain Chalmers took up the challenge, and between the years of 1985 and 1989, with Murray Enkin and Marc Kierse, prepared a series of summaries of research work based on the Oxford Database of Perinatal Trials.6 In 1989 these were published as the textbook Effective care in Pregnancy and Childbirth and was described by the British Journal of Obstetrics and Gynaecology as ‘probably the most important book in obstetrics to appear this century’. Archie Cochrane wrote the foreword, in which he withdrew the wooden spoon from O&G. The Cochrane Collaboration began in 1992. It pioneered a new way of working together. It was about collaboration and research, not competition; it was about teams of authors, inclusive of patients and non-medical experts, strong methodology and sharing of expertise and resources. The timing was important, as fast and efficient communication and dissemination was essential. By 1995 fast communication had arrived and the internet was taking off, electronic libraries were replacing volumes of Index Medicus, and design of electronic search strategies became a new science. There was also a software revolution underway with programs such as Powerpoint, Excel, and statistical packages, all available for desktop computers. Fast forward to today and Cochrane is a global organisation of 40 000 people, publishing more than 7300 reviews, has identified a million RCTs, and is generally considered the gold standard for evidence. It is currently the fifth most-cited medical journal in the world. In 2017, 2.2 billion people in 133 countries have one-click access to the Cochrane Library thanks to national licences such as we have in Australia, NZ and the UK, or through the WHO, which makes the Library available for low and middle income countries. There are three important tools for EBM. First, there are clinical trials which answer questions of treatment of effectiveness, which account for 85% of clinical decision making. Clinical trial design has improved, harms are increasingly reported, outcomes are more patient-centred, and trial designs are more pragmatic, reflecting everyday use. The second tool for EBM is systematic reviews. These are critical summaries of the body of evidence, which (crucially) incorporate quality assessment using formal methods to grade the quality of the body of evidence. Third, there are clinical guidelines, which facilitate implementation of the evidence. The evidence may be global but we need local decisions and evidence-based policies. Guidelines need a framework which takes into account the local context, resources and patient preferences. EBM has evolved over the past 20 years, with major improvements in clinical trial design and grading of evidence, and a stronger focus on dissemination of the evidence. We are now very well served with systematic reviews in O&G. In the Cochrane Library we have over 1500 such reviews (Table 1). Although it is a global enterprise, Australia and NZ have played an important role with about 20% of the Cochrane reviews having an Australian or NZ author. Let's turn to the future of EBM. How we are doing? There is some good news. For example, the reduction of perinatal mortality and morbidity from preterm birth is something to be proud of. US data from 1950 report that 90% of babies died if they were born weighing less than 1000 g, while today it is under 40%.7 There are various reasons for this, including implementation of the use of antenatal corticosteroids, use of magnesium sulphate for neuro protection, improvements in neonatal care (including use of surfactant), reduction in smoking during pregnancy, and improvements in antenatal care. Systematic reviews and guidelines, including our own RANZCOG guidelines, have been widely used and adopted.8,9 More good news is the reduction of unwanted pregnancies in New Zealand.10 Increased use of long-acting reversible contraceptives, including intrauterine systems and implants, may account for this decline. Systematic reviews support use of such devices in women following abortion, following pregnancy, and to prevent pregnancy.11-14 In NZ we have had a 28% reduction in abortions from 2007 to 201410 and in the Auckland region we have had a 37% reduction over the past 11 years.15 In 2016, nearly 90% of women received contraception at the time of the abortion, and 65% of those were long-acting reversible contraceptives.15 The management of heavy menstrual bleeding has also dramatically improved. The mainstay of treatment used to be hysterectomy. With the introduction of the levonorgestrel intrauterine device there are now several other options for women with heavy menstrual bleeding.16 Systematic reviews and NZ guidelines have supported use of non-hysterectomy options.17, 18 Another encouraging example comes from fertility treatments. In vitro fertilisation was a huge step forward and was introduced within 10 years in most developed countries. However, it came with the considerable downside of an increase in multiple pregnancies. By 2002 nearly one-third of pregnancies from IVF were multiple pregnancies (including high order pregnancies).19 In NZ and Australia we have now almost eliminated multiple births as the result of changing to single embryo transfer. The first trial was in 2002, published in the New England Journal and reported that live birth rates were comparable between double embryo transfer and single embryo transfers over two or three cycles.20 Since that time the number of multiple births has steadily reduced and we currently have <5% for NZ overall.11 So that was the good news but the reality in 2017 is also that some evidence is dismissed or ignored, and sometimes new innovations are introduced without evaluation. When new technology first appears, there is a phase of inflated expectations followed by disillusionment and eventually a slope of enlightenment. Meanwhile there is marketing of the product before the evidence from RCTs is available. Eventually there are more RCTs, summaries of RCTs, evidence about harms, and finally we come to the point of rational use. An example of this was the enthusiastic uptake of mesh for prolapse. This evidence for mesh in women with prolapse is the topic of a Cochrane review with 37 trials of over 4000 women.21 This review reported that women who had prolapse repair using permanent mesh were less likely to report symptoms of prolapse after surgery than women who had native tissue repair (10–15% with mesh vs 19% with native tissue). But the women who had mesh were also more likely to undergo repeat surgery for prolapse, incontinence or mesh exposure (composite outcome) (7–18% with mesh vs 5% with native tissue). Women who had mesh were also more likely to report de novo stress incontinence and more likely to have bladder injury. So how did we end up here? Mesh was introduced in the late 1990s and 2000s but the first trial was not published until 2001.21 The Food and Drug Administration (FDA) warnings were made in 2008 and 2011.22 Mesh continued to be implanted in spite of the warnings. It was not until 2013 that mesh use started to be reduced, and companies started to withdraw the product. The issue is currently attracting the attention of politicians and consumer groups and class actions are underway in Australia, the USA and the UK. There are parliamentary and senate enquiries in Scotland, Australia and NZ, a statement from the Accident and Compensation Commission (ACC) in NZ, and FDA reports. RANZCOG has a summary of all the responses and patient information.23 In late November 2017 the Therapeutic Goods Association removed mesh for prolapse in Australia.24 New Zealand followed with a similar announcement in December 2017.25 Devices need evaluation and regulation, but are currently not regulated in the same way as drugs. RCTs are needed to weigh up harms and benefits: this cannot be done by individual case series or by individual surgeons. We should not depend on a company marketing program for information or education. Clinicians need to seek evidence that a new intervention or device has been evaluated systematically and independently reviewed. Why do we embrace some new interventions so quickly while others are ignored? Possibly due to commercial marketing campaigns that are so well presented and easy to process. Yet we know that companies use dubious practices such as expedient trial design and presentation of data, suppression of unfavourable findings, and ghostwriting.26 A related issue is the financial connections between industry and doctors. On one hand, relationships between doctors and industry could be considered synergistic by allowing the development of improved treatments. On the other hand, payment (and other benefits) from industry to doctors may subtly shift the main objective of the collaboration from patient health to mutual benefits for doctors and industry. Conflicts of interest are also a hot topic.27 I suggest that obstetricians and gynaecologists need to be transparent about the receipt of financial benefits, including funding from industry, as such benefits may influence research outcomes and treatments that patients are offered. RANZCOG has a statement on EBM O&G – ‘RANZCOG strongly supports the principle that clinical decision-making should be based on the best and most relevant evidence available. It also recognises the NHMRC levels of evidence and associated grading of recommendations.’ However, many of the College statements are not based on evidence (even where there is evidence available) but are consensus statements. The reason for this needs exploring but it should be acknowledged that the work involved in preparing and updating evidenced-based guidelines is considerable and needs investment. Colleges also have an important role in reducing the impact of industry and in encouraging transparency.27 Industry sponsorship of medical societies and their educational events should be kept to a minimum and declared quantitatively in societies’ websites and scientific program brochures. Industry sponsorship of scientific meetings should not include the right to host educational symposia or speakers within the program, and should not include trainee education. All speakers should declare their conflicts of interest (COIs) at their meetings and these should be made public. Guideline groups should require all members to declare their financial COIs before meeting and such groups should exclude or limit participation of members with COIs. In the USA, the UK, Australia and the Netherlands this information is now published. Currently in Australia consumers can at least seek information about payments from pharmaceutical companies (the database does not include devices), but not in New Zealand.28 Some colleges have led the way. The Royal New Zealand College of General Practice no longer has commercial sponsors nor do they have input in the scientific program. Companies are allowed only in the trade area. Similarly, the RANZC of Psychiatrists (NZ branch) no longer has any industry sponsors or exhibits. So should RANZCOG consider being free of industry in both sponsorship and scientific programs of our annual meetings. We should also insist on industry-free trainee and subspecialty education. Finally, a utopian wish list from me. All evidence should come with a ‘trustworthiness’ score which would take into account any study and funding bias. Education should be free of industry influence. We should not be part of industry marketing plans and all financial COIs should be disclosed. Guideline developers should be properly acknowledged and compensated for their time, as guidelines are crucial and can save lives. Our patients deserve nothing less.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.