JBJS and Evidence-based Orthopaedics

The Journal of Bone and Joint Surgery (JBJS), as the leading journal in orthopaedics, has and will continue to publish some of the most important studies in orthopaedics. Evidence-based medicine has the potential to dramatically change how best practice is determined in pediatric orthopaedics.1 Since 2000, JBJS has promoted the concept of evidence-based orthopaedics with specific changes to the journal in response to the evidence-based medicine movement. The tagline of JBJS is “excellence through peer review.” The journal relies on peer review by orthopaedic surgeons and other consultant reviewers who evaluate and, through this process, improve submitted and subsequently published studies. As discussed in more detail below, the standards for studies have continuously improved since the establishment of JBJS>100 years ago. For example, whereas in the distant past simply providing pictures of changes in foot shape might have been sufficient as an outcome measure, now articles provide detailed descriptions of patients and their treatment, with multiple outcome assessments including objective clinical assessments and subjective patient appraisals with reliable and valid health status instruments. Study design has also improved to provide more valid answers to clinical questions. Evidence-based orthopaedics builds on this tradition by using individual expertise and the best evidence to make the best decisions with individual patients. EVIDENCE-BASED ORTHOPAEDICS SECTION The changes to the journal began under the leadership of then Editor-in-Chief, James Heckman, with a new section entitled “Evidence-based Orthopaedics.”2 On a quarterly basis, research librarians at McMaster University scour 40 journals for Level I evidence of interest to orthopaedic surgeons. These Level I studies are reviewed by the Associate Editor for Evidence-based Orthopaedics and the Deputy Editor for Outcome Research to identify 3 studies of the greatest impact and most relevance to orthopaedic surgeons published in journals other than the JBJS. In addition to the structured abstracts describing the findings of these studies, a commentary is solicited from a content expert, both to evaluate the study and to place the study into context for the practicing surgeon. The structured abstract and commentaries are published quarterly to apprise orthopaedic surgeons of the latest developments using best evidence. This section was specifically devoted to busy clinicians with real world questions. LEVELS OF EVIDENCE (LOE) The next major change to JBJS was the introduction of LOE.3,4 Surgeons use many sources of evidence to make decisions with individual patients. The medical and surgical literature is the major source of best evidence. LOE3 are a hierarchical rating system for classifying study design and quality. A working group from the American Academy of Orthopaedic Surgeons (AAOS) and JBJS developed a streamlined and explicit LOE. Each study published in JBJS is classified in one of 4 study types; therapeutic, prognostic, diagnostic, and economic or decision modeling. Authors submitting manuscripts are required to indicate a LOE for the primary research question. In the JBJS, Instructions to Authors is a table with the 4 categories and 5 levels with criteria to help the authors assign an appropriate LOE (Table 1). The LOE rating is reviewed by the Associate Editor for Evidence-based Orthopaedics and is included in the abstract of every published article.5TABLE 1: Levels of Evidence for Primary Research Question*The addition of LOE to published articles allows authors, reviewers, and readers to become familiar with the concept of LOE and research design. More importantly, the rating was meant to place a clinical study into context for the reader. Higher LOE are generally more convincing to surgeons in resolving clinical dilemmas. In introducing LOE to JBJS readers, several caveats were emphasized. First, LOE provides only a rough guide to study quality, and a full assessment of study quality requires a complete critical appraisal. Second, Level I studies are not always possible in orthopaedics. There is always a role to play for other types of evidence in making clinical decisions. Finally, no answer to a clinical question can or should be based on 1 study, as no single study provides a definitive answer. Including LOE with the abstract provides a rough guide to the reader on study quality with the intent that Level I and Level II studies would have greater interest for surgeons and provide more convincing evidence in making clinical decisions. The interobserver reliability of the ratings for LOE in JBJS has been shown for those with specific training in clinical epidemiology, to have a reliability of coefficient of 0.99 where 1.0 indicates perfect agreement. Those without specific training still had high correlations of 0.6 to 0.7.6 LOE have become a popular concept now with similar processes occurring in many other journals, such as Journal of Pediatric Orthopedics, and for abstracts submitted to professional societies.7 GRADES OF RECOMMENDATIONS The next change in the journal was the introduction of grades of recommendation.8 As noted, above answers to clinical decisions are not based on a single study but need to incorporate all prior studies that bear relevance. The concept of grades of recommendation, although not new, was adapted by a working group of the AAOS and JBJS and tied directly to the JBJS’ LOE. Beginning in 2005, grades of recommendation were incorporated into review articles published in JBJS. Thus, most recommendations within Current Concepts Reviews in JBJS have an attached grade of recommendation that places the recommendation in context for the practicing orthopaedic surgeon. In addition to their use in review articles, grades of recommendation are used as part of the AAOS’ guideline development process and have been introduced into several textbooks as a way of judging the strength of evidence in support of particular treatment recommendations.9 Grades of recommendation are meant to be a quick aid to surgeons in appraising the value of a recommendation. Searching the literature takes an enormous amount of time and frequently is incomplete. Thus, review articles are a frequent and valuable resource for orthopaedic surgeons. However, without some sense of the quality of the literature used to support a treatment recommendation, it is difficult for orthopaedic surgeons to determine how that recommendation should influence their practice (Table 2). Grade A recommendations are something that surgeons should do. Grade B recommendations are something that surgeons probably should do.10 Grade C recommendations are something that surgeons could do. Grade I recommendations provide no guidance one way or another due to conflicting or insufficient evidence. When there is not good evidence, patients can be apprised about the lack of good evidence, and decisions have to be made with surgeons, relying on professional judgment in concert with patient preferences.TABLE 2: Grades of Recommendation for Summaries or Reviews of Orthopaedic Surgical StudiesTRENDS IN ORTHOPAEDICS A review of the number of randomized clinical trials in JBJS American before 2000 showed that approximately 5% of the studies were Level I.11 The JBJS has shown a dramatic change in the type of manuscript that is published, with jumps from almost no Level I studies in 1975 to almost 25% in 2005. Level I, Level II, and Level III evidence studies now comprise >50% of published articles in JBJS,with a commensurate drop in case series or Level IV evidence studies to less than 50%.12 The concept of evidence-based orthopaedics is gradually increasing in orthopaedics. A survey of orthopaedic surgeons demonstrated that those of younger age, particularly those working in academic centers with higher degrees, were much more familiar and comfortable with the concepts of evidence-based orthopaedics.13 Although our most classic and most frequently cited articles may not be the highest quality, LOE ratings are correlated with the frequency of citation,14 suggesting as indicated above, that higher LOE are more relevant to practicing orthopaedic surgeons. There are innumerable conditions and treatments available in orthopaedics, many of which are not either amenable or not yet been studied by randomized clinical trial. Observational studies have a role to play, particularly when presenting brand new treatment options or with conditions with universally poor or positive outcome. As noted above, LOE are only 1 aspect of study quality and a full understanding of study quality requires a complete critical appraisal. Even with peer review, Dr John Ioannidis has suggested that there are significant flaws in many published studies. Dr Ioannidis has concluded that all Level IV studies will ultimately be shown to be incorrect and approximately 40% of randomized clinical trials will eventually be shown to be incorrect.15 CONCLUSIONS JBJS has taken a leadership role in advancing the concepts of evidence-based medicine. The number of randomized clinical trials is on the rise in the orthopaedic literature. Although many bemoan the lack of high-quality evidence, this trend is clearly changing, and with time we will have more Level I and Level II evidence to inform our clinical decision making. Orthopaedic journals have a major role to play in influencing treatment recommendations, and a move to higher quality studies will further help busy clinicians make difficult decisions with patients.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.