Are We Ready to Accept Follow-up Rates of 50% in Orthopaedic Research?

Commentary Research is important, but bias is everywhere1. Bias is defined as systematic deviation from the truth. Bias is different from non-systematic deviation from the truth, which is simply noise1. We seek research free from bias to answer clinical questions. The results of research are used by surgeons to make clinical decisions and to thereby improve care. The problem with bias is that it can lead to flawed conclusions on study questions1. Flawed conclusions can lead to wrong decisions and poor care. Reduction of bias is integral to the design and analyses of all research. In terms of design, holding all other things constant, randomized studies are generally more valid than nonrandomized studies, prospective studies are generally more valid than retrospective studies, and controlled studies are generally more valid than uncontrolled studies. These generalizations are the basis of Levels of Evidence2. Higher levels of research are generally freer from bias and more valid. However, there are well-performed Level-II studies that are less biased than a poorly performed Level-I study. This is because there are upwards of 100 described sources of bias, many of which can affect the results of any study1. Of the many sources of bias, a particularly important issue is how, when, and what percentage of the study population is assessed for study outcomes. Gaining a sufficient number of eligible patients to consent to participate is the first challenge in meeting sample size requirements. The final challenge is ensuring that a sufficient number of patients are available to ascertain study outcomes. Given that almost no studies achieve 100% follow-up, this potentially introduces bias1. The importance of loss to follow-up is dependent on whether patients are missing at random or not missing at random3. If patients are missing at random, then their missing data are simply noise. However, if the loss to follow-up is not random, then the frequency and how much missing patients deviate from the true population can affect the study results. For example, if patients with worse outcomes seek care somewhere else and are lost to follow-up, this can falsely elevate the rates of positive outcomes. The acceptable rate of missing data has been arbitrarily set at 20%. More than 20% loss to follow-up is generally believed to be problematic, whereas <5% loss to follow-up is generally believed to pose a minimal risk of bias4. However, as stated above, the risk is dependent on the amount of deviation from the truth, so that any threshold short of 100% follow-up cannot guarantee unbiased results. In their study, Spindler et al. evaluate different methods of follow-up and the effect on patient-reported outcome measures (PROMs). The traditional methods of in-person follow-up are being replaced in many studies by automated follow-up such as text and/or emails. Automated methods are appropriate and useful when patients do not need to be seen in person and the outcome such as a PROM can be validly obtained by automated methods. Manual methods such as telephone calls or in-person visits are used to achieve a higher percentage of follow-up, but these methods are expensive. The study by Spindler et al. demonstrated that manual follow-up increased response rates by >20%. However, despite differences in baseline characteristics between those receiving manual follow-up and those receiving automated follow-up, there was no clinically meaningful or significant difference in PROM scores. Thus, if the study had used a solely automated technique, a loss to follow-up of approximately 50% would not have affected the study outcomes. This research could have important implications for reducing the cost and complexity of clinical research. Boosting follow-up rates to get closer to 80% follow-up might not be needed. However, this is only 1 study in 1 setting, and the generalizability of the results to other settings is uncertain. Thus, the study must be replicated in other settings, with other outcomes and other types of interventions. Although exciting, orthopaedics is not yet ready to accept follow-up rates of 50% based on a single study in a single setting.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.