Bias busters: using the right risk-of-bias tools

Because systematic reviews support clinical decision-making, they are a crucial part of evidence-based practice. The actual value of a systematic review is determined by the quality of the studies that are included. To verify this quality, we need to look, amongst other things, at the risk of bias (RoB). Bias is the potential for a study’s results to be inaccurate because of problems with the study’s design, implementation, or reporting. It is possible that the recruitment procedures resulted in more severe cases being placed in the intervention arm, that each intervention group received different treatments (beyond the exposure being compared), that the outcomes were not consistently measured, or that there was not enough reporting. Because these problems can make an exposure or intervention appear better or worse than it actually is, we must look into them. Readers of the systematic review could be misinformed if bias is not taken into consideration. Different kinds of biases can arise in different study designs. For example, blinding and allocation concealment are two issues that randomized controlled trials (RCTs) may encounter. Confounding and selection bias are more likely to occur in observational studies. These differences require the use of specific RoB instruments tailored to the type of study. Over the past few decades, numerous RoB tools have been developed. According to a recent study, out of 226 RoB tools that were published between 1995 and 2023, 25% rated the quality of studies using a numerical scoring system (Siedler et al., 2025). According to research, numerical scores can be deceptive and untrustworthy; they do not account for the importance of relevant study elements, such as randomization or dropout rates in RCTs. Therefore, since numbers do not accurately represent the various ways that study quality should be assessed, the US Evidence-Based Practice Program and Cochrane handbooks recommend against using tools that rely on numerical scoring (Viswanathan et al., 2018; Boutron et al., 2024). An open-source database called LATITUDES Network (https://www.latitudes-network.org/) compiles tools for evaluating the quality of studies used in clinical evidence synthesis. The LATITUDES Network identifies the most appropriate key tools for assessing RoB in systematic reviews. These crucial tools (i) focus on the RoB; (ii) provide a way to determine the overall assessment of RoB or the RoB in a particular domain; (iii) were created in collaboration with stakeholders from different disciplines; and (iv) do not rely on summary numerical quality scores. As a result, many of the tools that were used in the past 10 years are now regarded as less appropriate for use as RoB tools. Examples of less appropriate tools are Newcastle-Ottawa Scale (NOS) (either for cohort studies or case-control studies), AMSTAR (A MeaSurement Tool to Assess systematic Reviews) and AMSTAR-2, and Joanna Briggs Institute (JBI) for RCTs. At Human Reproduction Update, we would like to promote the use of the LATITUDES Network key tools. Table 1 presents a selection of RoB tools for systematic reviews of study designs that are most commonly published in our journal. Key risk of bias tools for different study designs. Approved by, and awaiting addition to, LATITIUDES (see Riley et al., 2019). COSMIN ROB for PROMs, Consensus-based Standards for the selection of health Measurement INstruments Risk Of Bias for Patient-Reported Outcome Measure; CRIME-Q: Critical Appraisal of Methodological (technical) Quality, Quality of Reporting and Risk of Bias in Animal Research; JBI, Joanna Briggs Institute; PROBAST, Prediction model Risk Of Bias ASsessment Tool; QUADAS, QUality Assessment of Diagnostic Accuracy Studies; QUIPS, QUality In Prognostic Studies; RCT, Randomized Controlled Trial; RoB, risk of bias; ROBINS-I; Risk Of Bias in Non-randomized Studies—on Interventions; ROBINS-E, Risk Of Bias in Non-randomized Studies—of Exposures; SYRCLE RoB: SYstematic Review Centre for Laboratory animal Experimentation Risk of Bias; ROBIS: risk of bias in systematic reviews. Key risk of bias tools for different study designs. Approved by, and awaiting addition to, LATITIUDES (see Riley et al., 2019). COSMIN ROB for PROMs, Consensus-based Standards for the selection of health Measurement INstruments Risk Of Bias for Patient-Reported Outcome Measure; CRIME-Q: Critical Appraisal of Methodological (technical) Quality, Quality of Reporting and Risk of Bias in Animal Research; JBI, Joanna Briggs Institute; PROBAST, Prediction model Risk Of Bias ASsessment Tool; QUADAS, QUality Assessment of Diagnostic Accuracy Studies; QUIPS, QUality In Prognostic Studies; RCT, Randomized Controlled Trial; RoB, risk of bias; ROBINS-I; Risk Of Bias in Non-randomized Studies—on Interventions; ROBINS-E, Risk Of Bias in Non-randomized Studies—of Exposures; SYRCLE RoB: SYstematic Review Centre for Laboratory animal Experimentation Risk of Bias; ROBIS: risk of bias in systematic reviews. Using the most appropriate RoB tools improves the reliability and transparency of systematic reviews. Researchers can provide more accurate estimates of the certainty of the intervention or exposure effects. Furthermore, comprehensive RoB analyses may highlight areas of reproductive medicine with weak- or poor-quality evidence, helping the focus of future studies. We kindly request that authors clearly specify the RoB tool used in their proposals, abstracts, and methods section of the main manuscript for submitted systematic reviews. We will actively encourage authors to use the key RoB tools. For example, our latest systematic review used RoB for RCTs and ROBINS-I for non-randomized studies (Bülow et al., 2025). Visit the LATITUDES Network to learn more and gain access to a vast collection of RoB resources.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.