Risk of bias

Version 2 of the Cochrane risk-of-bias tool, called RoB 2, is the tool recommended for randomized trials in Cochrane reviews [3]. It is built on a fixed set of domains covering trial design, conduct and reporting [3]:

• bias arising from the randomization process
• bias due to deviations from intended interventions
• bias due to missing outcome data
• bias in measurement of the outcome
• bias in selection of the reported result

Each assessment applies to one specific result from a trial [3]. Within each domain, signaling questions draw out the relevant features of the trial, and an algorithm proposes a judgment of low risk of bias, some concerns or high risk of bias [3]. The overall rating for the result is the least favorable rating across the domains [3].

A combined analysis of three meta-epidemiological studies, covering 146 meta-analyses and 1346 trials, found that trials with subjective outcomes and inadequate or unclear allocation concealment showed exaggerated effect estimates (ratio of odds ratios 0.69), while trials with objective outcomes showed little evidence of bias [1]. The authors concluded that reviewers should routinely assess risk of bias and report meta-analyses restricted to trials at low risk of bias, either as the main analysis or alongside broader ones [1].

In a Cochrane review of trials comparing placebo with no treatment in clinical populations, the authors judged only 16 trials (8%) to be at low risk of bias [2].

When a supplement page cites a review, its risk-of-bias table shows how much of the evidence came from trials rated low risk and which domains caused concern.

• Randomized controlled trial: the design RoB 2 assesses.
• Double-blind: one way trials reduce bias in measuring outcomes.
• Pre-registration: the record used to check selective reporting.
• Systematic review: where risk-of-bias ratings are reported.