Confidence interval

The CONSORT reporting guideline asks trials to give a confidence interval with each effect estimate, describing it as a central range of uncertainty for the true effect [1]. It is customary to report a 95% interval: if similar studies were repeated many times, about 95 of every 100 intervals calculated this way would include the true value, provided the analysis assumptions hold [1][2].

A common misreading is that one specific 95% interval has a 95% chance of containing the true effect [2]. Greenland and colleagues explain that a reported interval is simply a range between two numbers: the 95% describes how often intervals computed from very many studies would contain the true effect [2]. The interval is also computed from many assumptions, and violations of those assumptions can shape the result [2].

The interval shows how precise an estimate is. CONSORT notes that after a trial is done, confidence intervals indicate its power better than a post hoc power calculation [1].

An interval that reaches the point of no difference deserves attention. In a Cochrane review of trials comparing placebo with no treatment in clinical populations, where a relative risk below 1 favored placebo over no treatment, the pooled relative risk for patient-reported outcomes was 0.93 with a 95% CI of 0.86 to 1.00 [3]. For several conditions the authors found no statistically significant effect but noted that the confidence intervals were wide [3]. Greenland and colleagues list as a misreading the idea that a result with P > 0.05 shows that there is no effect [2].

So a supplement result described as "not significant" with a wide interval is uncertain, not a demonstration that the supplement does nothing.

• Effect size: the estimate the interval surrounds.
• Meta-analysis: pooling studies into one estimate with its own interval.