Meta-analysis

Most meta-analysis methods are variations on a weighted average of the effect estimates from the individual studies [4]. The PRISMA 2020 reporting guideline describes it as a technique used when each study's effect estimate and its variance are available, yielding a quantitative summary of results [1]. It is one form of statistical synthesis; PRISMA also lists others, such as combining P values and vote counting based on the direction of effect [1].

The Cochrane Handbook lists potential advantages: better precision, answers to questions that single studies did not pose, and a way to settle conflicting claims [4]. The pooled result is often drawn as a diamond at the bottom of a plot, and the Handbook warns that it can be very misleading if the review question was not carefully formulated first [4]. Variation across studies, called heterogeneity, must be considered [4].

The Handbook says meta-analyses can mislead seriously when study designs, biases within studies, variation across studies and reporting biases are not considered carefully [4]. A combined analysis of 146 meta-analyses concluded that reviewers should routinely assess risk of bias in trial results and report meta-analyses restricted to trials at low risk of bias, either as the main analysis or alongside the broader one [3].

A Cochrane review of trials comparing placebo with no treatment in clinical populations shows the judgment involved: in 158 trials with continuous outcomes (10,525 patients), effects varied between small and large trials, and the authors called pooling all of them a questionable procedure [2].

When a supplement page cites a meta-analysis, it is worth checking how many trials were pooled, who took part, and whether the authors flagged heterogeneity.

• Systematic review: the structured search that usually comes before a meta-analysis.
• Effect size: the number a meta-analysis pools.
• Confidence interval: how precise the pooled number is.
• Risk of bias: how trustworthy each pooled trial is.