Systematic review
A systematic review is a review that uses explicit, systematic methods to find, select and combine the findings of studies that address a clearly formulated question.
What it means in practice
Systematic reviews aim to minimize bias by setting the research question and methods in advance, documenting them in a protocol, and basing findings on reliable research [4]. The Cochrane Handbook says the work should be done by a team with both subject and methods expertise, free of potential conflicts of interest [4].
The PRISMA 2020 reporting guideline asks authors to give a transparent, complete and accurate account of why the review was done, what they did, including how studies were identified and selected, and what they found [1]. A systematic review does not have to include a meta-analysis: PRISMA covers reviews with a statistical synthesis and reviews without one, for example when only one eligible study is identified [1].
The search itself is part of the record. A Cochrane review of placebo interventions, for example, lists each database with the period it covered, such as MEDLINE from 1966 to March 2008 [2].
Why it matters when reading nootropics research
A review can only work with what the included trials reported. In a study of 102 trials approved by Danish ethics committees in 1994-1995, which compared protocols with published articles, the authors concluded that published articles, and reviews that incorporate them, may be unreliable and overestimate the benefits of an intervention because outcomes were reported selectively [3].
When reading a review of a supplement, the methods section shows which databases were searched and when, which studies were left out and why, and how each included trial was rated for bias.
References
- Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021.
- Hróbjartsson A, Gøtzsche PC. Placebo interventions for all clinical conditions. Cochrane Database Syst Rev. 2010.
- Chan AW, Hróbjartsson A, Haahr MT, et al. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004.
- Cochrane. Cochrane Handbook for Systematic Reviews of Interventions, Chapter 1: Starting a review. last updated August 2021 (Handbook version 6.5).