Systematic reviews, scoping reviews and meta-analyses

There are a number of different terms used in relation to reviewing literature:

  • Literature reviews
  • Systematic reviews
  • Scoping reviews
  • Meta-analyses

Use of these terms is often confusing. In this page we look at the differences and similarities.

Literature reviews

A literature review is a generic term for the discussion of the literature on a given topic. Both scholarly, semi-scholarly and non-scholarly literature reviews exist but they are never described in these terms. The term ‘literature review’ is applied indiscriminately to all three. But there are differences between them.

  • Scholarly literature reviews are formal documents usually written for a doctoral dissertation or as part of a publication such as a journal article. They normally aim to uncover a research gap within a specific area of scholarship, with a view to exposing this gap by means via the formulation of a hypothesis or research question. This gap in then explored in detail in subsequent chapter/sections of the paper/thesis.
  • Semi-scholarly literature reviews appear in undergraduate essays and honours theses. While putatively ‘scholarly’ in terms of writing style, they closely approximate non-scholarly literature reviews as they rarely expose a research gap. They can be considered non-scholarly in all but writing style.
  • Non-scholarly literature reviews do not expose a research gap, and merely wish to present a broad overview of an area of scholarship often for entertainment and/or educational purposes. There is little attempt to be comprehensive. The prose is written in simple, informal language to appeal to non-specialist audiences. Academics often write non-scholarly literature reviews for non-specialist online and print media publications (e.g., The Conversation, The Economist, and major newspapers) to draw attention to their scholarly literature reviews published in peer-reviewed papers. This is a form of self-promotion.

We cover how to write a scholarly literature review here and here.

Systematic reviews

A Systematic review poses a specific (usually clinical) research question and then outlines the literature that relates to that specific question. It uses pre-specified criteria in deciding on the literature to include/exclude and the evidence to admit in the discussion (Mellor, nd). A systematic review will often first produce a research protocol that outlines the need, aims and participants involved in the research (Carmeli, et al 2018). These kinds of reviews are often produced in the health sciences and other applied disciplines. Reviews are normally registered and submitted to an international research network, the Campbell-Cochrane Collaboration Library.

There are a number of reasons to write a systematic review:

  • Uncover international evidence on a specific question
  • Confirm current practice/address any variation/identify new practices
  • Identify and inform areas for future research
  • Identify and investigate conflicting results
  • Produce statements to guide decision-making (Munn, et al. 2018)

Some of these are common to scholarly literature reviews, e.g., identify and inform areas of future research, but others are unique, e.g., guiding decision-making, in particular, clinical decisions. This distinguishes systematic review from literature reviews, as the latter tend to focus on theoretical contributions. Sometimes systematic review, like a literature review, can identify the need for further research (a ‘gap’), but this is usually a by-product of insufficient information to answer the clinical question posed. It is not the aim of a systematic review.

Typically, the following process occurs in a systematic review:

  • Identification of the research question
  • Definition of inclusion and exclusion criteria for selecting research articles
  • Conduct of a search
  • Selection of studies to review (sometimes done by two people independently)
  • Extraction of data from included studies
  • Evaluation of risk of bias in included studies
  • Assessment of validity of findings
  • Interpretation and presentation of results (Mellor, nd).

Scoping reviews

A Scoping review (aka a “scoping exercise” or a “scoping study”) is an exploratory review that aims to investigate a broad research area in order to decide on what to investigate further. It aims to help in deciding the most appropriate research question to ask.  Thus, a scoping review can lead to the need for a systematic review. In other words, one undertakes a scoping review to decide what to research ‘systematically’. Scoping reviews are used to determine the volume and extent of literature and for examining emerging evidence and when specific questions are still unclear. It does not attempt to answer a specific discrete question (Suchrew & Macaluso, 2019) Additionally, a scoping review will “map” or “chart” the evidence that might need further systematic review.

There are a number of reasons to write a scoping review:

  • To identify the types of available evidence in a given field
  • To clarify key concepts/definitions in the literature
  • To examine how research is conducted on a certain
  • topic or field
    To identify key characteristics or factors related to a
  • concept
  • As a precursor to a systematic review
  • To identify and analyse knowledge gaps (Munn, et al. 2018)

A similar process to systemic reviews is followed for scoping reviews with minor variations:

  • Identification of question or objectives
  • Definition of inclusion and exclusion criteria
  • Adoption of a systematic search strategy
  • Selection of studies to review
  • Extraction of data
  • Charting of evidence
  • Interpretation and presentation of results (Mellor, nd).

No such formal “charting” process is followed for literature reviews or systematic reviews. This makes literature reviews, systematic reviews and scoping reviews quite different.

Meta-analyses

A meta-analysis is a statistical analysis of the results from a systematic review. It is a type of systematic review distinguished in the use of statistical techniques to summarise and combine the results of the studies that are reviewed systematically. It is appropriate when:

  • Reporting quantitative results rather than findings or theories
  • Reporting trends, correlations, and prevalence rates
  • Establishing the relationship between constructs and associations between variables
  • When results from studies are bivariate/zero order relationships, involve degrees of freedom contrasts, or partial/multivariate effects
  • When results can be configured as standard effect sizes (Siddaway, nd).

A systematic review can contain a meta-analysis; however, a systematic review and a meta-analysis is not the same thing. Whether or not a meta-analysis is needed depends on the question being asked. If the data is heterogenous, and/or insufficient for a meta-analysis, other analyses might be used instead. A meta-analysis is only conducted as part of a systematic review, but not all such reviews are meta-analyses (Hanratty, 2018).

The attached table makes the similarities and differences between the different types of ‘review’ very clear.

References