This guide will take you through the steps involved in conducting a systematic review. Click one of the stages below to see information, tips, resources and support available for each part of the process.

*this is a rough estimate of the time needed to complete each stage, but will vary depending on the complexity of your review.

Systematic Review Service Charter

The Library provides tiered support for researchers conducting systematic reviews. Postgraduate students who conduct systematic reviews as part of their degree can request up to 3 hours of support per systematic review. Postdoctoral researchers and beyond can opt for more extensive support, depending on their requirements and our capacity. For details, please refer to the service charter.

Feedback

We are always improving our resources. If you'd like to share your feedback with us, please do so via our feedback form.

Is this a systematic review?

You might find that a systematic review isn't the best study type for your research topic perhaps due to the scope of your question or the nature of the existing body of literature. There are many other kinds of reviews that share some of the same characteristics.

The most common question is about the difference between a standard literature review and a systematic review. The table below explains some of the key differences. A full list of review types and their characteristics can be found in Tools & resources.

Systematic review vs. Literature review

Most systematic reviews are currently conducted in fields like health and medicine. Increasingly, systematic reviews are being published in other disciplines such as humanities and education.

Throughout this guide, we'll be making reference to bodies like Cochrane, the Joanna Briggs Institute, and the Campbell Collaboration. These organisations are responsible for the development of many best practice frameworks and guidelines. We always recommend consulting these original sources for more specific advice.

Scoping your topic

When starting a systematic review, it's a good idea to look in different databases and grey literature sources to see what evidence is available.

Conducting a scoping search allows you to put your research in context and justify its importance, identify any potential 'gaps' in your research area, find any existing systematic reviews, and see if there is enough evidence to write a systematic review.

Resources that can be used to scope a topic

Research topic vs. a review question

A 'research topic' is the area of study that you are researching, while a 'review question' is the more focused question that your systematic review aims to answer. For example, you could be studying non-therapeutic treatment of dementia but your review might focus on the question below:

If you’re stuck on focusing your research topic into a specific question, mnemonics are tools that can help you. Depending on the kind of question you’re asking, they highlight different concepts that are needed. Mnemonics can also help you build a search strategy. A list of different mnemonics can be found below, under ‘Search mnemonics’.

Types of systematic review questions

The most common type of systematic reviews are those answering efficacy questions (also known as therapy or intervention questions), where an intervention is evaluated to determine its effect on a population. These are the most common types of reviews conducted in health and medicine, but they are also common in the social sciences.

Our example is a clinical question assessing the effectiveness of an intervention. It can be broken down into PICO which is the mnemonic commonly used in these types of questions.

Search mnemonics

Why do you need to register a protocol?

• To help avoid duplication of effort through the public listing of reviews in progress
• To provide transparency and allow for peer review of the systematic review process
• To help readers assess the quality of the review in terms of bias, through comparison of the protocol and the published systematic review

They are published in public registers like PROSPERO. The table below shows an example of a protocol and what each section will typically describe.

Typical structure

There are many reporting standards available which outline the essential information you need to provide in a protocol. Requirements will change depending on where you will publish the protocol. Some of these standards include:

• PRISMA - a generic reporting standard for systematic reviews and meta-analyses
• PROSPERO - a public register which registers protocols and specifies essential registration fields which must be completed to describe the planned review in appropriate detail
• Joanna Briggs Institute - JBI registers protocols and specifies the components that comprise a JBI protocol in its reviewer's manual

It is often the case that reviewers may need to change their original protocol after a review has commenced. Any changes made should be reported and explained in an amendment to the protocol, as well as in the systematic review itself (Gough, Oliver, & Thomas, 2017). All registers have procedures for submitting amendments.

Inclusion and exclusion (eligibility) criteria

Inclusion and exclusion criteria help you screen papers for inclusion in your review. It's important to explicitly state these at the beginning, while you're constructing your search strategy. Some criteria include:

• Study design or characteristics - such as PICO, study design, setting, and time frame
• Report characteristics - including years considered, language, and publication status

It's important to justify each criterion in the methodology section of your systematic review. Different reporting guidelines may require different criteria.

Systematic reviews aim to evaluate all relevant information about a topic to answer a question. This means you need to locate all available evidence relevant to your question. Databases provide access to different collections of resources, so you’ll need to do some work thinking about which ones will contain the information you’re looking for. If you don't search enough, you'll potentially miss out on crucial studies. However there may be practical reasons for limiting the scope of your review, such as:

• lack of resources (people, funding)
• timeframe
• lack of evidence for reviews in emerging areas
• access to language translation

Creating a comprehensive literature search

A comprehensive literature search is one that eliminates bias during the search planning stage. Here are three kinds of bias to watch out for:

1. Publication bias - relevant research may be ‘unpublished’ and therefore unavailable through most databases which only index ‘published’ content. This may necessitate you searching sources outside of databases e.g. clinical trial registers.
2. Language / geographic bias - there are relevant studies that may be published in other languages and other regions. Do you need to account for these in your search?
3. Indexing bias - this may occur if you only search a limited number of databases or use only a few search terms.

Which databases should I search?

Depending on your review question, you need to consider searching both multidisciplinary and subject-specific databases to ensure your search is comprehensive. Multidisciplinary databases contain a high volume of literature from a diverse range of subjects. They are helpful in locating literature on your topic from disciplines you might not have considered. On the other hand, subject-specific databases contain in-depth coverage of literature which is highly specific to a field. A balance of both databases is required in your search strategy.

For a comprehensive list of databases, check the Library's list of databases.

Databases which are commonly used for searching in health and medicine include:

Cochrane considers Medline (via either Ovid or PubMed), Embase, and CENTRAL to be the most important sources of evidence to search when conducting a systematic review.

Regional databases include:

• LILACS Latin American and Caribbean Health Sciences Literature
• AIM African Index Medicus
• IMSEAR Index Medicus for the South East Asian Region
• Health Collection Australian content available via Informit

Search filters

Search filters are tried and tested methods for finding relevant research in health databases. Filters are commonly used to limit searches to a particular study type, or to find articles on a specific subject. They can be used to enhance your search, but can also be useful for identifying additional search terms.

When do I stop searching for new studies?

When conducting a systematic review, your aim is to search enough databases to be able to state with confidence that the literature has been comprehensively searched. Indicators of a comprehensive search strategy include:

• you look in a different database and don’t retrieve any new papers
• you can identify no new data
• you use a rule to decide when to stop searching

Do I need to search for grey literature?

Grey literature refers to research material that hasn't been published commercially like a book or a journal article has. The challenge lies in locating it, as it can be either published or unpublished.

Examples of published grey literature include:

• conference papers
• theses and dissertations
• government reports
• clinical trial registers

Examples of unpublished grey literature include:

• unpublished trials
• on-going studies
• unpublished data

Depending on your subject, it might be appropriate for you to include grey literature as evidence because it provides data which otherwise wouldn't be available through journal articles. Including grey literature also helps reduce publication bias.

Grey literature can sometimes be hard to locate, but we've compiled a list of tools and resources to assist you.

Other methods of locating relevant studies for your review:

• Handsearching - you can review the reference lists of papers you have already located for more relevant studies, or choose to search specific journals which apply to your research area. Journals selected for handsearching should be identified from your database searching (Aromataris & Riitano, 2014; Booth, 2016).
• Experts - you may wish to contact experts to clarify details of a study or request data. These experts may also provide papers that weren't retrieved through your comprehensive search (Aromataris & Riitano, 2014; Booth, 2016).
• Citation chaining – identify a small number of key papers on your topic, and then find all the publications they cite and are cited by to produce a set of closely related publications (Belter, 2015).

It’s time to perform your database searches as outlined in your protocol. You’ll have a search strategy, but you’ll need to adapt it to suit each database you search because they will use different interfaces and search operators (also called syntax). Once you’ve run your search, export the results into a reference manager.

Keep thorough records during this stage - the search process needs to be documented in enough detail that it can be reproduced. This is important for transparency, and assists with subsequent review updates.

Use a tool like the PRISMA flow diagram to document things like:

• The number of unique records identified by each database search
• The number of duplicate records removed
• The number of records excluded after preliminary screening
• The number of full text articles retrieved

Manage search results

To manage the results found in each database, use a reference manager. Reference managers can automate tedious steps like finding full text articles in bulk and de-duplicating records. When you export your results from each database, there are a number of essential fields you should download.

They include:

• Abstract
• Affiliation of the authors
• Digital object identifier (DOI)
• Clinical trial number (if available)
• Language
• Any comments that are available

Cochrane Handbook, Section 4.2 'Which fields to download'.

After you've conducted your searches and exported all your results into a reference manager, you can start screening articles. During this step, reviewers select which studies are relevant based on their inclusion and exclusion criteria.

To control bias and minimise the risk of excluding relevant studies, screening is usually done by two or more independent reviewers. They talk through any disagreements in decision making over study inclusion or exclusion.

A standard workflow for this stage might be:

1. Remove duplicate records from your pool of results
2. Examine titles and abstracts to remove articles which obviously don't meet the inclusion criteria
3. Find the full text of all potentially relevant articles
4. Link together any reports which discuss the same study
5. Screen full text articles and assess for inclusion in the review against the criteria outlined in your protocol.
6. Make any final decisions on study inclusion

The PRISMA flow diagram visually summarises the screening process. It makes the selection process transparent and clear to the reader by reporting on decisions made at various stages of the systematic review. When you're excluding articles at the full-text stage, you need to specify the reasons for exclusion. It's important to document this thoroughly.

Data extraction

Your next step is extracting the necessary data from your included studies. Not only will this help you assess the validity of the studies you're including in your review, but you're also prepping for the next stage: analysis and interpretation.

Before you extract all the data from the studies, you need to decide what data is necessary to extract. The below table is a sample of items to consider. The full table can be viewed in the Cochrane Handbook.

Once you've decided which items are needed, use a data extraction form as a consistent way to record the information.

Appraisal

When you're appraising the studies you've decided to include in your review, you're deciding whether you can draw conclusions about the effect of the intervention. This involves looking at external validity (did the study ask an appropriate research question?) and internal validity (did they answer the question 'correctly'?).

A key consideration to ask is, to what extent are the results of the included studies believable? You can assess this by examining the studies for various types of bias, which can lead to the study overstating or understating the true effect of the intervention. Section 8 of the Cochrane handbook names the following forms of bias:

• Selection bias
• Performance bias
• Detection bias
• Attrition bias
• Reporting bias
• Other biases

There are a number of appraisal tools which will help you assess the studies you're including in your review.

Incomplete reporting

Incomplete reporting can affect how you assess risk of bias. What's excluded has a bearing on whether you can trust the study as a whole, or if you can only trust specific outcomes. Some ways of detecting whether a study has been reported incompletely include:

• Using study appraisal checklists
• Comparing the methods to the results
• Comparing the outcomes of a protocol (if there’s one available) to the published report
• Asking the study authors for more information

After appraisal, you might find there are studies you’d like to exclude from the review. Make sure you record the reasoning behind exclusion as it will need to be reported.

During this stage, you'll be analysing and synthesising the data in order to come to a conclusion and answer your research question.

Your review will lend itself to different methods of analysis:

• A narrative analysis involves preparing a structured summary, comparing the data and coming up with themes to find evidence of an effect
• A statistical analysis is also known as a meta-analysis. It involves analysing the results across a number of studies in order to find the size and direction of an effect
• An integrative analysis combines both narrative and statistical analysis techniques, to reflect the diversity of the evidence included. For example, an integrative analysis could be a meta-analysis which appears alongside a narrative synthesis which provides additional information about the effect

Narrative synthesis techniques

Despite these advantages, a meta-analysis should only be performed if you have a group of studies which are similar - this similarity is referred to as homogeneity. At the very least, studies should be homogeneous in terms of participants, interventions, and outcomes so the meta-analysis can draw a meaningful conclusion.

Any kind of variability among studies is referred to as heterogeneity. There are several kinds - Cochrane names:

• Clinical diversity - where participants, interventions and outcomes differ between studies
• Methodological diversity - where there is variability in study design
• Statistical heterogeneity - where the observed intervention effect differs between studies because of chance

It is important to measure and address heterogeneity in your review because it affects the extent to which you can rely on the conclusions you draw.

Write

When you write up your review, not only are you reporting your findings and answering your question – you’re also reporting on the decisions made and the procedures followed to get you to that point. To make sure you don’t miss anything crucial in your write-up of the review, use the ‘Preferred Reporting Items for Systematic Reviews and Meta-Analyses’ (PRISMA) checklist.

Publish

Where you publish has a huge effect on the visibility and impact of your systematic review. It's important to be as well-informed as possible before making a decision on which journal to submit your paper to. If you've followed Cochrane, Joanna Briggs Institute, or Campbell Collaboration standards for conducting and writing a systematic review, you will already have an idea of where you want to publish.

The first hurdle is finding the right journals to publish in according to the scope of your review and your target audience. When considering who might take an interest in your research:

• Look at your reference list. Which journals do you regularly and consistently cite? Your work will likely be of interest to both the journal and its readers. Make sure you check the aims and scope in the 'information for authors' pages of each journal you'd like to publish in (e.g. Journal of Clinical Nursing).
• Use a tool like the Journal/Author Name Estimator (JANE) to retrieve a list of journals that may be interested in your systematic review, based on the similarity of your abstract to those of articles they have published. More tools for other disciplines are listed in Tools and resources.
• Identify influential journals in your field using metrics. There are a variety of metrics you can use to inform your decision. Some are more useful for certain disciplines than others. Check our Research Impact guide for more information.

How should I share my research?

When you submit your article to a journal for publishing, you often sign over copyright to the publishing journal. This affects how you are able to share it in the future, and how readers are able to access it. If it is important that clinicians and practitioners are able to access your research, it may be important to consider what sharing options a journal offers, or look at open access publication.

• For more information about copyright, check the copyright pages on the Library website.
• Open Access allows everyone to access your research. There are different ways of making your work Open Access. For more information, visit the Open Access pages on the Library website.