Systematic review

 

PRISMA Statement

PRISMA 2020 Statement (Preferred Reporting Items for Systematic Reviews and Meta-Analyzes) is a reporting guideline that includes a 27-point checklist, an abstract checklist and a flow chart (ref. 9). The checklist includes a number of items that should be addressed, when preparing and reporting a systematic review. The checklist generally serves to achieve transparency and improve the reporting of systematic reviews and meta-analyses. PRISMA flow diagram is a flow diagram that contributes graphically to outlining the information flow in a systematic review.

Research question

The research question is pivot for a systematic review, and essential for the preparation of the systematic search strategy (ref. 5). Conceptualizing models can be used to help formulating a clear and structured research question. The choice of model to use, depends on the specific subject:

PICO
The PICO model is the most commonly used model to help formulating clinical research questions (ref. 1). PICO is useful for framing clinical research questions that have do with the effect of interventions, and The Cochrane Handbook refer to this model (ref. 6).

CoCoPOP
The CoCoPOP model can be used when formulating a research question related to the incidence or prevalence of a disease, symptom, health condition, etc. (ref. 6a). Co (Condition), Co (Context) POP (Population)

The model describes four elements of a focused clinical research question: 
P (Patient / Problem / Population), I (Intervention), C (Comparator) og O (Outcome) (ref. 5, 7). 
This PICO scheme can be used to help formulating your research question, and structure your literature search.
PICO can be extended to PICOT, if T (Timing / Type of studie) or PICOS if S (Setting) is crusial. 

PICo
The PICo model can be used, when a particular phenomenon is interesting in relation to a patient group, for instance in a particular context:  P (Patient/problem/population), I  (Phenomenon of Interest) and Co (Context) (ref. 8). 
This PICo scheme can be used to help formulating your research question, and structure your literature search.

PIRD
The PIRD model can be used to formulate a research question that addresses diagnostic test accuracy. (ref . 8a) P (Population), I (Index test), R (Reference test), D (Diagnosis of interest).

PEO

The PEO model, P (Population), E (Exposure) and O (Outcome) can be used for topics that relate to the relationship between exposure (exposure) and a health outcome  (ref. 10)

SPICE
The SPICE model is formulated as follows: S (Setting), P (Population), I (Intervention), C (Comparison) and E (Evaluation) (11).
 
SPIDER
The SPIDER model is a second model to help framing qualitative research questions, and is formulated as follows: S (Sample), PI (Phenomenon of Interest), D (Design), E (Evaluation) and R (Research type) (ref. 12).
 

Protocol

According to Cochrane Handbook, a protocol is a structured plan that must be followed in a study. As part of the preparation of a systematic review (and to minimize bias in the process), the most appropriate is to devise a protocol that describes the rationale, hypothesis, where the literature has been searched and what additional methods that are going to be used when preparing the systematic review (ref. 13).
Several tools exist to help preparing a protocol (overview from PRISMA (ref. 14)):

• PRISMA for systematic review protocols (PRISMA-P)
• Institute of Medicine: STANDARD 2.6 Develop a systematic review protocol
• Cochrane Handbook Chap. 4 (ref. 15)

Protocols can be registered in PROSPERO, which is an international database of prospectively registered systematic reviews within health, social care, welfare, public health, education, crime, etc., where there is a health outcome. Protocols for systematic reviews can thus be registered in PROSPERO, if the systematic review falls under current requirements at PROSPERO. If the review topic does not meet the requirements set by PROSPERO, a protocol can be prepared based on PRISMA-P. The protocol can then be uploaded to a freely accessible place, e.g., Open Science Framework (OSF) Registries, Figshare or another accessible web page.

You can also publish the protocol in a scientific journal such as BMJ Open or Systematic Reviews or certain other journals.

  The literature search aims to collect all evidence that forms the basis of the systematic review, and is therefore a very important and essential part of a systematic review. The literature search therefore potentially has a large impact of the quality of the systematic review (ref. 15). Several studies indicate that the quality and reproducibility of literature searches in systematic reviews often are of a low quality (ref. 16-18).

In principle, a systematic literature search does not differ from a comprehensively conducted literature search. It is recommended that the literature search has a high recall (recall, a theoretical measure that indicates how much of the relevant literature in a database that has been identified in the literature search (ref. 19)), this may result in a low precision (precision, how much of the found literature of which is relevant) (ref. 20).

In practice this means that when you do a comprehensive, systematic literature search, which aims to collect as much evidence / literature as possible, this will result in some irrelevant literature as well (low precision). 

Preparation of the systematic literature search 

In order to conduct a systematic literature search, it is important to group / structure your search in correct search blocks (see the Literature search guide), so you can start preparing lists of search terms. If you have used a conceptualizing model (e.g. the PICO model), this may help to identify the most important elements to include in your search.

The preparation of the actual systematic literature search requires the knowledge of complex literature search, including subject headings, indexing, free text searching and search technical methods (see the Literature searching manual and Advanced search techniques manual) – as well as how to search the individual databases. There is a lot of work involved in preparing the search, before you can execute the final search and move on with the search result. Bramer et al.describes how to plan and perform a systematic literature search step by step (REFERENCE: Bramer, W.M., et al., A systematic approach to searching: an efficient and complete method to develop literature searches. J Med Libr Assoc, 2018. 106(4): p. 531-541.).

It is important to critically appraise the specific search strategy, and PRESS 2015 Guideline Evidence-Based Checklist (published in McGowan et al. (ref. 21)) can be used to this.

As previously mentioned, the validity and reproducibility of the systematic search is essential. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) has developed a PRISMA for Searching - a checklist with 16 reporting items that should be reported when reporting a systematic literature search (REFERENCE:Rethlefsen ML, Kirtley S, Waffenschmidt S, Ayala AP, Moher D, Page MJ, Koffel JB; PRISMA-S Group. PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. Syst Rev. 2021 Jan 26;10(1):39. doi: 10.1186/s13643-020-01542-z.).

The checklist ensures that what should be included in a systematic search is reported and clear, so that the search is valid, reproducible, and transparent. The checklist can thus also be used prospectively, i.e., in the planning of the systematic search.

Databases and additional information sources

In addition to conduct the literature search technically correct, it is important to search multiple databases and information sources, to identify as many relevant studies as possible (ref. 20). Choice of databases and information sources to search depends on the specific subject.

Bibliographic databases
The most widely used bibliographic databases within health science include: PubMed / Medline, Embase and Cochrane Library.

Subject specific databases
Subject specific (bibliographic) databases with more narrow focus exists, e.g. CINAHL (Cumulative Index to Nursing and Allied Health), PsycInfo and AMED (The Allied and Complementary Medicine Database) (see health scientific databases for an overview of databases).

Cross-disciplinary search engines
The cross-disciplinary search engines include: Science of Index (Expanded), Scopus and Google Scholar. Web of Science and Scopus (to a lesser degree PubMed) register citations and can be used to make a forward-looking citation search, identifying which publications have cited a given relevant publication (ref. 21).

In addition to the traditional databases and search engines, there are other sources of information and other types of literature that may be relevant to search, when preparing a systematic review:

Clinical trial records
Clinical trial records register clinical trial information, and may be an important sources of information if you want to identify, for example, randomized clinical trials (RCTs)(ref. 22) for example ClinicalTrials.gov .

 

Grey literature

Grey literature refers to literature that has not been completed for publication (ref. 23), and includes: research reports, dissertations, conference contributions, etc. Some of the most widely used databases and search engines contain some conference contributions, including Embase, Scopus and Web of Science. Grey Matters is a database including health-related grey literature.

Preprint servers
A preprint refers to a version of a scientific article that precedes peer review and publication. You can search relevant preprint servers to identify manuscripts and data that have not yet been formally published or accepted for publication and thus not searchable in traditional bibliographic databases, such as gray literature or elsewhere. Depending on the subject area, there are e.g. medRXV (health sciences), bioRxiv (biology) and SocArXiv (social sciences).

Reference Management Tools
Reference management tools, such as Endnote, help to keep track of references and are very relevant to use when preparing reviews. Endnote can (as well can Covidence) be used to delete the duplicates that occur when searching multiple databases. Read more about Endnote and find guides here.

 

Covidence
The software platform Covidence, is a reference management tool developed for the preparation of systematic reviews. The program can be used for sorting literature, quality assessment and data extraction in connection with systematic reviews. Read more about Covidence and find instructions for creating user access here.

The university Library offers courses on Endnote and Covidence. Find a list of relevant courses here.

(ref. 1) Green S, Higgins JPT, Alderson P, Clarke M, Mulrow CD, Oxman AD. Chapter 1: Introduction. In: Higgins JPT, Green S (editors), Cochrane Handbook of Systematic Reviews of Interventions. Chichester (UK): John Wiley & Sons (2008).

(ref. 2) Campbell Collaboration. Website. (last visited 9.8.2018).

ref. 3) Institute of Medicine of the National Academies. Standards for Systematic Reviews (2011). Link. 

(ref. 4) O´Connor D, Green S, Higgins JPT (editors). Chapter 5: Defining the review question and developing criteria for including studies. In: Higgins JPT, Green S (editors), Cochrane Handbook of Systematic Reviews of Interventions. Chichester (UK): John Wiley & Sons (2008

(ref. 5) Eriksen MB, Frandsen TF. The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: a systematic review. JMLA (2018), 106(4): 420-431.

(ref. 6) O´Connor D, Green S, Higgins JPT (editors). Chapter 5: Defining the review question and developing criteria for including studies. In: Higgins JPT, Green S (editors), Cochrane Handbook of Systematic Reviews of Interventions. Chichester (UK): John Wiley & Sons (2008).

(ref. 6a) Munn Z., Moola S., Lisy K, Riitano D., Tufanaru C. Methodological guidance for systematic reviews of observational epidemiological studies reporting prevalence and cumulative incidence data. Int J Evid Based Healthc. 2015 Sep;13(3):147-53.)

(ref. 7) Frandsen TF, Dyrvig AK, Christensen BC, Fasterholdt I, Ølholm AM. En guide til valide og reproducerbare systematiske litteratursøgninger. Ugeskrift for Læger (2014), 176(7):647- 651.

(ref. 8) Joanna Briggs Institute 2011. Joanna Briggs Institute Reviewers' Manual 2011. The University of Adelaide, South Australia, 2011.

(ref. 8a) Campbell J, Klugar M, Ding S, Carmody DP, Hakonsen SJ, Jadotte YT, White S, Munn Z. Diagnostic test accuracy: methods for systematic review and meta-analysis. Int J Evid Based Healthc. 2015 Sep;13(3):154-62.).

(ref. 9) Page MJ, McKenzie JE, Bossuyt PM, Boutron I , Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021; 372: n71. Doi: 10.1136 / bmj.n71).  

(ref. 10) Munn Z, Stern C, Aromataris E, Lockwood C, Jordan Z. What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences. BMC Med Res Methodol. 2018; 18: 5.).

(ref. 11) Booth A. Clear and present questions: formulating questions for evidence based practice. Libr Hi Tech. 2006;24(3):355-68.

(ref. 12) Cooke A, Smith D, Booth A. Beyond PICO: the SPIDER tool for qualitative evidence synthesis. Qual Health Res. 2012; 22(10):1435-43.

(ref. 13) Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.2 (updated February 2021). Cochrane, 2021, link.

(ref. 14) PRISMA (last visited 9.10.2018).

(ref. 15) McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefevbre C. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol 2016, 75: 40-46.

(ref. 16) Koffel JB, Rethlefsen ML. Reproducibility of Search Strategies Is Poor in Systematic Reviews Published in High-Impact Pediatrics, Cardiology and Surgery Journals: A Cross-Sectional Study. PLoS One. (2016); 11(9):e0163309.

(ref. 17) Meert D, Torabi N, Costella J. Impact of librarians on reporting of the literature searching component of pediatric systematic reviews. J Med Libr Assoc. (2016); 104(4):267-277.

(ref. 18) Faggion, CM, Jr,  Huivin R,  Aranda L,  Pandis N,  Alarcon M. The search and selection for primary studies in systematic reviews published in dental journals indexed in MEDLINE was not fully reproducible. J Clin Epidemiol (2018); 98: 53-61.

(ref. 19) Frandsen TF, Dyrvig AK, Christensen JB, Fasterholdt I, Ølholm AM. En guide til reproducerbare og systematiske litteratursøgninger. Ugeskr Læger (2014); 176: V02130141.

(ref. 20)  Lefebvre C, Glanville J, Briscoe S, Littlewood A, Marshall C, Metzendorf M-I, Noel-Storr A, Rader T, Shokraneh F, Thomas J, Wieland LS. Chapter 4: Searching for and selecting studies. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.0 (updated July 2019). Cochrane, 2019. Link

(ref. 21) McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefevbre C. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol (2016); 75: 40 – 46.

(ref. 22) Baudard M, Yavchitz A, Ravaud P, Perrodeau E, Boutron I. Impact of searching clinical trial registries in systematic reviews of pharmaceutical treatments: methodological systematic review and reanalysis of meta-analyses. BMJ (2017); 356: j448.

(ref. 23) Informationsordbogen, ordbogsopslag: Grå litteratur. Website (last visited 17.10.2018).

(ref. 24) The Method Handbook - Model for preparing national clinical guidelines. The National Board of Health (2017), version 3. Only available in Danish.

(ref. 25) BMJ Best Practice. What is GRADE. (last visited 25.10.2018).