To perform a true systematic review, you should have at least three people who have agree to be a part of the review.  It is important to identify these roles at the outset to avoid hiccups in the process.

• Project lead: This person will ensure that the project meets minimum criteria and stays on schedule. This is typically the person who will submit the manuscript for publication.
• Screeners: At least two individuals who have sufficient content expertise to decide whether articles meet the inclusion criteria or if they should be excluded for both the title/abstract screening and the full-text screening. This can be a time-intensive process depending on the number of articles identified in the search stage.  A third person with content expertise should be available to resolve any conflicts between decisions of the screeners.
• Search specialist: This person will develop the appropriate search strategy, translate the searches to each database, and deduplicate the records in a manner that meets the established guidelines for a reproducible systematic review.
• Statistician: This individual should have the statistical expertise to analyze the findings of the review. This is a critical role if you plan to conduct a meta-analysis.
• Methodology expert: Ideally at least one person should have experience conducting a systematic review or you should have access to an advisor who is willing to help guide your team through the process.

A well-defined, answerable research question is necessary to create a systematic review. Systematic reviews in the medical field typically follow the PICO framework to answer a clinical question. 

• Patient or Population: What characteristics of the patient/population are necessary in order to answer the question? Try to be precise in this identification. For example, are you answering a question about all cancer patients or just HER-2 positive patients with inflammatory breast cancer?
• Intervention: What is the intervention, prognostic factor, or treatment under consideration?
• Comparison: What is the comparator, if any?
• Outcome: What are the expected outcomes, improvements, or measures that you hope to find?

Once you have your research question, make sure that it has not already been answered in the existing medical literature. If you find an existing systematic review (or reviews), evaluate them carefully. You should proceed with your review only if you feel that existing reviews you found are not recent enough, had flaws in the methodology or reporting, or if your review will address a unique aspect. (HINT: Document this step as you will likely be asked to justify this to journal editors and/or as part of the peer review process.)

Consider registering your protocol. Registration helps to promote transparency in the review process and also reduces the potential for duplication.

Many journals require SRs to have registered protocols PRIOR to beginning the screening stage. Check the author guidelines section of prospective journals. 

PROSPERO

Use PROSPERO to register systematic review protocols. Note: they will not accept protocols for most other review types.

Open Science Framework (OSF)

Open Science Framework accepts Systematic review protocols and other review protocols including scoping, integrative, and rapid.

• Identify databases to conduct your search

You should plan to search at least 3 databases. Most reviewers expect to see Medline (either via PubMed or Ovid) as one of the databases for health topics. Additional databases should be included based on your subject area. For a list of all available databases, consult the USF Health Libraries A to Z database list. 

• Identify search terms to conduct your search

Searches for a systematic review need to be both comprehensive (prioritizing sensitivity over specificity) and reproducible. To make your search comprehensive, identify both key words and controlled vocabulary to describe the main concepts of your question.

• Develop search strategies for each database

Combine terms using appropriate Boolean operators and subject headings (if applicable) for every database. Searches should be as similar as possible between the database.

No matter how experienced you are in searching PubMed and the medical literature, you will likely need help creating a search strategy that meets the guidelines for evidence synthesis in the health sciences. In fact, both the Cochrane Handbook and the Institute of Medicine recommend including a librarian trained in systematic review searching on your team.  

While this may not be possible, you may wish to consult with a librarian early in the process to ensure that you do not miss critical terms and databases in your search. Once you have developed your search strategy, you may also wish to request that a librarian conduct a peer review of your strategy. This will ensure that your review is in compliance with Peer Review of Electronic Search Strategies (PRESS) guidelines for systematic reviews.

• Run each search in the databases you selected, documenting your search strategy for each one. You should include these in your manuscript, typically as an appendix.
• Consider searching additional literature sources to reduce the risk of publication bias in your review
  • clinical trial registries
  • conference proceedings and reports
  • white papers
  • theses and dissertations
• For more information on grey literature, see the USF Health Libraries "Grey Literature" guide.

The PRISMA diagram shows the flow of records through the phases of a systematic review. It maps the number of records identified, included and excluded, record sources, and the reasons for exclusions. Different templates are available depending on the type of review (new or updated) and sources used in the review.

• Collect your records from all databases
• Identify duplicates. Helpful tools: EndNote, Covidence, Rayyan (see toolbox)
• Record the number of duplicates for your PRISMA flow diagram

• Perform interrater reliability on a small set of records prior to screening.
• Review the title and abstract of each record against the inclusion/exclusion criteria. This should be done independently by a minimum of 2 people with a 3rd person familiar with the subject as tie breaker for any conflicts.
• Record the number of records excluded at this stage for your PRISMA flowchart.

• Obtain full text for all records that met your inclusion criteria. Note any full text not accessible for PRISMA flowchart.
• Perform interrater reliability on a small set of reports prior to starting screening.
• Evaluate each article independently by a minimum of 2 reviewers with a 3rd person to act as tie breaker.
• Record exclusions with the reasons for omitting the article for the PRISMA flowchart.

The PRISMA diagram shows the flow of records through the phases of a systematic review. It maps the number of records identified, included and excluded, record sources, and the reasons for exclusions. Different templates are available depending on the type of review (new or updated) and sources used in the review.

Evaluate the methodology for each study to identify Risk of Bias (RoB)  According to the Center for Evidence-Based Medicine (CEBM), "critical appraisal looks at the way a study is conducted and examines factors such as internal validity, generalizability and relevance."  Findings are generally reported in a tabular format based on domain/element of the assessment tool. 

Determine the Level of Confidence (LoC) in the comprehensive findings of the review. Look at each outcome identified in the protocol to see if the supporting evidence is strong or weak. For reviews of clinical interventions, GRADE is preferred. The LoC is often presented as an evidence summary table (e.g., GRADE Summary of Findings tables).

In this final stage, you will report on your findings by combining and evaluating the data from included studies. Your report should include documentation of each step of the process with enough detail that your review could be reproduced. The report should conclude with recommendations regarding your findings and the need for further research.

Your manuscript should follow accepted reporting guidelines. You can find the appropriate guidelines based on the specific review type  by consulting PRISMA 2020, the Equator Network Toolkit, as well as the guidelines required by your target journal

• manuscript format
• page limits
• image requirements
• additional instructions for preparation and submission of the article