Use of a search summary table to improve systematic review search methods, results, and efficiency
DOI:
https://doi.org/10.5195/jmla.2021.809Keywords:
Searching, Information Retrieval, Systematic ReviewsAbstract
Background: Systematic reviews are comprehensive, robust, inclusive, transparent, and reproducible when bringing together the evidence to answer a research question. Various guidelines provide recommendations on the expertise required to conduct a systematic review, where and how to search for literature, and what should be reported in the published review. However, the finer details of the search results are not typically reported to allow the search methods or search efficiency to be evaluated.
Case Presentation: This case study presents a search summary table, containing the details of which databases were searched, which supplementary search methods were used, and where the included articles were found. It was developed and published alongside a recent systematic review. This simple format can be used in future systematic reviews to improve search results reporting.
Conclusions: Publishing a search summary table in all systematic reviews would add to the growing evidence base about information retrieval, which would help in determining which databases to search for which type of review (in terms of either topic or scope), what supplementary search methods are most effective, what type of literature is being included, and where it is found. It would also provide evidence for future searching and search methods research.References
Yaffe J, Montgomery P, Hopewell S, Shepard LD. Empty reviews: a description and consideration of Cochrane systematic reviews with no included studies. PLOS One. 2012;7(5):e36626.
Borah R, Brown AW, Capers PL, Kaiser KA. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open. 2017 Feb 27;7(2):e012545.
Garner P, Hopewell S, Chandler J, MacLehose H, Akl EA, Beyene J, Chang S, Churchill R, Dearness K, Guyatt G, Lefebvre C, Liles B, Marshall R, Martínez García L, Mavergames C, Nasser M, Qaseem A, Sampson M, Soares-Weiser K, Takwoingi Y, Thabane L, Trivella M, Tugwell P, Welsh E, Wilson EC, Schünemann HJ; Panel for Updating Guidance for Systematic Reviews (PUGs). When and how to update systematic reviews: consensus and checklist. BMJ. 2016 Jul 20;354:i3507.
Koffel JB, Rethlefsen ML. Reproducibility of search strategies is poor in systematic reviews published in high-impact pediatrics, cardiology and surgery journals: a cross-sectional study. PLOS One. 2016 Sep 26;11(9):e0163309.
Yoshii A, Plaut DA, McGraw KA, Anderson MJ, Wellik KE. Analysis of the reporting of search strategies in Cochrane systematic reviews. J Med Libr Assoc. 2009 Jan;97(1):21–9. DOI: http://dx.doi.org/10.3163/1536-5050.97.1.004.
Sampson M, McGowan J. Errors in search strategies were identified by type and frequency. J Clin Epidemiol. 2006 Oct;59(10):1057–63.
Centre for Reviews and Dissemination. Systematic reviews: CRD’s guidance for undertaking reviews in health care. The Centre; 2009.
Campbell Collaboration. Campbell systematic reviews: policies and guidelines. The Collaboration; 2019. Contract no.: 1.
Collaboration for Environmental Evidence. Guidelines and standards for evidence synthesis in environmental management. The Collaboration; 2018.
Joanna Briggs Institute. The Joanna Briggs Institute reviewers’ manual 2015: methodology for JBI scoping reviews [Internet]. The Institute; 2015 [cited 13 Nov 2020]. <https://nursing.lsuhsc.edu/JBI/docs/ReviewersManuals/Scoping-.pdf>.
Townsend WA, Anderson PF, Ginier EC, MacEachern MP, Saylor KM, Shipman BL, Smith JE. A competency framework for librarians involved in systematic reviews. J Med Libr Assoc. 2017 Jul;105(3):268–75. DOI: http://dx.doi.org/10.5195/jmla.2017.189.
Rethlefsen ML, Farrell AM, Trzasko LCO, Brigham TJ. Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. J Clin Epidemiol. 2015 Jun;68(6):617–26.
Institute of Medicine. Finding what works in health care: standards for systematic reviews. The Institute; 2011.
Evidence Synthesis Team. Supplementary searching @ EAHIL 2018 [Internet]. The Team; 2018 [cited 13 Nov 2020]. <https://evidsynthteam.wordpress.com/2018/07/12/supplementary-searching-eahil-2018/>.
Paez A. Grey literature: an important resource in systematic reviews. J Evidence-Based Med. 2017 Aug;10(3):233–40.
Mahood Q, Van Eerd D, Irvin E. Searching for grey literature for systematic reviews: challenges and benefits. Res Synth Methods. 2014 Sep;5(3):221–34.
Hartling L, Featherstone R, Nuspl M, Shave K, Dryden DM, Vandermeer B. Grey literature in systematic reviews: a cross-sectional study of the contribution of non-English reports, unpublished studies and dissertations to the results of meta-analyses in child-relevant reviews. BMC Med Res Methodol. 2017 Apr 19;17(1):64.
McAuley L, Pham B, Tugwell P, Moher D. Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet. 2000 Oct 7;356(9237):1228–31.
Higgins JPT, Green S, eds. Cochrane handbook for systematic reviews of interventions. Version 5.1.0 (updated Mar 2011). Cochrane Collaboration; 2011.
Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLOS Med. 2009;6(7):e1000097.
Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, Moher D, Tugwell P, Welch V, Kristjansson E, Henry DA. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017 Sep 21;358:j4008.
Critical Appraisal Skills Programme. CASP (systematic review) checklist [Internet]. The Programme; 2018 [cited 13 Nov 2020]. <https://casp-uk.net/wp-content/uploads/2018/03/CASP-Systematic-Review-Checklist-2018_fillable-form.pdf>.
Cooper C, Varley-Campbell J, Booth A, Britten N, Garside R. Systematic review identifies six metrics and one method for assessing literature search effectiveness but no consensus on appropriate use. J Clin Epidemiol. 2018 Jul;99:53–63.
Rader T, Mann M, Stansfield C, Cooper C, Sampson M. Methods for documenting systematic review searches: a discussion of common issues. Res Synth Methods. 2014 Jun;5(2):98–115.
Atkinson KM, Koenka AC, Sanchez CE, Moshontz H, Cooper H. Reporting standards for literature searches and report inclusion criteria: making research syntheses more transparent and easy to replicate. Res Synth Methods. 2015 Mar;6(1):87–95.
Rethlefsen M, Ayala A, Kirtley S, Koffel J, Waffenschmidt S. PRISMA-S search reporting extension. Center for Open Science; 2019. DOI: http://dx.doi.org/10.17605/OSF.IO/YGN9W.
Kwon Y, Powelson SE, Wong H, Ghali WA, Conly JM. An assessment of the efficacy of searching in biomedical databases beyond MEDLINE in identifying studies for a systematic review on ward closures as an infection control intervention to control outbreaks. Syst Rev. 2014 Nov 11;3(1):135.
Beyer FR, Wright K. Can we prioritise which databases to search? a case study using a systematic review of frozen shoulder management. Health Inf Libr J. 2013 Mar;30(1):49–58.
Brettle AJ, Long AF. Comparison of bibliographic databases for information on the rehabilitation of people with severe mental illness. Bull Med Libr Assoc. 2001 Oct;89(4):353–62.
Rathbone J, Carter M, Hoffmann T, Glasziou P. A comparison of the performance of seven key bibliographic databases in identifying all relevant systematic reviews of interventions for hypertension. Syst Rev. 2016 Feb 9;5:27.
Goossen K, Tenckhoff S, Probst P, Grummich K, Mihaljevic AL, Büchler MW, Diener MK. Optimal literature search for systematic reviews in surgery. Langenbecks Arch Surg. 2018 Feb;403(1):119–29.
Aagaard T, Lund H, Juhl C. Optimizing literature search in systematic reviews—are MEDLINE, EMBASE and CENTRAL enough for identifying effect studies within the area of musculoskeletal disorders? BMC Med Res Methodol. 2016 Nov 22;16(1):161.
Bramer WM, Rethlefsen ML, Kleijnen J, Franco OH. Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev. 2017 Dec 6;6(1):245.
Hartling L, Featherstone R, Nuspl M, Shave K, Dryden DM, Vandermeer B. The contribution of databases to the results of systematic reviews: a cross-sectional study. BMC Med Res Methodol. 2016 Sep 26;16(1):127.
Levay P. The contributions of MEDLINE, other bibliographic databases and various search techniques to NICE public health guidance. Evid Based Libr Inf Pract. 2015;10(1):50–68.
Hunt H, Abbott R, Boddy K, Whear R, Wakely L, Bethel A, Morris C, Prosser S, Collinson A, Kurinczuk J, Thompson-Coon J. “They’ve walked the walk”: a systematic review of qualitative and qualitative evidence for parent-to-parent support for parents of babies in neonatal care. J Neonatal Nurs. 2019 Aug;25(4):166–76.
Wright K, Golder S, Lewis-Light K. What value is the CINAHL database when searching for systematic reviews of qualitative studies? Syst Rev. 2015 Jun 26;4:104. Erratum in: Syst Rev. 2015 Nov 20;4:169.
Cooper C, Booth A, Varley-Campbell J, Britten N, Garside R. Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies. BMC Med Res Methodol. 2018 Aug 14;18(1):85.
Pandor A, Kaltenthaler E, Martyn-St James M, Wong R, Cooper K, Dimairo M, O’Cathain A, Campbell F, Booth A. Delphi consensus reached to produce a decision tool for SelecTing Approaches for Rapid Reviews (STARR). J Clin Epidemiol. 2019 Oct;114:22–9.
Colquhoun HL, Levac D, O’Brien KK, Straus S, Tricco AC, Perrier L, Kastner M, Moher D. Scoping reviews: time for clarity in definition, methods, and reporting. J Clin Epidemiol. 2014 Dec;67(12):1291–4.
