Development of an efficient search filter to retrieve systematic reviews from PubMed
DOI:
https://doi.org/10.5195/jmla.2021.1223Keywords:
search filter, systematic reviews, PubMed, information retrieval, search strategiesAbstract
Objective: Locating systematic reviews is essential for clinicians and researchers when creating or updating reviews and for decision-making in health care. This study aimed to develop a search filter for retrieving systematic reviews that improves upon the performance of the PubMed systematic review search filter.
Methods: Search terms were identified from abstracts of reviews published in Cochrane Database of Systematic Reviews and the titles of articles indexed as systematic reviews in PubMed. Both the precision of the candidate terms and the number of systematic reviews retrieved from PubMed were evaluated after excluding the subset of articles retrieved by the PubMed systematic review filter. Terms that achieved a precision greater than 70% and relevant publication types indexed with MeSH terms were included in the filter search strategy.
Results: The search strategy used in our filter added specific terms not included in PubMed’s systematic review filter and achieved a 61.3% increase in the number of retrieved articles that are potential systematic reviews. Moreover, it achieved an average precision that is likely greater than 80%.
Conclusions: The developed search filter will enable users to identify more systematic reviews from PubMed than the PubMed systematic review filter with high precision.
References
Boluyt N, Tjosvold L, Lefebvre C, Klassen TP, Offringa M. Usefulness of systematic review search strategies in finding child health systematic reviews in MEDLINE. Arch Pediatr Adolesc Med. 2008;162(2):111–6. DOI: http://dx.doi.org/10.1001/archpediatrics.2007.40.
Ely JW. Obstacles to answering doctors’ questions about patient care with evidence: qualitative study. BMJ. 2002;324(7339):710. DOI: http://www.bmj.com/cgi/doi/10.1136/bmj.324.7339.710.
Shariff SZ, Sontrop JM, Haynes RB, Iansavichus AV, McKibbon KA, Wilczynski NL, Weir MA, Speechley MR, Thind A, Grag AX. Impact of PubMed search filters on the retrieval of evidence by physicians. CMAJ. 2012;184(3):184–90. DOI: http://doi.org/10.1503/cmaj.101661.
White VJ, Glanville JM, Lefebvre C, Sheldon TA. A statistical approach to designing search filters to find systematic reviews: objectivity enhances accuracy. J Inf Sci. 2001;27(6):357–70. DOI: http://doi.org/10.1177/016555150102700601.
Golder S, Wright K, Loke YK. The development of search filters for adverse effects of surgical interventions in MEDLINE and Embase. Health Info Libr J. 2018;35(2):121–9. DOI: http://dx.doi.org/10.1111/hir.12213.
Beale S, Duffy S, Glanville J, Lefebvre C, Wright D, Mccool R, Varley D, Boachi C, Fraser C, Harabour J, Smith L. Choosing and using methodological search filters: searchers’ views. Health Info Libr J. 2014;31(2):133–47. DOI: http://dx.doi.org/10.1111/hir.12062.
Health Information Research Unit. Hedges [Internet]. 2016 [cited 24 Nov 2020]. <https://hiru.mcmaster.ca/hiru/HIRU_Hedges_home.aspx>.
The InterTASC Information Specialists’ Sub-Group (ISSG). Systematic reviews: filters [Internet]. [cited 24 Nov 2020]. <https://sites.google.com/a/york.ac.uk/issg-search-filters-resource/home/systematic-reviews?authuser=0 >.
Lefebvre C, Glanville J, Beale S, Boachie C, Duffy S, Fraser C, Harbour J, McColl R, Smith L. Assessing the performance of methodological search filters to improve the efficiency of evidence information retrieval: five literature reviews and a qualitative study. Health Technol Assess. 2017;21(69). DOI: http://dx.doi.org/10.3310/hta21690.
Lunny C, McKenzie JE, McDonald S. Retrieval of overviews of systematic reviews in MEDLINE was improved by the development of an objectively derived and validated search strategy. J Clin Epidemiol. 2016;74:107–18. DOI: http://dx.doi.org/10.1016/j.jclinepi.2015.12.002.
Wong SS, Wilczynski NL, Haynes RB. Comparison of top-performing search strategies for detecting clinically sound treatment studies and systematic reviews in MEDLINE and EMBASE. J Med Libr Assoc. 2006;94(5):451–5.
Boynton J, Glanville J, McDaid D, Lefebvre C. Identifying systematic reviews in MEDLINE: developing an objective approach to search strategy design. J Inf Sci. 1998;24(3):137–54. DOI: http://doi.org/10.1177/016555159802400301.
Lee E, Dobbins M, Decorby K, McRae L, Tirilis D, Husson H. An optimal search filter for retrieving systematic reviews and meta-analyses. BMC Med Res Methodol. 2012;12:51. DOI: http://doi.org/10.1186/1471-2288-12-51.
Wilczynski NL, Haynes RB. EMBASE search strategies achieved high sensitivity and specificity for retrieving methodologically sound systematic reviews. J Clin Epidemiol. 2007;60(1):29–33. DOI: http://doi.org/10.1016/j.jclinepi.2006.04.001.
Prady SL, Uphoff EP, Power M, Golder S. Development and validation of a search filter to identify equity-focused studies: reducing the number needed to screen. BMC Med Res Methodol. 2018;18:106. DOI: http://doi.org/10.1186/s12874-018-0567-x.
Shojania KG, Bero LA. Taking advantage of the explosion of systematic reviews: an efficient MEDLINE search strategy. Eff Clin Pract. 2001;4(4):157–62.
Wong SS, Wilczynski NL, Haynes RB. Optimal CINAHL search strategies for identifying therapy studies and review articles. J Nurs Scholarsh. 2006;38(2):194–9.
Montori VM, Wilczynski NL, Morgan D, Haynes RB. Optimal search strategies for retrieving systematic reviews from Medline: analytical survey. BMJ. 2005;330(7482):68. DOI: http://doi.org/10.1136/bmj.38336.804167.47.
Neilson C, Lê ML. A failed attempt at developing a search filter for systematic review methodology articles in Ovid Embase. J Med Libr Assoc. 2019;107(2):203–9. DOI: http://doi.org/10.5195/jmla.2019.519.
Chatterley T, Dennett L. Utilisation of search filters in systematic reviews of prognosis questions. Health Info Libr J. 2012;29(4):309–22. DOI: http://doi.org/10.1111/hir.12004.
Gómez-García F, Ruano J, Aguilar-Luque M, Gay-Mimbrera J, Maestre-Lopez B, Sanz-Cabanillas JL, Carmona-Fernández PJ, González-Padilla M, García-Nieto AV, Isla-Tejera B. Systematic reviews and meta-analyses on psoriasis: role of funding sources, conflict of interest and bibliometric indices as predictors of methodological quality. Br J Dermatol. 2017;176(6):1633–44. DOI: http://doi.org/10.1111/bjd.15380.
Flores-Mir C, Major MP, Major PW. Search and selection methodology of systematic reviews in orthodontics (2000-2004). Am J Orthod Dentofacial Orthop. 2006;130(2):214–7. DOI: http://doi.org/10.1016/j.ajodo.2006.02.028.
Pölkki T, Kanste O, Kääriäinen M, Elo S, Kyngäs H. The methodological quality of systematic reviews published in high-impact nursing journals: a review of the literature. J Clin Nurs. 2014;23(3–4):315–32. DOI: http://doi.org/10.1111/jocn.12132.
Yaylali IE, Alaçam T. Critical assessment of search strategies in systematic reviews in endodontics. J Endod. 2016;42(6):854–60. DOI: http://doi.org/10.1016/j.joen.2016.02.018.
Vassar M, Carr B, Kash-Holley M, DeWitt E, Koller C, Day J, Day K, Herrmann D, Holzmann M. Database choices in endocrine systematic reviews. J Med Libr Assoc. 2015;103(4):189–92. DOI: http://doi.org/10.1111/hir.12176.
Faggion CM, Diaz KT. Overview authors rarely defined systematic reviews that are included in their overviews. J Clin Epidemiol. 2019; 109:70–9. DOI: http://doi.org/10.1016/j.jclinepi.2019.01.004.
Grindlay D. Search strategies for finding systematic reviews. Br J Dermatol. 2017;176(6):1672. DOI: http://doi.org/10.1111/bjd.15455.
Martín-Rodero H, Sanz-Valero J, Galindo-Villardón P. The methodological quality of systematic reviews indexed in the MEDLINE database a multivariate approach. Electron Libr. 2018;36(1):146–58. DOI: http://doi.org/10.1108/EL-01-2017-0002.
Salvador-Oliván J-A, Marco-Cuenca G, Arquero-Avilés R. Errors in search strategies used in systematic reviews and their effects on information retrieval. J Med Libr Assoc. 2019;107(2):210–21. DOI: http://doi.org/10.5195/jmla.2019.567.
Bradley SM. Examination of the clinical queries and systematic review “hedges” in EMBASE and MEDLINE. J Can Health Libr Assoc. 2010;31(2):27–37. DOI: http://doi.org/10.5596/c10-022.
Hunt DL, McKibbon KA. Locating and appraising systematic reviews. Ann Intern Med. 1997;126:532–8.
McAlister FA, Clark HD, van Walraven C, Straus SE, Lawson FME, Moher D, Mulrow CD. The medical review article revisited: has the science improved? Ann Intern Med. 1999;131(12):947–51. DOI: http://dx.doi.org/10.7326/0003-4819-131-12-199912210-00007.
US National Library of Medicine. Search strategy used to create the systematic reviews subset on PubMed [Internet]. 2019 [cited 24 Nov 2020]. <https://www.nlm.nih.gov/bsd/pubmed_subsets/sysreviews_strategy.html>.
Cooper C, Booth A, Varley-Campbell J, Britten N, Garside R. Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies. BMC Med Res Methodol. 2018;18(1):1–14. DOI: http://doi.org/10.1186/s12874-018-0545-3.
Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. Ann Intern Med. 2009;151(4):264–9. DOI: http://doi.org/10.1371/journal.pmed1000097.
Institute of Medicine. Finding what works in health care. Standards for systematic reviews. Medicine. Washington, DC: National Academies Press; 2011. DOI: http://doi.org/10.17226/13059.
McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016;75:40–6. DOI: http://dx.doi.org/10.1016/j.jclinepi.2016.01.021.
Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA, PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1. DOI: http://doi.org/10.1186/2046-4053-4-1.
39. Budhram D, Navarro-Ruan T, Haynes RB. The efficiency of database searches for creating systematic reviews was improved by search filters. J Clin Epidemiol. 2018;95:1–6. DOI: http://doi.org/10.1016/j.jclinepi.2017.11.017.
Beyer FR, Wright K. Can we prioritise which databases to search? A case study using a systematic review of frozen shoulder management. Health Info Libr J. 2013;30(1):49–58. DOI: http://doi.org/10.1111/hir.12009.
Gargon E, Williamson PR, Clarke M. Collating the knowledge base for core outcome set development: Developing and appraising the search strategy for a systematic review. BMC Med Res Methodol. 2015;15(1):1–7. DOI: http://doi.org/10.1186/s12874-015-0019-9.
Lefebvre C, Glanville J, Briscoe S, Littlewood A, Marshall C, Metzendorf M-I, Noel-Storr A, Rader T, Shokraneh F, Thomas J, Wieland LS. Chapter 4: Searching for and selecting studies. In: Higgins J, Thomas J, editors. Cochrane Handbook for Systematic Reviews of Interventions. Version 6.1 [Internet]. Cochrane; 2020 [updated Sept 2020; cited 24 Nov 2020]. <http://training.cochrane.org/handbook>
Vassar M, Yerokhin V, Sinnett PM, Weiher M, Muckelrath H, Carr B, Varney L, Cook G. Database selection in systematic reviews: an insight through clinical neurology. Health Info Libr J. 2017;34(2):156–64. DOI: http://doi.org/10.1111/hir.12176.
Centre for Reviews and Dissemination. Systematic Reviews. CRD’s guidance for undertaking reviews in health care [Internet]. New York: Centre for Reviews and Dissemination, University of York; 2009. [cited 24 Nov 2020]. <https://www.york.ac.uk/media/crd/Systematic_Reviews.pdf>.
Bramer WM, Giustini D, Kramer B, Anderson PF. The comparative recall of Google Scholar versus PubMed in identical searches for biomedical systematic reviews: a review of searches used in systematic reviews. Syst Rev. 2013;2(115). DOI: http://doi.org/10.1002/jrsm.1279.
Jenkins M. Evaluation of methodological search filters—a review. Health Info Libr J. 2004;21(3):148–63. DOI: http://doi.org/10.1111/j.1471-1842.2004.00511.x.
BMJ Knowledge Center. BMJ Best Practice. Study design search filters [Internet]. 2020 [cited 24 Nov 2020]. <https://bestpractice.bmj.com/info/toolkit/learn-ebm/study-design-search-filters/>.
Canadian Agency for Drugs and Technologies in Health. Strings attached: CADTH's database search filters [Internet]. 2019 [cited 24 Nov 2020]. <https://www.cadth.ca/resources/finding-evidence/strings-attached-cadths-database-search-filters>.
Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005 Feb 1;8(1):19–32. DOI: http://doi.org/10.1080/1364557032000119616.
Munn Z, Stern C, Aromataris E, Lockwood C, Jordan Z. What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences. BMC Med Res Methodol. 2018;18(1):1–9. DOI: http://doi.org/10.1186/s12874-017-0468-4.
Morris M, Boruff JT, Gore GC. Scoping reviews: establishing the role of the librarian. J Med Libr Assoc. 2016;104(4):346–54. DOI: http://doi.org/10.3163/1536-5050.104.4.020.
Colquhoun HL, Levac D, O’Brien KK, Straus S, Tricco AC, Perrier L, Kastner M, Moher D. Scoping reviews: time for clarity in definition, methods, and reporting. J Clin Epidemiol. 2014;67(12):1291–4. DOI: http://doi.org/10.1016/j.jclinepi.2014.03.013.
Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, Mcewen SA. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods. 2014;5(4):371–85. DOI: http://doi.org/10.1002/jrsm.1123.
Armstrong R, Hall BJ, Doyle J, Waters E. “Scoping the scope” of a Cochrane review. J Public Health. 2011;33(1):147–50. DOI: http://doi.org/10.1093/pubmed/fdr015.
Thomson D, Russell K, Becker L, Klassen T, Hartling L. The evolution of a new publication type: steps and challenges of producing overviews of reviews. Res Synth Methods. 2011;1(3–4):198–211. DOI: http://doi.org/10.1002/jrsm.30.
Higgins J, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions. Version 6.1 [Internet]. Cochrane; 2020 [updated Sept 2020; cited 24 Nov 2020]. <https://www.training.cochrane.org/handbook>.
Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009;26(2):91–108. DOI: http://doi.org/10.1111/j.1471-1842.2009.00848.x.
Whittemore R, Knafl K. The integrative review: updated methodology. J Adv Nurs. 2005;52(2):546–53. DOI: http://doi.org/10.1111/j.1365-2648.2005.03621.x.
Cooper H, Hedges L V, Valentine JC. The handbook of research synthesis and meta-analysis, second edition. New York, NY: Russell Sage Foundation; 2009.
Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009;9:59. DOI: http://doi.org/10.1186/1471-2288-9-59.
Mohammed MA, Moles RJ, Chen TF. Meta-synthesis of qualitative research: the challenges and opportunities. Int J Clin Pharm. 2016;38(3):695–704. DOI: http://doi.org/10.1007/s11096-016-0289-2.
O’Connor A, Sargeant J. Research synthesis in veterinary science: narrative reviews, systematic reviews and meta-analysis. Vet J. 2015;206(3):261–7. DOI: http://dx.doi.org/10.1016/j.tvjl.2015.08.025.
Rethlefsen ML, Kirtley S, Waffenschmidt S, Ayala AP, Moher D, Page MJ, Koffel JB; PRISMA-S Group. PRISMA-S: an extension to the PRISMA Statement for reporting literature searches in systematic reviews. Syst Rev. 2021;10(1):39. DOI: http://dx.doi.org/10.1186/s13643-020-01542-z.
Puljak L. Methodological studies evaluating evidence are not systematic reviews. J Clin Epidemiol. 2019;110:99–100. DOI: http://doi.org/10.1016/j.jclinepi.2019.02.002.
MacLure K, Paudyal V, Stewart D. Reviewing the literature, how systematic is systematic? Int J Clin Pharm. 2016;38(3):685–94. DOI: http://doi.org/10.1007/s11096-016-0288-3.
Ioannidis JPA. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q. 2016;94(3):485–514. DOI: http://doi.org/10.1111/1468-0009.12210.
Page MJ, Moher D. Evaluations of the uptake and impact of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement and extensions: a scoping review. Syst Rev. 2017; 6:1–14. DOI: http://doi.org/10.1186/s13643-017-0663-8.
Page MJ, Shamseer L, Altman DG, Tetzlaff J, Sampson M, Tricco AC, Catalá-López F, Li L, Reid EK, Shakis-Onofre R, Moher D. Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med. 2016;13(5):1–30. DOI: http://doi.org/10.1371/journal.pmed.1002028.
Faggion CM, Huivin R, Aranda L, Pandis N, Alarcon M. The search and selection for primary studies in systematic reviews published in dental journals indexed in MEDLINE was not fully reproducible. J Clin Epidemiol. 2018;98:53–61. DOI: http://doi.org/10.1016/j.jclinepi.2018.02.011.
Biocic M, Fidahic M, Puljak L. Reproducibility of search strategies of non-Cochrane systematic reviews published in anaesthesiology journals is suboptimal: primary methodological study. Br J Anaesth. 2019;122(6):e79–81. DOI: http://doi.org/10.1016/j.bja.2019.02.014.
Additional Files
- Appendix 5. Phrases of articles that contain the terms systematic and review in the title and are not retrieved in the systematic reviews [sb]
- Appendix 4. Comparison of the terms used in the final search filter with those used in other studies.
- Appendix 3. Tables 5 and 6
- Appendix 2. Concepts and terms
- Appendix 1. Terms and frequency
