Translation and validation of PubMed and Embase search filters for identification of systematic reviews, intervention studies, and observational studies in the field of first aid
DOI:
https://doi.org/10.5195/jmla.2021.1219Keywords:
search filters, guideline development, study design filters, first aidAbstract
Objective: The aim of this project was to validate search filters for systematic reviews, intervention studies, and observational studies translated from Ovid MEDLINE and Embase syntax and used for searches in PubMed and Embase.com during the development of evidence summaries supporting first aid guidelines. We aimed to achieve a balance among recall, specificity, precision, and number needed to read (NNR).
Methods: Reference gold standards were constructed per study type derived from existing evidence summaries. Search filter performance was assessed through retrospective searches and measurement of relative recall, specificity, precision, and NNR when using the translated search filters. Where necessary, search filters were optimized. Adapted filters were validated in separate validation gold standards.
Results: Search filters for systematic reviews and observational studies reached recall of ≥85% in both PubMed and Embase. Corresponding specificities for systematic review filters were ≥96% in both databases, with a precision of 9.7% (NNR 10) in PubMed and 5.4% (NNR 19) in Embase. For observational study filters, specificity, precision, and NNR were 68%, 2%, and 51 in PubMed and 47%, 0.8%, and 123 in Embase, respectively. These filters were considered sufficiently effective. Search filters for intervention studies reached a recall of 85% and 83% in PubMed and Embase, respectively. Optimization led to recall of ≥95% with specificity, precision, and NNR of 49%, 1.3%, and 79 in PubMed and 56%, 0.74%, and 136 in Embase, respectively.
Conclusions: We report validated filters to search for systematic reviews, observational studies, and intervention studies in guideline projects in PubMed and Embase.com.
References
Wallace BC, Trikalinos TA, Lau J, Brodley C, Schmid CH. Semi-automated screening of biomedical citations for systematic reviews. BMC Bioinformatics. 2010 Jan 26;11:55. DOI: http://doi.org/10.1186/1471-2105-11-55.
Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, Fervers B, Graham ID, Grimshaw J, Hanna SE, Littlejohns P, Makarski J, Zitzelsberger L, Consortium ANS. AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ. 2010 Dec 14;182(18):E839–42. DOI: https://doi.org/10.1503/cmaj.090449.
Borra V, Van Remoortel H, Vande Veegaete A, De Buck E, Vandekerckhove P. Development of evidence-based first aid guidelines for laypeople in Flanders, Belgium. J Epidemiol Public Health Rev. 2016;1(4). DOI: https://www.doi.org/10.16966/2471-8211.124.
De Buck E, Van Remoortel H, Geuvens H, vande Veegaete A, Singhal M, Vandekerckhove P, Agarwal SP. Evidence-based first aid and prevention guidelines for laypeople in India. Ind J Community Health 2015 27(2):176–90. https://www.iapsmupuk.org/journal/index.php/IJCH/article/view/555.
Van de Velde S, De Buck E, Vandekerckhove P, Volmink J. Evidence-based African first aid guidelines and training materials. PLoS Med. 2011 Jul;8(7):e1001059. DOI: https://doi.org/10.1371/journal.pmed.1001059.
De Buck E, Van Remoortel H, Dieltjens T, Verstraeten H, Clarysse M, Moens O, Vandekerckhove P. Evidence-based educational pathway for the integration of first aid training in school curricula. Resuscitation. 2015 Sep;94:8–22. DOI: https://doi.org/10.1016/j.resuscitation.2015.06.008.
De Buck E, Laermans J, Vanhove AC, Dockx K, Vandekerckhove P, Geduld H. An educational pathway and teaching materials for first aid training of children in sub-Saharan Africa based on the best available evidence. BMC Public Health. 2020 Jun 3;20(1):836. DOI: https://doi.org/10.1186/s12889-020-08857-5.
Centre for Evidence-Based Practice (CEBaP). Development of evidence-based practice guidelines and systematic reviews: methodological charter [Internet]. Mechelen, Belgium: Belgian Red Cross [cited Nov 17 2020]. <https://www.cebap.org/storage/cebap/inf-methodology-charter-cebap-v2.00.pdf>.
Scottish Intercollegiate Guidelines Network. Search filters [Internet]. Edinburgh, UK: Healthcare Improvement Scotland [cited Nov 25 2020]. <https://www.sign.ac.uk/what-we-do/methodology/search-filters/>.
Cochrane Effective Practice and Organisation of Care (EPOC) group. About us [Internet]. London, UK: The Cochrane Collaboration [cited Nov 25th 2020]. <https://epoc.cochrane.org/>.
The InterTASC Information Specialists’ Sub-Group. ISSG Search Filter Resource [Internet]. [cited Nov 25 2020]. <https://sites.google.com/a/york.ac.uk/issg-search-filters-resource/home>.
The InterTASC Information Specialists’ Sub-Group. What are search filters? [cited Mar 5 2021]. <https://sites.google.com/a/york.ac.uk/issg-search-filters-resource/home/what-is-the-issg-search-filter-resource#h.a2hpzd2997f8>.
Pienaar E, Grobler L, Busgeeth K, Eisinga A, Siegfried N. Developing a geographic search filter to identify randomised controlled trials in Africa: finding the optimal balance between sensitivity and precision. Health Info Libr J. 2011 Sep;28(3):210–5. DOI: https://doi.org/10.1111/j.1471-1842.2011.00936.
Pols DH, Bramer WM, Bindels PJ, van de Laar FA, Bohnen AM. Development and validation of search filters to identify articles on family medicine in online medical databases. Ann Fam Med. 2015 Jul-Aug;13(4):364–6. DOI: https://doi.org/10.1370/afm.1780.
Lefebvre C, Glanville J, Beale S, Boachie C, Duffy S, Fraser C, Harbour J, McCool R, Smith L. Assessing the performance of methodological search filters to improve the efficiency of evidence information retrieval: five literature reviews and a qualitative study. Health Technol Assess. 2017 Nov;21(69):1–148. DOI: https://doi.org/10.3310/hta21690.
Van de Glind EM, Van Munster BC, Spijker R, Scholten RJ, Hooft L. Search filters to identify geriatric medicine in Medline. J Am Med Inform Assoc. 2012 May-Jun;19(3):468–72. DOI: https://doi.org/10.1136/amiajnl-2011-000319.
Dickersin K, Scherer R, Lefebvre C. Identifying relevant studies for systematic reviews. BMJ. 1994 Nov 12;309(6964):1286–91. DOI: https://doi.org/10.1136/bmj.309.6964.1286.
White VJ, Glanville JM, Lefebvre C, Sheldon TA. A statistical approach to designing search filters to find systematic reviews: objectivity enhances accuracy. Journal of Information Science. 2001;27(6):357–70. DOI: https://doi.org/10.1177/016555150102700601.
Glanville JM, Lefebvre C, Miles JN, Camosso-Stefinovic J. How to identify randomized controlled trials in MEDLINE: ten years on. J Med Libr Assoc. 2006 Apr;94(2):130–6. DOI: https://www.ncbi.nlm.nih.gov/pubmed/16636704.
Bachmann LM, Coray R, Estermann P, Ter Riet G. Identifying diagnostic studies in MEDLINE: reducing the number needed to read. J Am Med Inform Assoc. 2002 Nov-Dec;9(6):653–8. DOI: https://doi.org/10.1197/jamia.M1124.
Sampson M, Zhang L, Morrison A, Barrowman NJ, Clifford TJ, Platt RW, Klassen TP, Moher D. An alternative to the hand searching gold standard: validating methodological search filters using relative recall. BMC Med Res Methodol. 2006 Jul 18;6:33. DOI: https://doi.org/10.1186/1471-2288-6-33.
Centre for Evidence-Based Practice (CEBaP). Certificate: ISO 9001: 2015 Quality Management System [cited Jan 19th 2021]. <https://www.cebap.org/storage/cebap/20-qms-6556.pdf>.
Centre for Evidence-Based Practice (CEBaP). First aid evidence summaries [Internet]. Mechelen, Belgium: Belgian Red Cross [cited Nov 19 2020]. <https://www.cebap.org/knowledge-dissemination/first-aid-evidence-summaries/>.
Flahault A, Cadilhac M, Thomas G. Sample size calculation should be performed for design accuracy in diagnostic test studies. J Clin Epidemiol. 2005 Aug;58(8):859–62. DOI: https://doi.org/10.1016/j.jclinepi.2004.12.009.
Jenkins M. Evaluation of methodological search filters—a review. Health Info Libr J. 2004 21(3):148–63. DOI: https://doi.org/10.1111/j.1471-1842.2004.00511.x.
Singletary EM, Zideman DA, Bendall JC, Berry DA, Borra V, Carlson JN, Cassan P, Chang WT, Charlton NP, Djarv T, Douma MJ, Epstein JL, Hood NA, Markenson DS, Meyran D, Orkin A, Sakamoto T, Swain JM, Woodin JA, De Buck E, De Brier N, O D, Picard C, Goolsby C, Oliver E, Klaassen B, Poole K, Aves T, Lin S, Handley AJ, Jensen J, Allan KS, Lee CC, First Aid Science C. 2020 International Consensus on First Aid Science With Treatment Recommendations. Resuscitation. 2020 Nov;156:A240-A82. DOI: https://doi.org/10.1161/CIR.0000000000000897.
International Federation of Red Cross and Red Crescent Societies. International first aid and resuscitation guidelines Geneva: International Federation of Red Cross/Red Crescent Societies [cited Jan 27 2021]. <https://www.ifrc.org/Global/Publications/Health/First-Aid-2016-Guidelines_EN.pdf>.
Leenaars M, Hooijmans CR, Van Veggel N, Ter Riet G, Leeflang M, Hooft L, Van der Wilt GJ, Tillema A, Ritskes-Hoitinga M. A step-by-step guide to systematically identify all relevant animal studies. Lab Anim. 2012 Jan;46(1):24–31. DOI: https://doi.org/10.1258/la.2011.011087.
Lee E, Dobbins M, Decorby K, McRae L, Tirilis D, Husson H. An optimal search filter for retrieving systematic reviews and meta-analyses. BMC Med Res Methodol. 2012 Apr 18;12:51. DOI: https://doi.org/10.1186/1471-2288-12-51.
Glanville J, Foxlee R, Wisniewski S, Noel-Storr A, Edwards M, Dooley G. Translating the Cochrane EMBASE RCT filter from the Ovid interface to Embase.com: a case study. Health Info Libr J. 2019 Sep;36(3):264–77. DOI: https://doi.org/10.1111/hir.12269.
Wilczynski NL, Haynes RB, Hedges T. EMBASE search strategies achieved high sensitivity and specificity for retrieving methodologically sound systematic reviews. J Clin Epidemiol. 2007 Jan;60(1):29–33. DOI: https://doi.org/10.1016/j.jclinepi.2006.04.001.
Montori VM, Wilczynski NL, Morgan D, Haynes RB, Hedges T. Optimal search strategies for retrieving systematic reviews from Medline: analytical survey. BMJ. 2005 Jan 8;330(7482):68. DOI: https://doi.org/10.1136/bmj.38336.804167.4.
Boynton J, Glanville J, McDaid D, Lefebvre C. Identifying systematic reviews in MEDLINE: Developing an objective approach to search strategy design. J Inform Sci. 1998 24:137–54. DOI: https://doi.org/10.1177/016555159802400301.
Glanville J, Kotas E, Featherstone R, Dooley G. Which are the most sensitive search filters to identify randomized controlled trials in MEDLINE? J Med Libr Assoc. 2020 Oct 1;108(4):556–63. DOI: https://doi.org/10.5195/jmla.2020.912.
Waffenschmidt S, Navarro-Ruan T, Hobson N, Hausner E, Sauerland S, Haynes RB. Development and validation of study filters for identifying controlled non-randomized studies in PubMed and Ovid MEDLINE. Res Synth Methods. 2020 Sep;11(5):617–26. DOI: https://doi.org/10.1002/jrsm.1425.
Li L, Smith HE, Atun R, Tudor Car L. Search strategies to identify observational studies in MEDLINE and Embase. Cochrane Database Syst Rev. 2019 Mar 12;3:MR000041. DOI: https://doi.org/10.1002/14651858.MR000041.pub2.
Fraser C, Murray A, Burr J. Identifying observational studies of surgical interventions in MEDLINE and EMBASE. BMC Med Res Methodol. 2006 Aug 18;6:41. DOI: https://doi.org/10.1186/1471-2288-6-41.
Furlan AD, Irvin E, Bombardier C. Limited search strategies were effective in finding relevant nonrandomized studies. J Clin Epidemiol. 2006 Dec;59(12):1303–11. DOI: https://doi.org/10.1016/j.jclinepi.2006.03.004.
