The NICE OECD countries’ geographic search filters: Part 2—validation of the MEDLINE and Embase (Ovid) filters
DOI:
https://doi.org/10.5195/jmla.2021.1224Keywords:
search filter, literature searching, geographyAbstract
Objective: We previously developed draft MEDLINE and Embase (Ovid) geographic search filters for Organisation for Economic Co-operation and Development (OECD) countries to assess their feasibility for finding evidence about the countries. Here, we describe the validation of these search filters.
Methods: We identified OECD country references from thirty National Institute for Health and Care Excellence (NICE) guidelines to generate gold standard sets for MEDLINE (n=2,065) and Embase (n=2,023). We validated the filters by calculating their recall against these sets. We then applied the filters to existing search strategies for three OECD-focused NICE guideline reviews (NG103 on flu vaccination, NG140 on abortion care, and NG146 on workplace health) to calculate the filters’ impact on the number needed to read (NNR) of the searches.
Results: The filters both achieved 99.95% recall against the gold standard sets. Both filters achieved 100% recall for the three NICE guideline reviews. The MEDLINE filter reduced NNR from 256 to 232 for the NG103 review, from 38 to 27 for the NG140 review, and from 631 to 591 for the NG146 review. The Embase filter reduced NNR from 373 to 341 for the NG103 review, from 101 to 76 for the NG140 review, and from 989 to 925 for the NG146 review.
Conclusion: The NICE OECD countries’ search filters are the first validated filters for the countries. They can save time for research topics about OECD countries by finding the majority of evidence about OECD countries while reducing search result volumes in comparison to no filter use.
References
Glanville J, Bayliss S, Booth A, Dundar Y, Fernandes H, Fleeman ND, Foster L, Fraser C, Fry-Smith A, Golder S, Lefebvre C, Miller C, Paisley S, Payne L, Price A, Welch K. So many filters, so little time: the development of a search filter appraisal checklist. J Med Libr Assoc. 2008 Oct;96(4):356–61. DOI: http://dx.doi.org/10.3163/1536-5050.96.4.011.
Jenkins M. Evaluation of methodological search filters: a review. Health Info Libr J. 2004 Aug;21(3):148–63. DOI: https://doi.org/10.1111/j.1471-1842.2004.00511.x.
Ayiku L, Craven J, Hudson T, Levay P. How to develop a validated geographic search filter: five key steps. EBLIP. 2020 Mar;15(1):170–8. DOI: https://doi.org/10.18438/eblip29633.
Bachmann L, Coray R, Estermann P, Ter Riet G. Identifying diagnostic studies in MEDLINE: reducing the number needed to read. J Am Med Inform Assoc. 2002 Nov-Dec;9(6):653–8. DOI: https://doi.org/10.1197/jamia.m1124.
Polanin JR, Pigott TD, Espelage DL, Grotpeter JK. Best practice guidelines for abstract screening large‐evidence systematic reviews and meta‐analyses. Res Synth Methods. 2019 May;10(3):330–42. DOI: https://doi.org/10.1002/jrsm.1354.
Organisation for Economic Co-operation and Development. Organisation for Economic Co-operation and Development website [Internet]. Paris: Organisation for Economic Co-operation and Development; 20] [cited 2 Feb 2021]. <http://www.oecd.org/>.
Ayiku L, Levay P, Hudson T. The NICE OECD countries geographic search filters: Part 1 - methodology for developing the draft MEDLINE and Embase (Ovid) filters. J Med Libr Assoc. 2021 Apr;109(2):258–66. DOI: https://doi.org/10.5195/jmla.2021.978.
Ayiku L, Levay P, Hudson T, Craven J, Barrett E, Finnegan A, Adams R. The MEDLINE UK filter: development and validation of a geographic search filter to retrieve research about the UK from OVID MEDLINE. Health Info Libr J. 2017 Jul;34(3):200–16. DOI: https://onlinelibrary.wiley.com/doi/full/10.1111/hir.12187
Ayiku L, Levay P, Hudson T, Craven J, Finnegan A, Adams R, Barrett E. The Embase UK filter: validation of a geographic search filter to retrieve research about the UK from OVID Embase. Health Info Libr J. 2019 Mar;36(2):121–33. DOI: https://onlinelibrary.wiley.com/doi/full/10.1111/hir.12252
Dickersin K, Scherer R, Lefebvre C. Identifying relevant studies for systematic reviews. BMJ. 1994 Nov;309(6964):1286–91. DOI: https://doi.org/10.1136/bmj.309.6964.1286.
Beynon R, Leeflang MM, McDonald S, Eisinga A, Mitchell RL, Whiting P, Glanville JM. Search strategies to identify diagnostic accuracy studies in MEDLINE and EMBASE. Cochrane Database Syst Rev. 2013 Sept;(9):Mr000022. DOI: https://doi.org/10.1002/14651858.MR000022.pub3.
Sampson M, Zhang L, Morrison A, Barrowman NJ, Clifford TJ, Platt RW, Klassen TP, Moher D. An alternative to the hand searching gold standard: validating methodological search filters using relative recall. BMC Med Res Methodol. 2006 Jul;6:33. DOI: https://doi.org/10.1186/1471-2288-6-33.
Neilson C, Lê L. A failed attempt at developing a search filter for systematic review methodology articles in Ovid Embase. J Med Libr Assoc. 2019 Apr;107(2):203–9. DOI: https://doi.org/10.5195/jmla.2019.519.
National Institute for Health and Care Excellence. NG103 Flu vaccination: increasing uptake. Evidence review 3: increasing uptake in clinical risk groups [Internet]. London: National Institute for Health and Care Excellence; 2018 [cited 2 Feb 2021]. <https://www.nice.org.uk/guidance/ng103/evidence/3-increasing-flu-vaccination-uptake-in-clinical-risk-groups-pdf-6532083616>.
National Institute for Health and Care Excellence. NG140 Abortion care. Evidence review P: Contraception after abortion [Internet]. London: National Institute for Health and Care Excellence; 2019 [cited 2 Feb 2021]. <https://www.nice.org.uk/guidance/ng140/evidence/p-contraception-after-abortion-pdf-248581907034>.
National Institute for Health and Care Excellence. NG146 Workplace health: long-term sickness absence and capability to work. Evidence review C: Facilitating the return to work of employees on long-term sickness absence and reducing risk of recurrence [Internet]. London: National Institute for Health and Care Excellence; 2019 [cited 2 Feb 2021]. <https://www.nice.org.uk/guidance/ng146/evidence/c-facilitating-the-return-to-work-of-employees-on-longterm-sickness-absence-and-reducing-risk-of-recurrence-pdf-6967146928>.
Ukah UV, Payne B, Lee T, Magee LA, von Dadelszen P, fullPIERS and miniPIERS Working Groups. External validation of the full PIERS Model for predicting adverse maternal outcomes in pregnancy hypertension in low- and middle-income countries. Hypertension. 2017 Apr;69(4):705–11. DOI: https://doi.org/10.1161/hypertensionaha.116.08706.
Payne BA, Hutcheon JA, Ansermino JM, Hall DR, Bhutta ZA, Bhutta SZ, Biryabarema C, Grobman WA, Groen H, Haniff F, Li J, Magee LA, Merialdi M, Nakimuli A, Qu Z, Sikandar R, Sass N, Sawchuck D, Steyn DW, Widmer M, Zhou J, von Dadelszen P, miniPIERS Study Working Group. A risk prediction model for the assessment and triage of women with hypertensive disorders of pregnancy in low-resourced settings: the miniPIERS (Pre-eclampsia Integrated Estimate of RiSk) multi-country prospective cohort study. PLoS Med. 2014 Jan;11(1):e1001589. DOI: https://doi.org/10.1371/journal.pmed.1001589.
Lefebvre C, Glanville J, Briscoe S, Littlewood A, Marshall C, Metzendorf M, Noel‐Storr A, Rader T, Shokraneh F, Thomas JL, Wieland S on behalf of the Cochrane Information Retrieval Methods Group. Chapter 4: Searching for and selecting studies. In: Higgins JPT, Thomas J, Chandler J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 6 [Internet]. London: Cochrane; 2020 [cited 2 Feb 2021]. <https://onlinelibrary.wiley.com/doi/abs/10.1002/9781119536604.ch4>.
Macmillan Dictionary. The BRIC countries [Internet]. London: Macmillan; 2021 [cited 2 Feb 2021]. <https://www.macmillandictionary.com/dictionary/british/the-bric-countries>.
The World Bank Group. High income [Internet]. Washington: The World Bank Group: Washington; 2021 [cited 2 Feb 2021]. <https://data.worldbank.org/income-level/high-income?view=chart>.
Glanville J, Lefebvre C, Wright K. The InterTASC information specialists’ sub‐group search filter resource [Internet]. York: University of York; 2021 [cited 2 Feb 2021]. <https://sites.google.com/a/york.ac.uk/issg-search-filters-resource/home>.
Pienaar E, Grobler L, Busgeeth K, Eisinga A, Siegfried N. Developing a geographic search filter to identify randomised controlled trials in Africa: finding the optimal balance between sensitivity and precision. Health Info Libr J. 2011; 28(3):210–5. DOI: https://doi.org/10.1111/j.1471-1842.2011.00936.x.
Valderas J, Mendivil J, Parada A, Losada‐Yáñez M, Alonso J. Development of a geographic filter for PubMed to identify studies performed in Spain. Revista Española de Cardiología. 2006; 59(12):1244–51. DOI: https://doi.org/10.1016/S1885-5857(07)60080-2.
Cheung A, Popoff E, Szabo SM. Using text-mining to develop a U.S.-specific geographic search filter to facilitate systematic reviews in Ovid MEDLINE. In: Advances in Evidence Synthesis: special issue Cochrane Database of Systematic Reviews 2020;(9 Suppl 1). https://doi.org/10.1002/14651858.CD202001.
