The NICE search filters for treating and managing COVID-19: validation in MEDLINE and Embase (Ovid)
DOI:
https://doi.org/10.5195/jmla.2024.1806Keywords:
Search filters, COVID-19, MEDLINE, Embase, Systematic literature reviewAbstract
Objective: In this paper we report how the United Kingdom's National Institute for Health and Care Excellence (NICE) search filters for treating and managing COVID-19 were validated for use in MEDLINE (Ovid) and Embase (Ovid). The objective was to achieve at least 98.9% for recall and 64% for precision.
Methods: We did two tests of recall to finalize the draft search filters. We updated the data from an earlier peer-reviewed publication for the first recall test. For the second test, we collated a set of systematic reviews from Epistemonikos COVID-19 L.OVE and extracted their primary studies. We calculated precision by screening all the results retrieved by the draft search filters from a targeted sample covering 2020-23. We developed a gold-standard set to validate the search filter by using all articles available from the "Treatment and Management" subject filter in the Cochrane COVID-19 Study Register.
Results: In the first recall test, both filters had 99.5% recall. In the second test, recall was 99.7% and 99.8% in MEDLINE and Embase respectively. Precision was 91.1% in a deduplicated sample of records. In validation, we found the MEDLINE filter had recall of 99.86% of the 14,625 records in the gold-standard set. The Embase filter had 99.88% recall of 19,371 records.
Conclusion: We have validated search filters to identify records on treating and managing COVID-19. The filters may require subsequent updates, if new SARS-CoV-2 variants of concern or interest are discussed in future literature.
References
World Health Organization. Statement on the fifteenth meeting of the IHR (2005) Emergency Committee on the COVID-19 pandemic [Internet]. 2023. Available from: https://www.who.int/news/item/05-05-2023-statement-on-the-fifteenth-meeting-of-the-international-health-regulations-(2005)-emergency-committee-regarding-the-coronavirus-disease-(covid-19)-pandemic.
Palayew A, Norgaard O, Safreed-Harmon K, Andersen TH, Rasmussen LN, Lazarus JV. Pandemic publishing poses a new COVID-19 challenge. Nat Hum Behav. 2020;4:666–9. Available from: http://www.nature.com/articles/s41562-020-0911-0.
Ayiku L, Levay P, Hudson T, Craven J, Barrett E, Finnegan A, Adams R. The MEDLINE UK filter: development and validation of a geographic search filter to retrieve research about the UK from OVID MEDLINE. Health Info Libr J. 2017;34(3):200–16. Available from: http://doi.wiley.com/10.1111/hir.12187.
Jenkins M. Evaluation of methodological search filters-a review. Heal Inf Libr J. 2004;21(3):148–63. Available from: http://doi.wiley.com/10.1111/j.1471-1842.2004.00511.x.
Sampson M, Zhang L, Morrison A, Barrowman NJ, Clifford TJ, Platt RW, Klassen TP, Moher D. An alternative to the hand searching gold standard: validating methodological search filters using relative recall. BMC Med Res Methodol. 2006;6(1):33. Available from: https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/1471-2288-6-33.
Wolters Kluwer. Embase: Excerpta Medica Database Guide [Internet]. 2023. Available from: https://ospguides.ovid.com/OSPguides/embase.htm.
Wolters Kluwer. MEDLINE 2021 Database Guide [Internet]. 2023. Available from: https://ospguides.ovid.com/OSPguides/medline.htm.
National Institute for Health and Care Excellence. Developing NICE guidelines: the manual [Internet]. Manchester; 2023. Available from: https://www.nice.org.uk/process/pmg20/chapter/introduction.
Southall S, Taske N, Power E, Desai M, Baillie N. Spotlight on COVID-19 rapid guidance: NICE’s experience of producing rapid guidelines during the pandemic. J Public Health. 2021;43(1):e103–6. Available from: https://academic.oup.com/jpubhealth/article/43/1/e103/5946945.
Levay P, Finnegan A. The NICE COVID-19 search strategy for Ovid MEDLINE and Embase: developing and maintaining a strategy to support rapid guidelines. medRxiv. 2021;2021.06.11.21258749. Available from: https://www.medrxiv.org/content/10.1101/2021.06.11.21258749v1.
Brody S, Loree S, Sampson M, Mensinkai S, Coffman J, Mueller MH, Askin N, Hamill C, Wilson E, McAteer MB, Staines H. Searching for evidence in public health emergencies: a white paper of best practices. J Med Libr Assoc. 2023;111(1/2):566–78. Available from: https://jmla.pitt.edu/ojs/jmla/article/view/1530.
World Health Organization. Naming the coronavirus disease (COVID-19) and the virus that causes it [Internet]. 2020. Available from: https://www.who.int/emergencies/diseases/novel-coronavirus-2019/technical-guidance/naming-the-coronavirus-disease-(covid-2019)-and-the-virus-that-causes-it.
Shokraneh F, Russell-Rose T. Lessons from COVID-19 to future evidence synthesis efforts: first living search strategy and out of date scientific publishing and indexing industry. J Clin Epidemiol. 2020;123:171–3. Available from: https://www.sciencedirect.com/science/article/pii/S0895435620303450.
World Health Organization. Tracking SARS-CoV-2 variants. 2021. Available from: https://www.who.int/en/activities/tracking-SARS-CoV-2-variants.
InterTASC Information Specialists’ Sub-Group. ISSG Search Filter Resource [Internet]. 2023. Available from: https://sites.google.com/a/york.ac.uk/issg-search-filters-resource/home.
Lazarus JV, Palayew A, Rasmussen LN, Andersen TH, Nicholson J, Norgaard O. Searching PubMed to retrieve publications on the COVID-19 pandemic: comparative analysis of search strings. J Med Internet Res. 2020;22(11):e23449. Available from: https://www.jmir.org/2020/11/e23449.
Shokraneh F, Adams CE. Study-based registers reduce waste in systematic reviewing: discussion and case report. Syst Rev. 2019;8(1):129. Available from: https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-019-1035-3.
Cochrane. Cochrane COVID-19 Study Register [Internet]. 2023. Available from: https://covid-19.cochrane.org.
Epistemonikos. COVID-19 Living OVerview of Evidence (L.OVE) [Internet]. 2023. Available from: https://app.iloveevidence.com/loves/5e6fdb9669c00e4ac072701d?utm=ile.
Metzendorf MI, Featherstone RM. Evaluation of the comprehensiveness, accuracy and currency of the Cochrane COVID-19 Study Register for supporting rapid evidence synthesis production. Res Synth Methods. 2021;12(5):607–17. Available from: https://onlinelibrary.wiley.com/doi/10.1002/jrsm.1501.
Verdugo-Paiva F, Vergara C, Ávila C, Castro-Guevara JA, Cid J, Contreras V, Jara I, Jiménez V, Lee MH, Muñoz M, Rojas-Gómez AM, Rosón-Rodríguez P, Serrano-Arévalo K, Silva-Ruz I, Vásquez-Laval J, Zambrano-Achig P, Zavadzki G, Rada G. COVID-19 Living OVerview of Evidence repository is highly comprehensive and can be used as a single source for COVID-19 studies. J Clin Epidemiol. 2022;149:195–202. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0895435622001172.
Shemilt I, Noel-Storr A, Thomas J, Featherstone R, Mavergames C. Machine learning reduced workload for the Cochrane COVID-19 Study Register: development and evaluation of the Cochrane COVID-19 Study Classifier. Syst Rev. 2022;11(1):15. Available from: https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-021-01880-6.
Ayiku L, Levay P, Hudson T, Craven J, Finnegan A, Adams R, Barrett E. The Embase UK filter: validation of a geographic search filter to retrieve research about the UK from OVID Embase. Health Info Libr J. 2019;36(2):121–33. Available from: https://onlinelibrary.wiley.com/doi/abs/10.1111/hir.12252.
Butcher R, Sampson M, Couban RJ, Malin JE, Loree S, Brody S. The currency and completeness of specialized databases of COVID-19 publications. J Clin Epidemiol. 2022;147:52–9. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0895435622000671.
Haddaway NR, Grainger MJ, Gray CT. Citationchaser: A tool for transparent and efficient forward and backward citation chasing in systematic searching. Res Synth Methods. 2022;13(4):533–45. Available from: https://onlinelibrary.wiley.com/doi/10.1002/jrsm.1563.
Siciliano RF, Gualandro DM, Bittencourt MS, Paixão M, Marcondes-Braga F, de Matos Soeiro A, Strunz C, Pacanaro AP, Puelacher C, Tarasoutchi F, Di Somma S, Caramelli B, de Oliveira Junior MT, Mansur AJ, Mueller C, Barretto ACP, Strabelli TMV. Biomarkers for prediction of mortality in left-sided infective endocarditis. Int J Infect Dis. 2020;96:25–30. Available from: https://linkinghub.elsevier.com/retrieve/pii/S1201971220301284.
Zhang L, Peng X, Zhao B, Zhu Z, Wang Y, Tian D, Yan Z, Yao L, Liu J, Qiu L, Xing X, Chen L. Clinical and laboratory features of female Gitelman syndrome and the pregnancy outcomes in a Chinese cohort. Nephrology. 2020;25(10):749–57. Available from: https://onlinelibrary.wiley.com/doi/10.1111/nep.13743.
Aqeel F, Geetha D. Tixagevimab and Cilgavimab (Evusheld) in Rituximab-treated Antineutrophil Cytoplasmic Antibody Vasculitis Patients. Kidney Int Reports. 2022;7(11):2537–8. Available from: https://linkinghub.elsevier.com/retrieve/pii/S246802492201720X.
Hama R. Imbalance in baseline characteristics in molnupiravir trials. BMJ. 2022;o977. Available from: https://www.bmj.com/content/377/bmj.o977.
Holmer HK, Mackey K, Fiordalisi CV., Armstrong C, Gean E, Arkhipova-Jenkins I, Helfand M. Antibody response following SARS-CoV-2 infection and implications for immunity: final update of a rapid, living review. 2022. Available from: https://effectivehealthcare.ahrq.gov/products/immunity-after-covid/rapid-review.
Brophy JM. Molnupiravir’s authorisation was premature. BMJ. 2022;o443. Available from: https://www.bmj.com/lookup/doi/10.1136/bmj.o443.
De Franceschi L, Costa E, Dima F, Morandi M, Olivieri O. Acute hemolysis by hydroxycloroquine was observed in G6PD-deficient patient with severe COVD-19 related lung injury. Eur J Intern Med. 2020;77:136–7. Available from: http://www.ncbi.nlm.nih.gov/pubmed/32381323.
Shukla U, Srivastava S, Gupta P, Ujjaliya N. A retrospective analysis of the effect of the intervention of Arogya Kashayam in COVID-19 positive cases in Madhya Pradesh. Ayu. 2019;40(4):209–15. Available from: http://www.ncbi.nlm.nih.gov/pubmed/33935437.
Lilli C, Biggeri A, Zingaretti C, Vertogen B, Frassineti V, Vespignani R, Grossi V, Florescu C, Matteucci L, Pazzi C, Bongiovanni A, Limarzi F, Fausti V, Bertoni L, Donati C, Galardi F, Gentili N, Mazza F, Martinelli G, Nanni O. È possibile condurre studi clinici durante una pandemia? L’esempio di uno studio sull’idrossiclorochina. Is it possible to conduct clinical trials during a pandemic? The example of a trial of hydroxychloroquine. Epidemiol Prev. 2021;45(1–2):28–36. Available from: http://www.ncbi.nlm.nih.gov/pubmed/33884840.
Boudry C, Howard K, Mouriaux F. Poor visibility of retracted articles: a problem that should no longer be ignored. BMJ. 2023;e072929. Available from: https://www.bmj.com/lookup/doi/10.1136/bmj-2022-072929.
Downloads
Published
Issue
Section
License
Copyright (c) 2023 Paul Levay, Amy Finnegan
This work is licensed under a Creative Commons Attribution 4.0 International License.
