Which are the most sensitive search filters to identify randomized controlled trials in MEDLINE?
DOI:
https://doi.org/10.5195/jmla.2020.912Keywords:
Literature Search Effectiveness, Literature Searching, Randomized Controlled Trials, Search FiltersAbstract
Objective: The Cochrane Handbook of Systematic Reviews contains search filters to find randomized controlled trials (RCTs) in Ovid MEDLINE: one maximizing sensitivity and another balancing sensitivity and precision. These filters were originally published in 1994 and were adapted and updated in 2008. To determine the performance of these filters, the authors tested them and thirty-six other MEDLINE filters against a large new gold standard set of relevant records.
Methods: We identified a gold standard set of RCT reports published in 2016 from the Cochrane CENTRAL database of controlled clinical trials. We retrieved the records in Ovid MEDLINE and combined these with each RCT filter. We calculated their sensitivity, relative precision, and f-scores.
Results: The gold standard comprised 27,617 records. MEDLINE searches were run on July 16, 2019. The most sensitive RCT filter was Duggan et al. (sensitivity=0.99). The Cochrane sensitivity-maximizing RCT filter had a sensitivity of 0.96 but was more precise than Duggan et al. (0.14 compared to 0.04 for Duggan). The most precise RCT filters had 0.97 relative precision and 0.83 sensitivity.
Conclusions: The Cochrane Ovid MEDLINE sensitivity-maximizing RCT filter can continue to be used by Cochrane reviewers and to populate CENTRAL, as it has very high sensitivity and a slightly better precision relative to more sensitive filters. The results of this study, which used a very large gold standard to compare the performance of all known RCT filters, allows searchers to make better informed decisions about which filters to use for their work.References
Higgins JPT, Green S, eds. Cochrane handbook for systematic reviews of interventions. Version 5.1.0. Cochrane; 2011 [updated Mar 2011].
Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, eds. Cochrane handbook for systematic reviews of interventions version 6.0. Cochrane; 2019 [updated Jul 2019].
Agency for Healthcare Research Quality. Methods guide for effectiveness and comparative effectiveness reviews. Rockville, MD: The Agency; 2014.
Committee on Standards for Systematic Reviews of Comparative Effectiveness Research; Institute of Medicine. Finding what works in health care: standards for systematic reviews. Washington, DC: National Academies Press; 2011.
Lefebvre C, Glanville J, Beale S, Boachie C, Duffy S, Fraser C, Harbour J, McCool R, Smith L. Assessing the performance of methodological search filters to improve the efficiency of evidence information retrieval: five literature reviews and a qualitative study. Health Technol Assess. Nov;21(69):1–148. DOI: http://dx.doi.org/10.3310/hta21690.
Cochrane Library. Cochrane Central Register of Controlled Trials (CENTRAL) [Internet]. The Library [cited 26 Jun 2020]. <https://www.cochranelibrary.com/central/about-central>.
InterTASC Information Specialists Sub-Group (ISSG). Search filter resource: filters to identify randomized controlled trials and other trials [Internet]. The Sub-Group [cited 26 Jun 2020]. <https://sites.google.com/a/york.ac.uk/issg-search-filters-resource/filters-to-identify-randomized-controlled-trials-and>.
McKibbon KA, Wilczynski NL, Haynes RB. Retrieving randomized controlled trials from MEDLINE: a comparison of 38 published search filters. Health Inf Libr J. 2009 Sep;26(3):187–202.
InterTASC Information Specialists Sub-Group (ISSG). Search filter resource: the InterTASC Information Specialists’ Sub-Group search filter resource [Internet]. The Sub-Group [cited 26 Jun 2020]. <https://sites.google.com/a/york.ac.uk/issg-search-filters-resource/home>.
Guns R, Lioma C, Larsen B. The tipping point: f-score as a function of the number of retrieved items. Inf Process Manag. 2012 Nov;48(6):1171–80. DOI: http://dx.doi.org/10.1016/j.ipm.2012.02.009.
Duggan et al. L, Morris M, Adams. C. Prevalence study of the randomized controlled trials in the Journal of Intellectual Disability Research: 1957–1994. J Intellect Disabil Res. 1997 Jun;41(3):232–7.
Dumbrigue HB, Esquivel JF, Jones JS. Assessment of MEDLINE search strategies for randomized controlled trials in prosthodontics. J Prosthodont. 2000 Mar;9(1):8–13.
Robinson KA, Dickersin K. Development of a highly sensitive search strategy for the retrieval of reports of controlled trials using PubMed. Int J Epidemiol. 2002 Feb;31(1):150–3.
Edward G. Miner Library. Evidence-based filters for Ovid MEDLINE [Internet]. Rochester, NY: University of Rochester; 2009 [cited 26 Jun 2020]. <https://urresearch.rochester.edu/fileDownloadForInstitutionalItem.action?itemId=5976&itemFileId=9379>.
National Center for Biotechnology Information, US National Library of Medicine. PubMed clinical queries [Internet]. Bethesda, MD: The Center; 2005 [cited 18 Dec 2019]. <https://www.ncbi.nlm.nih.gov/pubmed/clinical>.
Marson AG, Chadwick DW. How easy are randomized controlled trials in epilepsy to find on MEDLINE? the sensitivity and precision of two MEDLINE searches. Epilepsia. 1996 Apr;37(4):377–80.
Adams CE, Power A, Frederick K, Lefebvre C. An investigation of the adequacy of MEDLINE searches for randomized controlled trials (RCTs) of the effects of mental health care. Psychol Med. 1994 Aug;24(3):741–8.
Chow TKF, To E, Goodchild CS, McNeil JJ. A simple, fast, easy method to identify the evidence base in pain relief research: validation of a computer search strategy used alone to identify quality randomized controlled trials. Anest Analg. 2004 Jun;98(6):1557–565.
Royle P, Waugh N. Making literature searches easier: a rapid and sensitive search filter for retrieving randomized controlled trials from PubMed. Diabet Med. 2007 Mar;24(3):308–11.
Glanville JM, Lefebvre C, Miles JNV, Camosso-Stefinovic J. How to identify randomized controlled trials in MEDLINE: ten years on. J Med Libr Assoc. 2006 Apr;94(2):130–6.
Scottish Intercollegiate Guidelines Network. Search filters [Internet]. Edinburgh, UK: The Network [cited 18 Dec 2019]. <https://www.sign.ac.uk/search-filters.html>.
Dickersin K, Scherer R, Lefebvre C. Systematic reviews: identifying relevant studies for systematic reviews. BMJ. 1994 Nov 12;309(6964):1286–91.
Nwosu CR, Khan KS, Chien PF. A two-term MEDLINE search strategy for identifying randomized trials in obstetrics and gynecology. Obstet Gynecol. 1998 Apr;91(4):618–22.
Corrao S, Colomba D, Arnone S, Argano C, Di Chiara T, Scaglione R, Licata G. Improving efficacy of PubMed clinical queries for retrieving scientifically strong studies on treatment. J Am Med Inform Assoc. 2006 Sep–Oct;13(5):485–7.
Jadad A, McQuay H. A high-yield strategy to identify randomized controlled trials for systematic reviews. Online J Curr Clin Trials. 1993 Feb 27;doc no 33.
Eisinga A, Siegfried N, Clarke M. The sensitivity and precision of search terms in phases I, II and III of the Cochrane Highly Sensitive Search Strategy for identifying reports of randomized trials in MEDLINE in a specific area of health care—HIV/AIDS prevention and treatment interventions. Health Inf Libr J. 2007 Jun;24(2):103–9.
Sampson M, Zhang L, Morrison A, Barrowman NJ, Clifford TJ, Platt RW, Klassen TP, Moher D. An alternative to the hand searching gold standard: validating methodological search filters using relative recall. BMC Med Res Methodol. 2006 Jul 18;6:33.
Wallace BC, Noel-Storr A, Marshall IJ, Cohen AM, Smalheiser NR, Thomas J. Identifying reports of randomized controlled trials (RCTs) via a hybrid machine learning and crowdsourcing approach. J Am Med Inform Assoc. 2017 Nov 1;24(6):1165–8.
