Artificial intelligence behind the scenes: PubMed’s Best Match algorithm
DOI:
https://doi.org/10.5195/jmla.2022.1236Keywords:
PubMed, Best Match, information systems, artificial intelligence, information-seeking behaviorAbstract
This article focuses on PubMed’s Best Match sorting algorithm, presenting a simplified explanation of how it operates and highlighting how artificial intelligence affects search results in ways that are not seen by users. We further discuss user search behaviors and the ethical implications of algorithms, specifically for health care practitioners. PubMed recently began using artificial intelligence to improve the sorting of search results using a Best Match option. In 2020, PubMed deployed this algorithm as the default search method, necessitating serious discussion around the ethics of this and similar algorithms, as users do not always know when an algorithm uses artificial intelligence, what artificial intelligence is, and how it may impact their everyday tasks. These implications resonate strongly in health care, in which the speed and relevancy of search results is crucial but does not negate the importance of a lack of bias in how those search results are selected or presented to the user. As a health care provider will not often venture past the first few results in search of a clinical decision, will Best Match help them find the answers they need more quickly? Or will the algorithm bias their results, leading to the potential suppression of more recent or relevant results?
References
Kitchin R. Thinking critically about and researching algorithms. Information, Communication & Society. 2017;20(1):14–29. DOI: https://doi.org/10.1080/1369118X.2016.1154087.
Napoli PM. Automated media: an institutional theory perspective on algorithmic media production and consumption. Commun Theory. 2014;(24)3:340–60. DOI: https://doi.org/10.1111/comt.12039.
Ananny M. Toward an ethics of algorithms: convening, observation, probability, and timeliness. Sci Technol Human Values. 2016;41(1):93–117. DOI: https://doi.org/10.1177%2F0162243915606523.
Fiorini N, Leaman R, Lipman DJ, Lu Z. How user intelligence is improving PubMed. Nat Biotechnol. 2018;36(10):937–45. <https://www.nature.com/articles/nbt.4267>.
Merriam-Webster.com dictionary. n.d. Ethical. [accessed June 10, 2021].
American Medical Association. Code of medical ethics preface & preamble [Internet]. 2016. <https://www.ama-assn.org/about/publications-newsletters/code-medical-ethics-preface-preamble>.
Dilmengani C. Bias in AI: what it is, types & examples of bias & tools to fix it [Internet]. AI Multiple; Sept 12 2021 [updated Oct 8 2021; cited June 3 2021]. <https://research.aimultiple.com/ai-bias/>.
Danks D, London AJ. Algorithmic bias in autonomous systems. IJCAI. 2017:4691–7.
National Library of Medicine. PubMed Overview National Library of Medicine [cited Sept 6 2019]. <https://pubmed.ncbi.nlm.nih.gov/about/>.
National Library of Medicine. Key MEDLINE indicators National Library of Medicine [cited Aug 27 2018]. <https://www.nlm.nih.gov/bsd/bsd_key.html>.
National Library of Medicine. Welcome to Medical Subject Headings [cited Dec 5 2019]. <https://www.nlm.nih.gov/mesh/meshhome.html>.
Fiorini N, Canese K, Starchenko G, Kireev E, Kim W, Miller V, Osipov M, Kholodov M, Ismagilov R, Mohan S, Ostell J, Lu Z. Best Match: new relevance search for PubMed. PLoS Biol. 2018 Aug;16(8):e2005343. DOI: https://doi.org/10.1371/journal.pbio.2005343.
Aakre CA, Maggio LA, Fiol GD, Cook DA. Barriers and facilitators to clinical information seeking: a systematic review. J Am Med Inform Assoc. 2019 Oct 1;26(10):1129–40. <https://www.ncbi.nlm.nih.gov/pubmed/31127830>.
Scott SD, Albrecht L, Given LM, Hartling L, Johnson DW, Jabbour M, Klassen TP. Pediatric information seeking behaviour, information needs, and information preferences of health care professionals in general emergency departments: Results from the Translating Emergency Knowledge for Kids (TREKK) Needs Assessment. CJEM. 2018 Jan;20(1):89-99. https://www.ncbi.nlm.nih.gov/pubmed/28067181.
Beck JB, Tieder JS. Electronic resources preferred by pediatric hospitalists for clinical care. J Med Libr Assoc. 2015 Oct;103(4):177-83.
Herasevich V, Pickering BW, Peters SG, Cimino JJ, Homan JM, Ellsworth MA. A Survey from a Large Academic Medical Center. Applied Clinical Informatics. 2017 06(02):305-17.
Brennan N, Edwards S, Kelly N, Miller A, Harrower L, Mattick K. Qualified doctor and medical students' use of resources for accessing information: what is used and why? Health Info Libr J. 2014 Sep;31(3):204–14.
Xia L, Deng S, Liu Y. Seeking health information online: the moderating effects of problematic situations on user intention. J Data Inf Sci. 2017;2(2):76–95.
Lee DL, Huei C, Seamons K. Document ranking and the vector-space model. IEEE Software. 1997;14(2):67–75. <https://ieeexplore.ieee.org/document/582976/?arnumber=582976>.
NCBI. Machine-learning based pipeline relying on LambdaMART currently used in PubMed for relevance (Best Match) searches GitHub. [cited April 30 2018]. <https://github.com/ncbi-nlp/PubMed-Best-Match>.
Turnbull D. OpenSource Connections2015. [cited 2020]. Available from: https://opensourceconnections.com/blog/2015/10/16/bm25-the-next-generation-of-lucene-relevation/.
Incorporating values for indexing method in MEDLINE/PubMed XML. NLM Tech Bull 2018;423(e2). Available from: https://www.nlm.nih.gov/pubs/techbull/ja18/ja18_indexing_method.html.
Marshall IJ, Wallace BC. Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Syst Rev. 2019 Jul 11;8(1):163. <https://www.ncbi.nlm.nih.gov/pubmed/31296265>.
Mittelstadt BD, Allo P, Taddeo M, Wachter S, Floridi L. The ethics of algorithms: mapping the debate. Big Data Soc. 2016;3(2). DOI: https://doi.org/10.1177/2053951716679679.
Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, eds. Cochrane handbook for systematic reviews of interventions. Hoboken, NJ: Wiley-Blackwell; 2019. Available from: https://doi.org/10.1002/9781119536604.
Glasziou PP, Sawicki PT, Prasad K, Montori VM. Not a medical course, but a life course. 2011;86(11):e4. DOI: https://doi.org/ 10.1097/ACM.0b013e3182320ec9.
American Medical Association. AMA passes first policy recommendations on augmented intelligence. American Medical Association; 2018. Available from: https://www.ama-assn.org/press-center/press-releases/ama-passes-first-policy-recommendations-augmented-intelligence.
American Medical Association. Board policy summary: augmented intelligence in health care. American Medical Association; 2019.Available from: https://www.ama-assn.org/system/files/2019-08/ai-2018-board-policy-summary.pdf.
Sampson M, Nama N, O'Hearn K, Murto K, Nasr A, Katz SL, Macartney G, Momoli F, McNally JD. Creating enriched training sets of eligible studies for large systematic reviews: the utility of PubMed's Best Match algorithm. Int J Technol Assess Health Care. 2020 Dec 18;37:e7.
Published
Issue
Section
License
Copyright (c) 2022 Lucy Kiester, Clara Turp
This work is licensed under a Creative Commons Attribution 4.0 International License.
