Multicenter randomized comparative trial of Micromedex, Micromedex with Watson, or Google to answer drug information questions
DOI:
https://doi.org/10.5195/jmla.2021.1085Keywords:
Information Storage and Retrieval, Internet, Medical Informatics, Pharmacy StudentAbstract
Objective: The purpose of this study was to compare pharmacy students’ ability to correctly answer drug information questions using Micromedex with Watson, Micromedex without Watson, or Google.
Methods: This multicenter randomized trial compared pharmacy student responses to drug information questions using Micromedex with Watson, Micromedex without Watson, or Google from January to March of 2020. First- to fourth-year pharmacy students at two institutions were included. The primary outcome was the number of correct answers. Secondary outcomes were the time taken to answer the questions and differences in number of correct answers by pharmacy student year and institution.
Results: The analysis included 162 participants: 52 students in the Micromedex group, 51 students in the Watson group, and 59 students in the Google group. There was a significant difference among groups in the total number of questions answered correctly (p=0.02). Post-hoc analysis revealed that participants in the Micromedex group answered more questions correctly than those in the Google group (p=0.015). There were no significant differences between Micromedex and Watson groups (p=0.52) or between Watson and Google groups (p=0.22). There was also no difference in time to complete the questions among groups (p=0.72).
Conclusion: Utilizing Google did not save students time and led to more incorrect answers. These findings suggest that health care educators and health sciences librarians should further reinforce training on the appropriate use of drug information resources.
References
Rowlands I, Nicholas D. Information behaviour of the researcher of the future [Internet]. UK Web Archive (UKWA); 2008 [cited 15 Jul 2020]. <http://www.jisc.ac.uk/media/documents/programmes/reppres/gg_final_keynote_11012008.pdf>.
Spring H. Health professionals of the future: teaching information skills to the Google generation. Health Inf Libr J. 2010 Jun;27(2):158–62.
Wang L, Wang J, Wang M, Li Y, Liang Y, Xu D. Using Internet search engines to obtain medical information: a comparative study. J Med Internet Res. 2012 May 16;14(3):e74.
Trivedi H, Mesterhazy J, Laguna B, Vu T, Sohn JH. Automatic determination of the need for intravenous contrast in musculoskeletal MRI examinations using IBM Watson’s natural language processing algorithm. J Digit Imaging. 2018 Apr;31(2):245–51.
Somashekhar SP, Sepúlveda MJ, Puglielli S, Norden AD, Shortliffe EH, Rohit Kumar C, Rauthan A, Arun Kumar N, Patil P, Rhee K, Ramya Y. Watson for oncology and breast cancer treatment recommendations: agreement with an expert multidisciplinary tumor board. Ann Oncol. 2018 Feb 1;29(2):418–23.
Kim YY, Oh SJ, Chun YS, Lee WK, Park HK. Gene expression assay and Watson for oncology for optimization of treatment in ER-positive, HER2-negative breast cancer. PLoS One. 2018 Jul 6;13(7):e0200100.
Ely JW, Osheroff JA, Ebell MH, Bergus GR, Levy BT, Chambliss ML, Evans ER. Analysis of questions asked by family doctors regarding patient care. BMJ. 1999 Aug 7;319(7206):358–61.
Green ML, Ciampi MA, Ellis PJ. Residents’ medical information needs in clinic: are they being met? Am J Med. 2000 Aug 15;109(3):218–23.
O’Carroll AM, Westby EP, Dooley J, Gordon KE. Information-seeking behaviors of medical students: a cross-sectional web-based survey. JMIR Med Educ. 2015 Jun 29;1(1):e4.
Falagas ME, Ntziora F, Makris GC, Malietzis GA, Rafailidis PI. Do PubMed and Google searches help medical students and young doctors reach the correct diagnosis? a pilot study. Eur J Intern Med. 2009 Dec;20(8):788–90.
Jhaveri KD, Schrier PB, Mattana J. Paging doctor Google! heuristics vs. technology. F1000Res. 2013 Apr 10;2(90):e1–15.
Graber ML, Tompkins D, Holland JJ. Resources medical students use to derive a differential diagnosis. Med Teach. 2009 Jun;31(6):522–7.
Kim S, Noveck H, Galt J, Hogshire L, Willett L, O’Rourke K. Searching for answers to clinical questions using Google versus evidence-based summary resources: a randomized controlled crossover study. Acad Med. 2014 Jun;89(6):940–3.
Egle JP, Smeenge DM, Kassem KM, Mittal VK. The Internet school of medicine: use of electronic resources by medical trainees and the reliability of those resources. J Surg Educ. 2015 Mar–Apr;72(2):316–20.
