Straight to the point: evaluation of a Point of Care Information (POCI) resource in answering disease-related questions
DOI:
https://doi.org/10.5195/jmla.2024.1770Keywords:
evidence-based information, Clinical Decision Support Systems, point-of-care resources, Information RetrievalAbstract
Objective: To evaluate the ability of DynaMedex, an evidence-based drug and disease Point of Care Information (POCI) resource, in answering clinical queries using keyword searches.
Methods: Real-world disease-related questions compiled from clinicians at an academic medical center, DynaMedex search query data, and medical board review resources were categorized into five clinical categories (complications & prognosis, diagnosis & clinical presentation, epidemiology, prevention & screening/monitoring, and treatment) and six specialties (cardiology, endocrinology, hematology-oncology, infectious disease, internal medicine, and neurology). A total of 265 disease-related questions were evaluated by pharmacist reviewers based on if an answer was found (yes, no), whether the answer was relevant (yes, no), difficulty in finding the answer (easy, not easy), cited best evidence available (yes, no), clinical practice guidelines included (yes, no), and level of detail provided (detailed, limited details).
Results: An answer was found for 259/265 questions (98%). Both reviewers found an answer for 241 questions (91%), neither found the answer for 6 questions (2%), and only one reviewer found an answer for 18 questions (7%). Both reviewers found a relevant answer 97% of the time when an answer was found. Of all relevant answers found, 68% were easy to find, 97% cited best quality of evidence available, 72% included clinical guidelines, and 95% were detailed. Recommendations for areas of resource improvement were identified.
Conclusions: The resource enabled reviewers to answer most questions easily with the best quality of evidence available, providing detailed answers and clinical guidelines, with a high level of replication of results across users.
References
Maggio LA, Aakre CA, Del Fiol G, Shellum J, Cook DA. Impact of clinicians' use of electronic knowledge resources on clinical and learning outcomes: Systematic review and meta-analysis. J Med Internet Res 2019;21:e13315. DOI:10.2196/13315.
Banzi R, Liberati A, Moschetti I, Tagliabue L, Moja L. A review of online evidence-based practice point-of-care information summary providers. J Med Internet Res 2010;12:e26. DOI:10.2196/jmir.1288.
Chan R, Stieda V. Evaluation of three point-of-care healthcare databases: BMJ Point-of-Care, Clin-eguide and Nursing Reference Centre. Health Info Libr J. 2011;28(1):50-58. DOI:10.1111/j.1471-1842.2010.00920.x.
Baxter SL, Lander L, Clay B, Bell B, Hansen K, Walker A, Tai-Seale M. Comparing the use of DynaMed and UpToDate by physician trainees in clinical decision-making: A randomized crossover trial. Appl Clin Inform 2022;13:139-147. DOI:10.1055/s-0041-1742216.
Charbonneau DH, James LN. DynaMed Plus®: An evidence-based clinical reference resource. Med Ref Serv Q 2018;37:168-176.
Lenaerts G, Bekkering GE, Goossens M, Coninck LD, Delvaux N, Cordyn S, Adriaenssens J, Vankrunkelsven P. Tools to assess the trustworthiness of evidence-based point-of-care information for health care professionals: Systematic review. J Med Internet Res 2020;22:e15415. DOI:10.2196/15415.
Rambaran KA, Huynh HA, Zhang Z, Robles J. The gap in electronic drug information resources: A systematic review. Cureus 2018;10:e2860. DOI:10.7759/cureus.2860.
Bradley-Ridout G, Nekolaichuk E, Jamieson T, Jones C, Morson N, Chuang R, Springall E. UpToDate versus DynaMed: A cross-sectional study comparing the speed and accuracy of two point-of-care information tools. J Med Libr Assoc 2021;109:382-387. DOI:10.5195/jmla.2021.1176.
Rui A, Garabedian PM, Marceau M, Syrowatka A, Volk LA, Edress HH, Seger DL, Amato MG, Cambre J, Dulgarian S, Newmark LP, Nanji KC, Schultz P, Jackson GP, Rozenblum R, Bates DW. Performance of a web-based reference database with natural language searching capabilities: Usability evaluation of DynaMed and Micromedex with Watson. JMIR Hum Factors 2023;10:e43960. DOI:10.2196/43960.
Nickum A, Raszewski R, Vonderheid SC. A comparative evaluation of three point-of-care tools by registered nurses. J Med Libr Assoc. 2022;110:323-331. DOI:10.5195/jmla.2022.1388.
IBM Watson health and EBSCO information services collaborate to launch integrated clinical decision support solution. EBSCO. [2023-07-20]. https://www.ebsco.com/news-center/press-releases/ibm-and-ebsco-launch-clinical-decision-support-solution.
DynaMed® and Micromedex® with Watson™ is now DynaMedex™ EBSCO. [2023-07-20]. https://www.ebsco.com/news-center/press-releases/dynamed-and-micromedex-watson-now-dynamedex.
Cooper D, Scanlon L. Introducing DynaMedex and Micromedex with Watson. National Institutes of Health. [2023-07-20]. https://www.nihlibrary.nih.gov/about-us/news/introducing-dynamedex-and-micromedex-watson.
Edrees, HH, Seger DL, Amato MG, Syrowatka A, Garabedian PM, Dulgarian S, Schultz P, Jackson GP, Bates D. Getting to the information clinicians need quickly: An evaluation of DynaMed and Micromedex with Watson. Abstract presented at: American Medical Informatics Association Annual Symposium. November 5-9, 2022: Washington, DC.
Wiener C, Fauci AS, Hauser SL, Longo DL, Jameson JL, Kasper DL, Loscalzo J. 20th ed. Harrison’s principles of internal medicine self-assessment and board review. McGraw Hill; 2021.
Henderson MC, Tierney LM, Smetana GW. 2nd ed. The patient history: An evidence-based approach to differential diagnosis. McGraw Hill; 2012.
Prepare for your exam. United States Medical Licensing Exam (USMLE). Accessed December 1, 2022-February 28, 2023. https://www.usmle.org/prepare-your-exam.
Clauson KA, Marsh WA, Polen HH, Seamon MJ, Ortiz BI. Clinical decision support tools: analysis of online drug information databases. BMC Med Inform Decis Mak 2007;7:7. DOI:10.1186/1472-6947-7-7.
Campbell JM, Umapathysivam K, Xue Y, Lockwood C. Evidence-Based Practice Point-of-Care Resources: A Quantitative Evaluation of Quality, Rigor, and Content. Worldviews Evid Based Nurs. 2015;12:313-327. DOI:10.1111/wvn.12114.
Ketchum AM, Saleh AA, Jeong K. Type of evidence behind point-of-care clinical information products: a bibliometric analysis. J Med Internet Res 2011;13:e21. DOI:10.2196/jmir.1539.
Johnson E, Emani VK, Ren J. Breadth of coverage, ease of use, and quality of mobile point-of-care tool information summaries: An evaluation. JMIR Mhealth Uhealth. 2016;4:e117. DOI:10.2196/mhealth.6189.
Kwag KH, González-Lorenzo M, Banzi R, Bonovas S, Moja L. Providing doctors with high-quality information: An updated evaluation of web-based point-of-care information summaries. J Med Internet Res 2016;18:e15. DOI:10.2196/jmir.5234.
Preininger AM, South B, Heiland J, Buchold A, Baca M, Wang S, Nipper R, Kutub N, Bohanan B, Jackson GP. Artificial intelligence-based conversational agent to support medication prescribing. JAMIA Open 2020;3:225-232. DOI:10.1093/jamiaopen/ooaa009.
Downloads
Additional Files
Published
Issue
Section
License
Copyright (c) 2023 Rachel L. Wasserman; Diane L. Seger; Mary G. Amato; Zoe Co; Aqsa Mugal; Angela Rui; Pamela M. Garabedian; Marlika Marceau; Ania Syrowatka; Lynn A. Volk; David W. Bates
This work is licensed under a Creative Commons Attribution 4.0 International License.
