How searching under time pressure impacts clinical decision making
DOI:
https://doi.org/10.5195/jmla.2020.915Keywords:
Clinical Decision Making, Information Retrieval, Medical SearchAbstract
Objective: Clinicians encounter many questions during patient encounters that they cannot answer. While search systems (e.g., PubMed) can help clinicians find answers, clinicians are typically busy and report that they often do not have sufficient time to use such systems. The objective of this study was to assess the impact of time pressure on clinical decisions made with the use of a medical literature search system.
Design: In stage 1, 109 final-year medical students and practicing clinicians were presented with 16 clinical questions that they had to answer using their own knowledge. In stage 2, the participants were provided with a search system, similar to PubMed, to help them to answer the same 16 questions, and time pressure was simulated by limiting the participant’s search time to 3, 6, or 9 minutes per question.
Results: Under low time pressure, the correct answer rate significantly improved by 32% when the participants used the search system, whereas under high time pressure, this improvement was only 6%. Also, under high time pressure, participants reported significantly lower confidence in the answers, higher perception of task difficulty, and higher stress levels.
Conclusions: For clinicians and health care organizations operating in increasingly time-pressured environments, literature search systems become less effective at supporting accurate clinical decisions. For medical search system developers, this study indicates that system designs that provide faster information retrieval and analysis, rather than traditional document search, may provide more effective alternatives.References
Ely JW, Osheroff JA, Gorman PN, Ebell MH, Chambliss ML, Pifer EA, Stavri PZ. A taxonomy of generic clinical questions: classification study. BMJ. 2000 Aug 12;321(7258):429–32.
Coumou HCH, Meijman FJ. How do primary care physicians seek answers to clinical questions? a literature review. J Med Libr Assoc. 2006 Jan;94(1):55–60.
Ely JW, Osheroff JA, Ebell MH, Bergus GR, Levy BT, Chambliss ML, Evans ER. Analysis of questions asked by family doctors regarding patient care. BMJ. 1999 Aug 7;319(7206):358–61.
Ely JW, Osheroff JA, Chambliss ML, Ebell MH, Rosenbaum ME. Answering physicians’ clinical questions: obstacles and potential solutions. J Am Med Inform Assoc. 2005 Mar–Apr;12(2):217–24.
Gorman PN, Helfand M. Information seeking in primary care: how physicians choose which clinical questions to pursue and which to leave unanswered. Med Decis Mak. 1995 Apr–Jun;15(2):113–9.
Verhoeven AAH, Schuling J. Effect of an evidence-based answering service on GPs and their patients: a pilot study. Health Inf Libr J. 2004 Sep;21(supp 2):27–35.
Schilling LM, Steiner JF, Lundahl K, Anderson RJ. Residents’ patient-specific clinical questions: opportunities for evidence-based learning. Acad Med. 2005 Jan;80(1):51–6.
Magrabi F, Coiera EW, Westbrook JI, Gosling AS, Vickland V. General practitioners’ use of online evidence during consultations. Int J Med Inform. 2005 Jan;74(1):1–12.
Marshall JG, Sollenberger J, Easterby-Gannett S, Morgan LK, Klem ML, Cavanaugh SK, Oliver KB, Romanosky N, Hunter S. The value of library and information services in patient care: results of a multisite study. J Med Libr Assoc. 2013 Jan;101(1):38–46. DOI: http://dx.doi.org/10.3163/1536-5050.101.1.007.
Maggio LA, Aakre CA, Del Fiol G, Shellum J, Cook DA. Impact of electronic knowledge resources on clinical and learning outcomes: systematic review and meta-analysis. J Med Internet Res. 2019 Jul;21(7):e13315.
Dunn K, Marshall JG, Wells AL, Backus JEB. Examining the role of MEDLINE as a patient care information resource: an analysis of data from the Value of Libraries study. J Med Libr Assoc. 2017 Oct;105(4):336–46. DOI: http://dx.doi.org/10.5195/jmla.2017.87.
Davies KS. Physicians and their use of information: a survey comparison between the United States, Canada, and the United Kingdom. J Med Libr Assoc. 2011 Jan;99(1):88–91. DOI: http://dx.doi.org/10.3163/1536-5050.99.1.015.
Westbrook JI, Coiera EW, Gosling AS. Do online information retrieval systems help experienced clinicians answer clinical questions? J Am Med Inform Assoc. 2005 May–Jun;12(3):315–21.
Hersh WR, Crabtree MK, Hickam DH, Sacherek L, Friedman CP, Tidmarsh P, Mosbaek C, Kraemer D. Factors associated with success in searching MEDLINE and applying evidence to answer clinical questions. J Am Med Inform Assoc. 2002 May–Jun;9(3):283–93.
Cook DA, Sorensen KJ, Hersh W, Berger RA, Wilkinson JM. Features of effective medical knowledge resources to support point of care learning: a focus group study. PLoS One. 2013;8(11):e80318.
Brassil E, Gunn B, Shenoy AM, Blanchard R. Unanswered clinical questions: a survey of specialists and primary care providers. J Med Libr Assoc. 2017 Jan;105(1):4–11. DOI: http://dx.doi.org/10.5195/jmla.2017.101.
Irving G, Neves AL, Dambha-Miller H, Oishi A, Tagashira H, Verho A, Holden J. International variations in primary care physician consultation time: a systematic review of 67 countries. BMJ Open. 2017;7(10):e017902.
Osborn R, Moulds D, Schneider EC, Doty MM, Squires D, Sarnak DO. Primary care physicians in ten countries report challenges caring for patients with complex health needs. Health Aff (Millwood). 2015 Dec;34(12):2104–12.
British Medical Association. Working in a system that is under pressure [Internet]. The Association; 2018 [cited 17 Jul 2020]. <https://www.bma.org.uk/collective-voice/influence/key-negotiations/nhs-pressures/working-in-a-system-under-pressure>.
van der Vegt A, Zuccon G, Koopman B, Deacon A. Impact of a search engine on clinical decisions under time and system effectiveness constraints: research protocol. JMIR Res Protoc. 2019 May 28;8(5):e12803.
Ordonez L, Benson III L. Decisions under time pressure: how time constraint affects risky decision making. Organ Behav Hum Decis Process. 1997 Aug;71(2):121–40.
Crescenzi A, Kelly D, Azzopardi L. Impacts of time constraints and system delays on user experience. In: Proceedings of the 2016 Association for Computing Machinery (ACM) Conference on Human Information Interaction and Retrieval; 2016. p. 141–50.
Westbrook JI, Gosling AS, Coiera EW. The impact of an online evidence system on confidence in decision making in a controlled setting. Med Decis Mak. 2005 Mar–Apr;25(2):178–85.
Hoogendam A, Stalenhoef AFH, de Vries Robbé PF, Overbeke AJPM. Answers to questions posed during daily patient care are more likely to be answered by UpToDate than PubMed. J Med Internet Res. 2008 Oct 3;10(4):e29.
Ramos K, Linscheld R, Schafer S. Real-time information-seeking behavior of residency physicians. Fam Med. 2003 Apr;35(4):257–60.
Simpson MS, Voorhees EM, Hersh W. Overview of the TREC 2015 clinical decision support track. Proc Twenty-Third Text Retr Conf TREC. 2015;2:1–8.
Simpson MS, Voorhees E, Hersh W. Overview of the TREC 2014 clinical decision support track. Proc Twenty-Third Text Retr Conf TREC; 2014. p. 1–7.
Sanderson M, Paramita ML, Clough P, Kanoulas E. Do user preferences and evaluation measures line up? In: Proceedings of the 33rd International Association for Computing Machinery (ACM) Special Interest Group on Information Retrieval (SIGIR) Conference on Research and Development in Information Retrieval; 2010. p. 555–62.
Agresti A. Models for matched pairs. In: Agresti A, ed. Symmetry models: categorical data analysis. New York, NY: John Wiley & Sons; 1990. p. 350–4.
Crescenzi A, Capra R, Arguello J. Time pressure, user satisfaction and task difficulty. Proc Assoc Inf Sci Technol. 2013;50(1):1–4.
Smith JF, Mitchell TR, Beach LR. A cost-benefit mechanism for selecting problem-solving strategies: some extensions and empirical tests. Organ Behav Hum Perform. 1982 Jun;29(3):370–96.
Ellsworth MA, Homan JM, Cimino JJ, Peters SG, Pickering BW, Herasevich V. Point-of-care knowledge-based resource needs of clinicians: a survey from a large academic medical center. Appl Clin Inform. 2015 May 6;6(2):305–17. DOI: http://dx.doi.org/10.4338/ACI-2014-11-RA-0104.
Koopman B, Russell J, Zuccon G. Task-oriented search for evidence-based medicine. Int J Digit Libr. 2017;1–13.
Bota H, Zhou K, Jose JM. Playing your cards right: the effect of entity cards on search behaviour and workload. In: Proceedings of the 2016 Association for Computing Machinery (ACM) on Conference on Human Information Interaction and Retrieval; 2016. p. 131–40.
Kelly D, Gyllstrom K. An examination of two delivery modes for interactive search system experiments: remote and laboratory. In: Proceedings of the Special Interest Group on Computer–Human Interaction (SIGCHI) Conference on Human Factors in Computing Systems; 2011. p. 1531–40.
