Question formulation skills training using a novel rubric with first-year medical students
DOI:
https://doi.org/10.5195/jmla.2021.935Keywords:
Question Formulation, Evidence-Based Medicine, Evidence-Based Practice, Information Seeking Behavior, Medical Education, Undergraduate, Problem-Solving, Clinical Competence, Professional CompetenceAbstract
Objective: The research used an assessment rubric to measure medical students’ improvement in question formulation skills following a brief evidence-based medicine (EBM) training session conducted by a health sciences librarian.
Method: In a quasi-experimental designed study, students were assessed on their pre-instructional skills in formulating answerable EBM questions, based on a clinical scenario, using a rubric. Following their training, they were assessed using the same scenario and rubric. Student pre- and post-test scores were compared using a paired t-test.
Results: Students demonstrated statistically significant improvement in their question formulation skills on their post-instructional assessments. The average score for students on the pre-test was 45.5 (SD 11.1) and the average score on the post-test was 65.6 (SD 5.4) with an increase of 20.1 points on the 70-point scale, p<0.001.
Conclusion: The brief instructional session aided by the rubric improved students’ performance in question formulation skills.References
Oxman AD, Guyatt GH. Guidelines for reading literature reviews. CMAJ. 1988 Apr 15;138(8):697–703.
National Institutes of Health, National Library of Medicine. Medical subject headings (MeSH) [Internet]. The Institutes [cited 30 Apr 2020]. <https://www.ncbi.nlm.nih.gov/mesh?otool=unmlib>.
Guyatt G, Rennie D, Meade M, Cook D, eds; American Medical Association. Users’ guides to the medical literature: a manual for evidence-based clinical practice. 3rd ed. New York, NY: McGraw-Hill Education; 2015.
Liaison Committee on Medical Education. Functions and structure of a medical school: critical judgment/problem-solving skills: standard 7.4 [Internet]. Chicago, IL: The Committee; 2020 [cited 14 May 2020]. <https://lcme.org/publications/>.
Nicholson J, Spak JM, Kovar-Gough I, Lorbeer ER, Adams NE. Entrustable professional activity 7: opportunities to collaborate on evidence-based medicine teaching and assessment of medical students. BMC Med Educ. 2019 Sep 3;19(1):330. DOI: http://dx.doi.org/10.1186/s12909-019-1764-y.
Maggio LA, Tannery NH, Chen HC, ten Cate O, O’Brien B. Evidence-based medicine training in undergraduate medical education: a review and critique of the literature published 2006–2011. Acad Med. 2013 Jul;88(7):1022–8. DOI: http://dx.doi.org/10.1097/ACM.0b013e3182951959.
Maggio LA, ten Cate O, Chen HC, Irby DM, O’Brien BC. Challenges to learning evidence-based medicine and educational approaches to meet these challenges: a qualitative study of selected EBP curricula in U.S. and Canadian medical schools. Acad Med. 2016 Jan;91(1):101–6.
Taylor RS. Question-negotiation and information seeking in libraries. Coll Res Libr. 1968 May;29(3):178–94.
American Library Association. The reference interview: connecting in person and in cyberspace. presentations and responses from the RUSA president’s program, 2002 ALA Annual Conference, Atlanta, June 17, 2002. Ref User Serv Q. 2003;43(1):37–51.
Ross CS. The reference interview: why it needs to be used in every (well, almost every) reference transaction. Ref User Serv Q. 2003 Fall;43(1):38–43.
Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinical question: a key to evidence-based decisions. ACP J Club. 1995 Nov–Dec;123(3):A12–3.
Straus SE, Glasziou P, Richardson WS, Haynes RB. Evidence-based medicine: how to practice and teach EBM. 5th ed. Edinburgh, UK: Elsevier; 2019. p. 22.
Ramos KD, Schafer S, Tracz SM. Validation of the Fresno test of competence in evidence based medicine. BMJ. 2003 Feb 8;326(7384):319–21.
McCluskey A, Bishop B. The adapted Fresno test of competence in evidence-based practice. J Contin Educ Health Prof. 2009 Spring;29(2):119–26.
Thomas RE, Kreptul D. Systematic review of evidence-based medicine tests for family physician residents. Fam Med. 2015 Feb;47(2):101–17.
Shaneyfelt T, Baum KD, Bell D, Feldstein D, Houston TK, Kaatz S, Whelan C, Green M. Instruments for evaluating education in evidence-based practice: a systematic review. JAMA. 2006 Sep 6;296(9):1116–27.
University of California San Francisco, Family Practice Residency Program. Fresno test of evidence based medicine. grading rubric (form A). Fresno, CA: The University.
Fritsche L, Greenhalgh T, Falck-Ytter Y, Neumayer HH, Kunz R. Do short courses in evidence based medicine improve knowledge and skills? validation of Berlin questionnaire and before and after study of courses in evidence based medicine. BMJ. 2002 Dec 7;325(7376):1338–41.
Wyer PC, Naqvi Z, Dayan PS, Celentano JJ, Eskin B, Graham MJ. Do workshops in evidence-based practice equip participants to identify and answer questions requiring consideration of clinical research? a diagnostic skill assessment. Adv Health Sci Educ Theory Pract. 2009 Oct;14(4):515–33.
Cooke A, Smith D, Booth A. Beyond PICO: the SPIDER tool for qualitative evidence synthesis. Qual Health Res. 2012 Oct;22(10):1435–43. DOI: http://dx.doi.org/10.1177/1049732312452938. Epub: 24 Jul 2012.
Davies KS. Formulating the evidence based practice question: a review of the frameworks. Evid Based Libr Inf Pract. 2011 Jun 24;6(2):75–80. DOI: http://dx.doi.org/10.18438/B8WS5N.
Huang X, Lin J, Demner-Fushman D. Evaluation of PICO as a knowledge representation for clinical questions. AMIA Annu Symp Proc. 2006:359–63.
Ely JW, Osheroff JA, Ebell MH, Bergus GR, Levy BT, Chambliss ML, Evans ER. Analysis of questions asked by family physicians regarding patient care. West J Med. 2000 May;172(5):315–9.
Bjerre LM, Paterson NR, McGowan J, Hogg W, Campbell CM, Viner G, Archibald D. What do primary care practitioners want to know? a content analysis of questions asked at the point of care. J Contin Educ Health Prof. 2013 Fall;33(4):224–34.
Schardt C, Adams MB, Owens T, Keitz S, Fontelo P. Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Med Inform Decis Mak. 2007 Jun 15;7:16.
Hoogendam A, de Vries Robbé PF, Overbeke AJPM. Comparing patient characteristics, type of intervention, control, and outcome (PICO) queries with unguided searching: a randomized controlled crossover trial. J Med Libr Assoc. 2012 Apr;100(2):121–6. DOI: http://dx.doi.org/10.3163/1536-5050.100.2.010.
Eriksen MB, Frandsen TF. The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: a systematic review. J Med Libr Assoc. 2018 Oct;106(4):420–31. DOI: http://dx.doi.org/10.5195/jmla.2018.345.
Horsley T, O’Neill J, McGowan J, Perrier L, Kane G, Campbell C. Interventions to improve question formulation in professional practice and self-directed learning. Cochrane Database Syst Rev. 2010 May 12;(5):CD007335. DOI: http://dx.doi.org/10.1002/14651858.CD007335.pub2.
Bradley DR, Rana GK, Martin PW, Schumacher RE. Real-time, evidence-based medicine instruction: a randomized controlled trial in a neonatal intensive care unit. J Med Libr Assoc. 2002 Apr;90(2):194–201.
Villanueva EV, Burrows EA, Fennessy PA, Rajendran M, Anderson JN. Improving question formulation for use in evidence appraisal in a tertiary care setting: a randomised controlled trial [ISRCTN66375463]. BMC Med Inform Decis Mak. 2001;1:4. Epub: 8 Nov 2001.
Schaafsma F, Hulshof C, de Boer A, van Dijk F. Effectiveness and efficiency of a literature search strategy to answer questions on the etiology of occupational diseases: a controlled trial. Int Arch Occup Environ Health. 2007 Jan;80(3):239–47. Epub: 21 Jun 2006.
Cheng GY. Educational workshop improved information-seeking skills, knowledge, attitudes and the search outcome of hospital clinicians: a randomised controlled trial. Health Info Libr J. 2003 Jun;20(suppl 1):22–33.
Shadish WR, Cook TD, Campbell DT. Experimental and quasi-experimental designs for generalized causal inference. Belmont, CA: Wadsworth; 2011. p. 103–15.
Shek TS, Wu J. Quasi-experimental designs. In: Frey B, ed. The SAGE encyclopedia of educational research, measurement, and evaluation. Vol. 4. Thousand Oaks, CA: SAGE Publications; 2018. DOI: http://dx.doi.org/10.4135/9781506326139.
Del Fiol G, Workman TE, Gorman PN. Clinical questions raised by clinicians at the point of care: a systematic review. JAMA Intern Med. 2014 May;174(5):710–8. DOI: http://dx.doi.org/10.1001/jamainternmed.2014.368.
Reddy YM, Andrade H. A review of rubric use in higher education. Assess Eval High Educ. 2010 Jul;35(4):435–48.
Becker HS. Tricks of the trade. Chicago, IL: University of Chicago Press; 1998. p. 194–212.
Browne MN, Keeley SM. Asking the right questions: a guide to critical thinking. Upper Saddle River, NJ: Prentice-Hall Press; 2004.
Babbie E. Practice of social research. Belmont, CA: Wadsworth Cengage Learning; 2013. p. 54–6.
Education section - what does the question mean? J Evid Based Med. 2014 Feb;7(1):65–6.
Education section - what does the question mean, continued. J Evid Based Med. 2014 May;7(2):151.
Willett WC. Nutritional epidemiology. In: Rothman KJ, Greenland S, Lash TL. Modern epidemiology. 3rd ed. Philadelphia, PA: Lippincott Williams & Wilkins; 2008: 582.
Kerlinger FN. Foundations of behavioral research. 2nd ed. New York, NY: Holt, Rinehart and Winston; 1973. p. 262–3.
Campbell DT, Stanley JC. Experimental and quasi-experimental designs for research. Chicago, IL: Rand McNally College Publishing Company; 1963. p. 5–6.
Creswell JW. Research design: qualitative, quantitative, and mixed methods approaches. 3rd ed. Thousand Oaks, CA: Sage Publications; 2009. p. 162–6.
Wang J, Wang D, Chen Y, Zhou Q, Xie H, Chen J, Li Y. The effect of an evidence-based medicine course on medical student critical thinking. J Evid Based Med. 2017 Nov;10(4):287–92. DOI: http://dx.doi.org/10.1111/jebm.12254. Epub 2017 May 24.
