Effectiveness of a question formulation rubric with second-year medical students: a randomized controlled trial
DOI:
https://doi.org/10.5195/jmla.2023.1529Keywords:
Question formulation, Evidence Based Practice, Scoring Rubrics, Medical education, Educational measurementAbstract
Objective: The FAC (Focus, Amplify, Compose) rubric for assessing medical students’ question formulation skills normally accompanies our Evidence Based Practice (EBP) training. The combined training and assessment rubric have improved student scores significantly. How much does the rubric itself contribute to improved student scores? This study sought to measure student improvement using the rubric either with or without a linked 25-minute training session.
Methods: Randomized Controlled Trial. The authors tested the hypothesis that a 25-minute training session combined with use of a rubric would lead to higher scores than a brief explanation of this rubric alone. All 72 participating second-year medical students had a question formulation rubric briefly explained to them following a pre-test. Students in the intervention groups were taught how to formulate EBP questions for 25 minutes using the rubric followed with another 30 minutes of EBP search training. Students in the control group only received the 30 minutes of EBP search training in their small group labs. All 72 students took the post-test in which they formulated a question in response to a clinical vignette. Statistical analysis to test the hypothesis consisted of a two-sample paired t-test to measure between-group differences.
Discussion: The rubric itself unexpectedly contributed similarly to the Intervention groups students’ improvement, thereby saving scarce curricular time.
References
National Institutes of Health. National Library of Medicine. Medical Subject Headings (MeSH). [Internet]. 2022 January 4. 2022 [cited 2022 January 30]. https://www.ncbi.nlm.nih.gov/mesh/68055317 .
Horsley T, O'Neill J, McGowan J, Perrier L, Kane G, Campbell C. Interventions to improve question formulation in professional practice and self-directed learning. Cochrane Database Syst Rev. 2010 May 12;(5):CD007335. doi: 10.1002/14651858.CD007335.pub2.
Liaison Committee on Medical Education. Functions and structure of a medical school. Critical judgment/problem-solving skills. Standard 7.4. Chicago, IL: LCME, 2020: 10. [Internet]. 2020 May 14. [cited 2022 October 20]. https://lcme.org/publications/ .
Nicholson J, Spak JM, Kovar-Gough I, Lorbeer ER, Adams NE. Entrustable professional activity 7: opportunities to collaborate on evidence-based medicine teaching and assessment of medical students. BMC Med Educ. 2019 Sep 3;19(1):330. doi: 10.1186/s12909-019-1764-y.
Maggio LA, Tannery NH, Chen HC, ten Cate O, O'Brien B. Evidence-based medicine training in undergraduate medical education: a review and critique of the literature published 2006-2011. Acad Med. 2013 Jul;88(7):1022-8. doi: 10.1097/ACM.0b013e3182951959.
Accreditation Council for Graduate Medical Education. Program director’s guide to common program requirements (residency). Section IV.B. Chicago, IL: ACGME, [Internet].2020 January 31. [cited 2022 February 3]. https://www.acgme.org/Program-Directors-and-Coordinators/Welcome/Program-Directors-Guide-to-the-Common-Program-Requirements .
Accreditation Council for Graduate Medical Education. Milestones guidebook for residents and fellows. Chicago, IL, 2020. [Internet]. 2020. [cited 2022 March 10]. https://www.acgme.org/Portals/0/PDFs/Milestones/MilestonesGuidebookforResidentsFellows.pdf .
Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinical question: a key to evidence-based decisions. ACP J Club. 1995 Nov-Dec;123(3):A12-3. PMID: 7582737.
Huang X, Lin J, Demner-Fushman D. Evaluation of PICO as a knowledge representation for clinical questions. AMIA Annu Symp Proc. 2006:359-63. PMID: 17238363
Ely JW, Osheroff JA, Ebell MH, Bergus GR, Levy BT, Chambliss ML, Evans ER. Analysis of questions asked by family physicians regarding patient care. West J Med. 2000 May;172(5):315-9. doi: 10.1136/ewjm.172.5.315.
Bjerre LM, Paterson NR, McGowan J, Hogg W, Campbell CM, Viner G, Archibald D. What do primary care practitioners want to know? A content analysis of questions asked at the point of care. J Contin Educ Health Prof. 2013 Fall;33(4):224-34. doi: 10.1002/chp.21191
Schardt C, Adams MB, Owens T, Keitz S, Fontelo P. Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Med Inform Decis Mak. 2007 Jun 15;7:16. doi: 10.1186/1472-6947-7-16.
Hoogendam A, de Vries Robbé PF, Overbeke AJ. Comparing patient characteristics, type of intervention, control, and outcome (PICO) queries with unguided searching: a randomized controlled crossover trial. J Med Libr Assoc. 2012 Apr;100(2):121-6. doi: 10.3163/1536-5050.100.2.010.
Eriksen MB, Frandsen TF. The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: a systematic review. J Med Libr Assoc. 2018 Oct;106(4):420-431. doi: 10.5195/jmla.2018.345. Epub 2018 Oct 1.
Mintzker Y, Blum D, Adler L. Replacing PICO in non-interventional studies. BMJ Evid Based Med 2022 Jan. Epub ahead of print: doi: 10.1136/bmjEBP-2021-111889.
Booth A, O'Rourke AJ, Ford NJ. Structuring the pre-search reference interview: a useful technique for handling clinical questions. Bull Med Libr Assoc. 2000;88(3):239-246. PMID: 10928709
Davies KS. Formulating the Evidence Based Practice Question: A Review of the Frameworks. Evid Based Libr Inf Pract [Internet]. 2011 Jun 24 [cited 2022 January 30] ;6(2):75-0. https://journals.library.ualberta.ca/eblip/index.php/EBLIP/article/view/9741
Eldredge J, Schiff MA, Langsjoen JO, Jerabek RN. Question formulation skills training using a novel rubric with first-year medical students. J Med Libr Assoc. 2021;109(1):68-74. doi:10.5195/jmla.2021.935
Gallagher P, Wilson N, Jaine R. The efficient use of movies in a crowded curriculum. Clin Teach. 2014;11(2):88-93. doi:10.1111/tct.12178
Social Psychology Network. Research Randomizer. [Internet]. [cited 2020 October 20] https://www.randomizer.org/
Bell BB. Prettest-posttest designs. In: Encyclopedia of Research Design. Thousand Oaks, Calif.: Sage; 2010:
Larkin MJ. Rubrics. In: Salkind NJ, Rasmussen K, editors. Encyclopedia of educational psychology. Volume 2; Thousand Oaks, CA: Sage; 2008. p 859-61.
Eldredge JD, Nogar C. Physician assistant student training in question formulation: a quasi-experiment. J Physician Assist Educ. 2022 Mar; 33 (1): 47-50. doi: 10.1097/JPA.0000000000000409
Carroll AJ, Hallman SJ, Umstead KA, McCall J, DiMeo AJ. Using information literacy to teach medical entrepreneurship and health care economics. J Med Libr Assoc. 2019;107(2):163-171. doi: 10.5195/jmla.2019.577. Epub 2019 Apr 1.
Eldredge JD, Bear DG, Wayne SJ, Perea PP. Student peer assessment in evidence-based medicine (EBM) searching skills training: an experiment. J Med Libr Assoc. 2013;101(4):244-251. doi: 10.3163/1536-5050.101.4.003
Kloda LA, Boruff JT, Cavalcante AS. A comparison of patient, intervention, comparison, outcome (PICO) to a new, alternative clinical question framework for search skills, search results, and self-efficacy: a randomized controlled trial. J Med Libr Assoc. 2020;108(2):185-194. doi: 10.3163/1536-5050.101.4.003
Krupat E, Richards JB, Sullivan AM, Fleenor TJ Jr, Schwartzstein RM. Assessing the Effectiveness of Case-Based Collaborative Learning via Randomized Controlled Trial. Acad Med. 2016 May;91(5):723-9. doi: 10.1097/ACM.0000000000001004.
Kolb DA. Experiential learning: experience as the source of learning and development. Second edition. Upper Saddle River, NJ: Pearson, 2015: 50-64.
Anderson LW, Krathwohl DR. A Taxonomy for Learning, Teaching, and Assessing : A Revision of Bloom's Taxonomy of Educational Objectives. Complete ed. New York: Longman; 2001. 352 p.
Keim Janssen SA, VanderMeulen SP, Shostrom VK, Lomneth CS. Enhancement of anatomical learning and developing clinical competence of first-year medical and allied health profession students. Anat Sci Educ. 2014 May-Jun;7(3):181-90. doi: 10.1002/ase.1398. Epub 2013 Sep 3.
Versteeg M, van Blankenstein FM, Putter H, Steendijk P. Peer instruction improves comprehension and transfer of physiological concepts: a randomized comparison with self-explanation. Adv Health Sci Educ Theory Pract. 2019 Mar;24(1):151-165. doi: 10.1007/s10459-018-9858-6. Epub 2018 Oct 20.
Downloads
Published
Issue
Section
License
Copyright (c) 2023 Dr. Jonathan D. Eldredge, Dr. Melissa A. Schiff, Dr. Jens O. Langsjoen
This work is licensed under a Creative Commons Attribution 4.0 International License.
