Assessment of knowledge and skills in information literacy instruction for rehabilitation sciences students: a scoping review
DOI:
https://doi.org/10.5195/jmla.2018.227Keywords:
Information Literacy, Evidence-Based Practice, Assessment, Allied Health, Occupational Therapy, Physical Therapy, Communication Sciences and Disorders, Speech-Language PathologyAbstract
Objective: This scoping review investigates how knowledge and skills are assessed in the information literacy (IL) instruction for students in physical therapy, occupational therapy, or speech-language pathology, regardless of whether the instruction was given by a librarian. The objectives were to discover what assessment measures were used, determine whether these assessment methods were tested for reliability and validity, and provide librarians with guidance on assessment methods to use in their instruction in evidence-based practice contexts.
Methods: A scoping review methodology was used. A systematic search strategy was run in Ovid MEDLINE and adapted for CINAHL; EMBASE; Education Resources Information Center (ERIC) (EBSCO); Library and Information Science Abstracts (LISA); Library, Information Science & Technology Abstracts (LISTA); and Proquest Theses and Dissertations from 1990 to January 16, 2017. Forty articles were included for data extraction.
Results: Three major themes emerged: types of measures used, type and context of librarian involvement, and skills and outcomes described. Thirty-four measures of attitude and thirty-seven measures of performance were identified. Course products were the most commonly used type of performance measure. Librarians were involved in almost half the studies, most frequently as instructor, but also as author or assessor. Information literacy skills such as question formulation and database searching were described in studies that did not involve a librarian.
Conclusion: Librarians involved in instructional assessment can use rubrics such as the Valid Assessment of Learning in Undergraduate Education (VALUE) when grading assignments to improve the measurement of knowledge and skills in course-integrated IL instruction. The adapted Fresno test could be modified to better suit the real-life application of IL knowledge and skills.References
Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ. 1996 Jan 13;312(7023):71–2.
Canadian Association of Speech-Language Pathologists and Audiologists. Assessing and certifying clinical competency: foundations of clinical practice for audiology and speech-language pathology 2004 [Internet]. The Association [cited 9 Oct 2017]. <http://www.sac-oac.ca/professional-resources/resource-library/assessing-and-certifying-clinical-competency-foundations>.
National Physical Therapy Advisory Group. Essential competency profile for physiotherapists in Canada (2009) [Internet]. The Association [cited 9 Oct 2017]. <http://npag.ca/English/joint.html>.
Canadian Association of Occupational Therapists. Profile of practice of occupational therapists in Canada (2012) [Internet]. The Association [cited 9 Oct 2017]. <http://npag.ca/PDFs/Joint%20Initiatives/PTA%20profile%202012%20English.pdf>.
Franzen S, Bannon CM. Merging information literacy and evidence-based practice in an undergraduate health sciences curriculum map. Commun Inf Lit. 2016;10(2):245–63.
Adams NE. A comparison of evidence-based practice and the ACRL information literacy standards: implications for information literacy practice. Coll Res Libr. 2014 Mar;75(2):232–48.
Boruff JT, Thomas A. Integrating evidence-based practice and information literacy skills in teaching physical and occupational therapy students. Health Inf Libr J. 2011 Dec;28(4):264–72.
Straus SE, Gasziou P, Richardson WS, Haynes RB. Evidence-based medicine: how to practice and teach it. Edinburgh, Scotland, UK: Elsevier Churchill Livingstone; 2011.
Rosenberg W, Donald A. Evidence based medicine: an approach to clinical problem-solving. BMJ. 1995 Apr 29;310(6987):1122–6.
Thomas A, Saroyan A, Snider LM. Evidence-based practice behaviours: a comparison amongst occupational therapy students and clinicians. Can J Occup Ther. 2012 Apr;79(2):96–107.
Coomarasamy A, Khan KS. What is the evidence that postgraduate teaching in evidence based medicine changes anything? a systematic review. BMJ. 2004 Oct 30;329(7473):1017.
Shaneyfelt T, Baum KD, Bell D, Feldstein D, Houston TK, Kaatz S, Whelan C, Green M. Instruments for evaluating education in evidence-based practice - a systematic review. JAMA. 2006 Sep 6;296(9):1116–27.
Glegg SMN, Holsti L. Measures of knowledge and skills for evidence-based practice: a systematic review. Can J Occup Ther. 2010 Oct;77(4):219–32.
Schilling K, Applegate R. Best methods for evaluating educational impact: a comparison of the efficacy of commonly used measures of library instruction. J Med Libr Assoc. 2012 Oct;100(4):258–69. DOI: http://dx.doi.org/10.3163/1536-5050.100.4.007.
Thomas A, Saroyan A, Dauphinee WD. Evidence-based practice: a review of theoretical assumptions and effectiveness of teaching and assessment interventions in health professions. Adv Health Sci Educ. 2011 May;16(2):253–76.
Gilchrist D, Oakleaf M. An essential partner: the librarian’s role in student learning assessment 2012 [Internet]. National Institute for Learning Outcomes Assessment [cited 9 Oct 2017]. <http://www.learningoutcomeassessment.org/occasionalpaperfourteen.htm>.
Oakleaf M. Dangers and opportunities: a conceptual map of information literacy assessment approaches. Portal Libr Acad. 2008 Jul;8(3):233–53.
Warner DA. Programmatic assessment: turning process into practice by teaching for learning. J Acad Librariansh. 2003 May;29(3):169–76.
Cull BW. Voices in the wilderness: a report on academic information literacy instruction in Atlantic Canada. Can J Inf Libr Sci. 2005 Mar;29(1):1–26.
Crawford J. The use of electronic information services and information literacy. J Librariansh Inf Sci. 2006;38(1):33–44.
Schilling K, Applegate R. Evaluating library instruction: measures for assessing educational quality and impact. Proceedings of the Thirteenth National Conference of the Association of College and Research Libraries. Chicago, IL: Association of College and Research Libraries; 2007. p. 206–14.
Molteni VE, Chan EK. Student confidence/overconfidence in the research process. J Acad Librariansh. 2015 Jan;41(1):2–8.
Swartz B, Ratcliff A, Ivanitskaya L. Correlation of attitudes and beliefs with actual abilities of speech-language pathology students regarding aspects of information literacy. Contemp Issues Commun Sci Disord. 2015 Fall;42:215–25.
Stevenson P. Evaluating educational interventions for information literacy. Health Inf Libr J. 2012 Mar;29(1):81–6.
Brettle A. Evaluating information skills training in health libraries: a systematic review. Health Inf Libr J. 2007 Dec;24(supp 1):18–37.
Walsh A. Information literacy assessment. where do we start? J Librariansh Inf Sci. 2009;41(1):19–28.
Maggio LA, Kung JY. How are medical students trained to locate biomedical information to practice evidence-based medicine? a review of the 2007–2012 literature. J Med Libr Assoc. 2014 Jul;102(3):184–91. DOI: http://dx.doi.org/10.3163/1536-5050.102.3.008.
Just ML. Is literature search training for medical students and residents effective? a literature review. J Med Libr Assoc. 2012 Oct;100(4):270–6. DOI: http://dx.doi.org/10.3163/1536-5050.100.4.008.
Swanberg SM, Dennison CC, Farrell A, Machel V, Marton C, O’Brien KK, Pannabecker V, Thuna M, Holyoke AN. Instructional methods used by health sciences librarians to teach evidence-based practice (EBP): a systematic review. J Med Libr Assoc. 2016 Jul;104(3):197–208. DOI: http://dx.doi.org/10.3163/1536-5050.104.3.004.
Dizon JMR, Grimmer-Somers KA, Kumar S. Current evidence on evidence-based practice training in allied health: a systematic review of the literature. Int J Evid Based Healthc. 2012 Dec;10(4):347–60.
Buchanan H, Siegfried N, Jelsma J. Survey instruments for knowledge, skills, attitudes and behaviour related to evidence-based practice in occupational therapy: a systematic review. Occup Ther Int. 2016;23(2):59–90.
Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.
McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016 Jul;75:40–6.
Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009 Jul 21;339:b2535.
Alverson KL. Evidence-based practice in early training of speech-language pathologists [master’s of arts thesis]. Ann Arbor, MI: Wayne State University; 2009.
Zipoli JRP, Kennedy M. Evidence-based practice among speech-language pathologists attitudes, utilization, and barriers. Am J Speech Lang Pathol. 2005 Aug;14(3):208–20.
Bennett S, Hoffmann T, Arkins M. A multi-professional evidence-based practice course improved allied health students’ confidence and knowledge. J Eval Clin Pract. 2011 Aug;17(4):635–9.
Bennett S, Tooth L, McKenna K, Rodger S, Strong J, Ziviani J, Mickan S, Gibson L. Perceptions of evidence-based practice: a survey of Australian occupational therapists. Aust Occup Ther J. 2003 Mar;50(1):13–22.
Young JM, Glasziou P, Ward JE. General practitioners’ self ratings of skills in evidence based medicine: validation study. BMJ. 2002 Apr 20;324(7343):950–1.
Bortone JM. Critical thinking and evidence-based practice in problem-based learning tutorial groups: a critical case study [doctor of education dissertation]. Ann Arbor, MI: Fordham University; 2007.
Straus SE. Evidence-based medicine: how to practice and teach EBM. Edinburgh, Scotland, UK; New York, NY: Elsevier/Churchill Livingstone; 2005.
Facione PA. The California critical thinking skills test—college level. Technical report #1. Experimental validation and content validity. 1990. Document no.: TM 015 818. [Accessed from ERIC.]
Bozzolan M, Simoni G, Balboni M, Fiorini F, Bombardi S, Bertin N, Da Roit M. Undergraduate physiotherapy students’ competencies, attitudes and perceptions after integrated educational pathways in evidence-based practice: a mixed methods study. Physiother Theory Pract. 2014 Nov;30(8):557–71.
Bozzolan M, Pavanello A, Barbieri R, Spada M, Del Giovane C, Gaiani R. The adapted Fresno test to assess knowledge and skills in evidence based practice: the Italian version/Il test di Fresno adattato per valutare conoscenze ed abilita’ in evidence based practice: la versione italiana. Scienza Riabilitativa. 2011;13:12+.
Brooks SV, Bigelow S. Preparing students for research: faculty/librarian collaboration in a pre-doctoral physical therapy research course. Health Inf Libr J. 2015 Dec;32(4):332–8.
Burnham JF. Information management education for students in the health care professions: a coordinated, integrated plan. Med Ref Serv Q. 1994;13(2):45–62.
Cobus-Kuo L, Waller J. Teaching information literacy and evidence-based practice in an undergraduate speech-language pathology program: a student reflection. Contemp Issues Commun Sci Disord. 2016 Spring;43:35–49.
Cohn E, Coster W, Kramer J. Facilitated learning model to teach habits of evidence-based reasoning across an integrated master of science in occupational therapy curriculum. Am J Occup Ther. 2014 Sep–Oct;68(suppl 2):S73–82.
Crabtree J, Justiss M, Swinehart S. Occupational therapy master-level students’ evidence-based practice knowledge and skills before and after fieldwork. Occup Ther Health Care. 2012 Apr–Jul;26(2–3):138–49.
McCluskey A, Lovarini M. Providing education on evidence-based practice improved knowledge but did not change behaviour: a before and after study. BMC Med Educ. 2005 Dec 19;5:40.
McCluskey A, Bishop B. The adapted Fresno test of competence in evidence-based practice. J Contin Educ Health Prof. 2009 Spring;29(2):119–26.
Crist P. Adapting research instruction to support the scholarship of practice: practice-scholar partnerships. Occup Ther Health Care. 2010 Jan;24(1):39–55.
Bieschke KJ, Bishop RM, Garcia VL. The utility of the research self-efficacy scale. J Career Assess. 1996;4(1):59–75.
Cusack T, O’Donoghue G. The introduction of an interprofessional education module: students’ perceptions. Qual Prim Care. 2012;20(3):231–8.
Durando P, Oakley P. Developing information literacy skills in nursing and rehabilitation therapy students. J Can Health Libr Assoc. 2005;26(1):7–11.
Evenson ME. Preparing for fieldwork: students’ perceptions of their readiness to provide evidence-based practice. Work. 2013;44(3):297–306.
Grant MJ, Brettle AJ. Developing and evaluating an interactive information skills tutorial. Health Inf Libr J. 2006 Jun;23(2):79–88.
Rosenberg WM, Deeks J, Lusher A, Snowball R, Dooley G, Sackett D. Improving searching skills and evidence retrieval. J R Coll Physicians Lond. 1998 Nov–Dec;32(6):557–63.
Griffin N, Schumm RW. Instructing occupational therapy students in information retrieval. Am J Occup Ther. 1992 Feb;46(2):158–61.
Hoberecht T, Randall K, Schweikhard AJ. Library tutorials in an allied health evidence-based practice class. Med Ref Serv Q. 2015;34(2):240–8.
Lechner DL. Graduate student research instruction: testing an interactive web-based library tutorial for a health sciences database. Res Strateg. 2007;20(4):469–81.
Lederer JM. Evidence for occupational therapy interventions: a student educational assignment. Occup Ther Health Care. 2004;18(4):29–40.
Lewis LK, Wong SC, Wiles LK, McEvoy MP. Diminishing effect sizes with repeated exposure to evidence-based practice training in entry-level health professional students: a longitudinal study. Physiother Can. 2016;68(1):73–80.
McEvoy MP, Williams MT, Olds TS. Development and psychometric testing of a trans-professional evidence-based practice profile questionnaire. Med Teach. 2010;32(9):e373–80.
Lewis L, Williams MT, Olds TS. Development and psychometric testing of an instrument to evaluate cognitive skills of evidence based practice in student health professionals. BMC Med Educ. 2011 Oct 3;11:77.
Maritz CA, Thielman G, Campolo M. Using a capstone project to prepare students to become evidence-based practitioners. J Fac Dev. 2011 May;25(2):12–7.
McEvoy M, Williams MT, Olds TS. Evidence based practice profiles: differences among allied health professions. BMC Med Educ. 2010 Oct 12;10:69.
McEvoy M, Williams MT, Olds TS, Lewis LK, Petkov J. Evidence-based practice profiles of physiotherapists transitioning into the workforce: a study of two cohorts. BMC Med Educ. 2011 Nov 29;11:100.
Olsen N, Lygren H, Espehaug B, Nortvedt MW, Bradley P, Bjordal JM. Evidence-based practice exposure and physiotherapy students’ behaviour during clinical placements: a survey. Physiother Res Int. 2014 Dec;19(4):238–47.
Portney LG. Evidence-based practice and clinical decision making: it’s not just the research course anymore. J Phys Ther Educ. 2004 Winter;18(3):46–51.
Raghavendra P. Teaching evidence-based practice in a problem-based learning course in speech-language pathology. Evid Based Commun Assess Interv. 2009;3(4):232–7.
Ratcliff A, Swartz B, Ivanitskaya L. Information literacy skills in speech-language pathology students: skill differences across academic levels. Contemp Issues Commun Sci Disord. 2013 Spring;40:31–9.
Ivanitskaya L, Laus R, Casey AM. Research readiness self-assessment. J Libr Adm. 2004;41(1–2):167–83.
Reynolds S. Teaching evidence-based practice in a distance education occupational therapy doctoral program: strategies for professional growth and advancing the profession. Occup Ther Health Care. 2010 Jan;24(1):56–67.
Sabus C. The effects of modeling evidence-based practice during the clinical internship. J Phys Ther Educ. 2008;22(3):74–84.
Scott P, Altenburger PA, Kean J. A collaborative teaching strategy for enhancing learning of evidence-based clinical decision-making. J Allied Health. 2011 Fall;40(3):120–7.
Spek B, Wieringa-de Waard M, Lucas C, van Dijk N. Teaching evidence-based practice (EBP) to speech-language therapy students: are students competent and confident EBP users? Int J Lang Commun Disord. 2013 Jul–Aug;48(4):444–52.
Spek B, Wieringa-de Waard M, Lucas C, Dijk N. Competent in evidence-based practice (EBP): validation of a measurement tool that measures EBP self-efficacy and task value in speech-language therapy students. Int J Lang Commun Disord. 2013 Jul–Aug;48(4):453–7.
Spek B, de Wolf GS, van Dijk N, Lucas C. Development and validation of an assessment instrument for teaching evidence-based practice to students in allied health care: the Dutch modified Fresno. J Allied Health. 2012 Summer;41(2):77–82.
Stronge M, Cahill M. Self-reported knowledge, attitudes and behaviour towards evidence-based practice of occupational therapy students in Ireland. Occup Ther Int. 2012 Mar;19(1):7–16.
Johnston JM, Leung GM, Fielding R, Tin KY, Ho LM. The development and validation of a knowledge, attitude and behaviour questionnaire to assess undergraduate evidence-based practice teaching and learning. Med Educ. 2003 Nov;37(11):992–1000.
Turbow DJ, Evener J. Norming a VALUE rubric to assess graduate information literacy skills. J Med Libr Assoc. 2016 Jul;104(3):209–14. DOI: http://dx.doi.org/10.3163/1536-5050.104.3.005.
Finley AP. How reliable are the VALUE rubrics? Peer Rev. 2011 Fall/Winter;13/14(4/1):31–3.
Rhodes TL, Finley AP. Using the VALUE rubrics for improvement of learning and authentic assessment. Washington, DC: Association of American Colleges and Universities; 2013. 44 p.
Van Moorsel G. Library-sponsored instruction improves core informatics competencies among allied health students: a research-based case study. J Allied Health. 2005 Fall;34(3):145–52.
Villeneuve M, Maranda S. Preparing entry-level practitioners for evidence-based practice. J Can Health Libr Assoc. 2005;26(1):13–21.
Vogel K. Librarians and occupational therapy faculty: a collaboration for teaching evidence-based practice. J Allied Health. 2012 Spring;41(1):e15–20.
Wolter JA, Corbin-Lewis K, Self T, Elsweiler A. An evidence-based practice model across the academic and clinical settings. Commun Disord Q. 2011;32(2):118–32.
Ramos KD, Schafer S, Tracz SM. Validation of the Fresno test of competence in evidence based medicine. BMJ (Clinical research ed). 2003 Feb 8;326(7384):319–21.
Tilson JK. Validation of the modified Fresno test: assessing physical therapists’ evidence based practice knowledge and skills. BMC Med Educ. 2010 May 25;10:38.
Oakleaf M. The information literacy instruction assessment cycle: a guide for increasing student learning and improving librarian instructional skills. J Doc. 2009;65(4):539–60.
Oakleaf M. Using rubrics to assess information literacy: an examination of methodology and interrater reliability. J Assoc Inf Sci Technol. 2009 May;60(5):969–83.
Thomas A, Saroyan A, Lajoie SP. Creation of an evidence-based practice reference model in falls prevention: findings from occupational therapy. Disabil Rehabil. 2012;34(4):311–28.
