Paper

Assessment of knowledge and skills in information literacy instruction for rehabilitation sciences students: a scoping review

Jill T. Boruff, AHIP, Pamela Harrison

doi: http://dx.doi.org/10.5195/jmla.2018.227

Received March 2017: Accepted September 2017

ABSTRACT

Objective

This scoping review investigates how knowledge and skills are assessed in the information literacy (IL) instruction for students in physical therapy, occupational therapy, or speech-language pathology, regardless of whether the instruction was given by a librarian. The objectives were to discover what assessment measures were used, determine whether these assessment methods were tested for reliability and validity, and provide librarians with guidance on assessment methods to use in their instruction in evidence-based practice contexts.

Methods

A scoping review methodology was used. A systematic search strategy was run in Ovid MEDLINE and adapted for CINAHL; EMBASE; Education Resources Information Center (ERIC) (EBSCO); Library and Information Science Abstracts (LISA); Library, Information Science & Technology Abstracts (LISTA); and Proquest Theses and Dissertations from 1990 to January 16, 2017. Forty articles were included for data extraction.

Results

Three major themes emerged: types of measures used, type and context of librarian involvement, and skills and outcomes described. Thirty-four measures of attitude and thirty-seven measures of performance were identified. Course products were the most commonly used type of performance measure. Librarians were involved in almost half the studies, most frequently as instructor, but also as author or assessor. Information literacy skills such as question formulation and database searching were described in studies that did not involve a librarian.

Conclusion

Librarians involved in instructional assessment can use rubrics such as the Valid Assessment of Learning in Undergraduate Education (VALUE) when grading assignments to improve the measurement of knowledge and skills in course-integrated IL instruction. The Adapted Fresno Test could be modified to better suit the real-life application of IL knowledge and skills.

INTRODUCTION

Since the term evidence-based medicine was first coined in the mid-1990s [1], the integration of evidence-based practice (EBP) in health professional education has continued to grow. While EBP has its roots in medicine, it is now established in the required competencies of allied health professions such as physical therapy, occupational therapy, and speech-language pathology [2–4]. Along with this growth has come the establishment of information literacy (IL) instruction to support EBP [5–7].

The EBP cycle has been well defined in the literature: instructors in rehabilitation sciences tend to use a five-step model: (i) formulating the clinical question, (ii) searching the evidence, (iii) appraising the evidence, (iv) incorporating evidence into decision making, and (v) evaluating the process [1, 8–10]. Teaching and assessment in EBP education is categorized into knowledge, skills, attitudes, and behaviors [11–13] and can be applied to IL instruction. Knowledge in the context of EBP is “an awareness of the sources of evidence and an understanding of the evidence itself” [13]. Skills include the application of this knowledge [12]. An IL example would be the knowledge of how a database functions and the skill of how to conduct searches in this database. Attitudes refer to the recognition of the need to use certain knowledge or skills while working on an academic project or in clinical practice [11]. This differs slightly from attitude in the context of measures of attitude, which are self-reported perceived learning [14]. An EBP behavior is when someone “applies the knowledge and skills to solve [an] issue in practice” [11]. These categories help instructors align their teaching and assessment methods with learning outcomes.

IL knowledge and skills are necessary at every step of the EBP cycle. The Association of College and Research Libraries Information Literacy Competency Standards of “Determine, Access, Evaluate, Apply, and Ethics” correlate with the five EBP steps of “Formulate, Search, Appraise, Incorporate, Evaluate” [5] and provide a structure for teaching the IL knowledge and skills needed for EBP [6, 7]. Librarians are experts in such knowledge and skills as question formulation (patient-intervention-comparison-outcome [PICO] format and other research question formats), database searching, resource selection, article selection, and article appraisal (critical appraisal), and can be the model for future clinicians who will use such knowledge and skills in practice [15].

As recommended by good IL practice, librarians teaching in EBP contexts “routinely state specific instructional goals, explain rationales for teaching methods, identify ways they expect students to be impacted by IL instruction, and detail the effect of library instruction…on student success” [16]. This last point, examining the effect of instruction on student success, can be the most difficult to implement, though good assessment is important to help instructors understand if students are learning what the instructor intends them to learn [17].

Studies that looked at how librarians included assessment in their practice revealed that the assessment methods were primarily indirect assessments or assessment of attitude, which was defined as “self-reporting of perceived skills, learning, behaviors, or attitudes” [14]. Warner found in her review of the library assessment literature that there were “serious limitations to the instruments most typically used, primarily that they have failed to adequately assess student learning” [18]. In interviewing eighteen librarians, Cull observed that “most librarians…did not objectively assess student learning at all but only solicited student reactions to instructional content, methods, and instructor effectiveness, typically using brief end-of-class student evaluation forms” [19]. Two other studies also noted that self-reported attitude assessments were the most frequently cited form of evaluation in library instruction [14, 20].

While assessments of attitude are important for understanding how students feel about their own learning and about the instructor’s teaching, students are not good evaluators of their own skill levels [21–23]. It is important, therefore, to assess their learning with measures of performance that document knowledge, skills, or behaviors based on actual student work [14]. Another aspect to consider is whether an assessment measure has been tested for reliability and validity. Validated assessment measures increase the accuracy and the reliability of results, facilitate comparison across different studies [24], and allow a richer understanding of the effectiveness of the instruction.

The importance of well-developed assessment measures is clear, but applying this knowledge can be a challenge, since librarians and other EBP instructors may not know which assessment measures would best align with their learning outcomes and instructional methods. In an effort to assist instructors with this choice, literature reviews in library and information science and health sciences education have summarized the assessment measures being used in IL and EBP learning contexts, but not all educational contexts and measure types have been well addressed. These reviews leave major gaps in the literature.

The authors found two gaps in the library and information science literature. First, all of the reviews conducted by librarians looked only at studies where librarians were involved in the design or delivery of instruction and assessment [14, 25–29]. Given the alignment of IL instruction objectives and EBP instruction objectives [5–7], it is valuable to consider studies where instruction and assessment of IL outcomes took place without explicit librarian involvement, as these studies might provide librarians with examples of assessment methods used in health sciences education and novel opportunities for assessment.

Second, none of the reviews that librarians conducted covered the assessment of rehabilitation sciences students. The earliest review was conducted in 2006, before the Adapted Fresno Test for occupational therapy was created and does not have much content from rehabilitation sciences [25]. Two more recent reviews provided excellent overviews of the assessments that librarians were conducting but were broader in subject coverage, were narrower in date coverage, and did not provide specific recommendations for assessment measures in EBP instruction [14, 26]. Three other recent reviews specifically focused on instruction in health sciences but still left a gap for assessment in rehabilitation sciences. One included teaching methods but not assessment measures [27]; the other two recommended the Fresno Test, but no or few studies outside of medicine were included [28, 29].

We found one major gap in the health sciences, in which none of the reviews focused on the performance assessment of students in physical therapy, occupational therapy, or speech-language pathology. The reviews in medicine focused on studies with medical students and clinicians [11, 12], and the reviews in rehabilitation sciences focused on practicing occupational and physical therapists [13, 30] or only found measures of attitude [31]. Three reviews found the Fresno Test [12] or the Adapted Fresno Test [13, 30] as the only validated assessments of performance, but they were used with medical students or rehabilitation sciences clinicians. Assessments of rehabilitation sciences clinicians or of medical students and clinicians are not sufficient. Rehabilitation sciences professionals approach practice differently than medical professionals, and “discipline-specific measures that address a profession’s own EBP needs and concerns may be warranted” [13, 30]. Students will not have the same level of EBP skills and knowledge as an expert clinician, and assessments of students should reflect this difference [15].

To address these gaps in the literature, this scoping review investigates how knowledge and skills are assessed in the IL instruction of physical therapy, occupational therapy, and speech-language pathology students, regardless of whether the instruction was designed and delivered by a librarian.

This project had three main objectives: (1) to determine what assessment methods were being used, (2) to determine whether these assessment methods had been tested for reliability and validity, and (3) to provide librarians with guidance on assessment methods that could be used in their own instruction. We chose to report on both measures of attitude as well as measures of performance, though our focus was on the latter.

METHODS

We chose a scoping review rather than a systematic review methodology for this review, because we addressed a broad question, “aiming to summarize and disseminate research findings” as proposed by Arksey and O’Malley in their definition and outlined in the stages below [32].

Stage 1: Identifying the research question

The original research question stated: How are knowledge and skills being assessed in the IL instruction of nursing and allied health students? After the first round of full-text screening, we narrowed the scope to include only physical therapy, occupational therapy, and speech-language pathology students, and we removed nursing from the research question due to reasons explained in stage 3 below.

Stage 2: Identifying relevant studies

A systematic search strategy was constructed by one author and reviewed by the second author as recommended by the PRESS standard [33]. This strategy was then run in MEDLINE (Ovid) and adapted for CINAHL; EMBASE (Ovid); Education Resources Information Center (ERIC) (EBSCO); Library and Information Science Abstracts (LISA); Library, Information Science & Technology Abstracts (LISTA); and Proquest Theses and Dissertations from 1990 to January 16, 2015. Four thousand ninety-seven articles were found, with 2,747 articles remaining after duplicates were removed. No limits for language or publication type were applied at the searching stage. EndNote citation software was used for duplicate removal and screening. The MEDLINE search strategy can be found in the supplemental appendix.

Stage 3: Selecting studies

The two authors each independently screened the titles and abstracts of all 2,747 articles, resulting in the exclusion of 2,143 articles. We then screened the remaining 604 full-text articles for eligibility. A third librarian was available to resolve any disagreements that were not resolved by discussion between the 2 authors; however, external consultation was not needed.

To be included in the review, the studies had to describe the assessment of IL skills in undergraduate or graduate physical therapy, occupational therapy, speech-language pathology, nursing, or general allied health academic degree programs. We defined IL skills as any instruction on question formulation, searching, database use, critical appraisal, or any library skills that pertained to these aspects of EBP. A librarian did not have to be involved in the instruction or assessment, except in studies that focused exclusively on critical appraisal, in which case librarian involvement was a requirement for inclusion, in order to keep the focus on areas of librarian expertise.

We excluded articles if they were not in English or in French. One hundred thirty-five studies were initially included for data extraction. Due to the large number of included studies, we conducted a further round of screening to put aside the articles that pertained to nursing or allied health disciplines other than physical therapy, occupational therapy, or speech-language pathology.

This paper reports on the physical therapy, occupational therapy, and speech-language pathology portion of the data set. The final round of screening resulted in 33 articles for data extraction. For the update, the first author conducted a search from January 17, 2015, to January 16, 2017, and found 238 references, 141 after duplicates were removed. We screened these references using the new criteria, yielding 7 studies for inclusion. The PRISMA flow diagram (Figure 1) details the screening process [34].

Stage 4: Charting the data

The two authors independently extracted the data from the forty included articles using an Excel spreadsheet with predefined fields for the following information in each study: country of study, degree level of learners, health profession of learners, librarian involvement (none or as instructor, assessor, author), measures of attitude used, measures of performance used, testing of measures for reliability or validity, instructional context of the IL instruction, knowledge and skills taught, and stated learning outcomes.

To determine whether the study assessed IL knowledge and skills, we had to explicitly define what IL knowledge and skills were important for EBP [15]. Using the study by Shaneyfelt et al. [12] as a model, and our own experience and research in EBP and IL to further refine, we used the following definitions while coding the articles:

Knowledge and/or skills in question formulation (PICO format and other research question formats); database searching (any mention of databases or the mechanics of searching); search quality (explicit content addressing appraisal of a search); search development (explicit content addressing how to develop a good search); resource selection (how to choose a database or another resource); article selection (content to select based on study design and relevance to topic); and article appraisal (critical appraisal)

To categorize the assessments found, we used the definitions in Schilling and Applegate:

Measures of performance are tests, course products, and portfolios; measures of attitude are self-report surveys, interviews, focus groups. [14]

We had hoped to report on whether the assessments were formative (in-process) measures or summative (at the end) and at what stage in the EBP cycle they took place, but the articles did not report enough information to make coding these data points possible.

We resolved any differences in data charting by discussion, with both authors returning to the original article to make sure that the data matched the article. No formal quality assessment was undertaken as the goal was to find all assessments being used, regardless of the quality of the study in which they were used.

Stage 5: Collating, summarizing, and reporting the results

We used the data charted above to collate the data into descriptive statistics and summarize the qualitative themes that emerged. The qualitative themes arose from discussions between the two authors.

RESULTS

Of the 40 included studies, the level of learners were undergraduate (n=15), graduate (n=19), or mixed undergraduate and graduate (n=6). Most of the studies were conducted in the United States (n=25) with other studies in Canada (n=4), Australia (n=5), continental Europe (n=3), Republic of Ireland (n=2), and the United Kingdom (n=1). The studies were evenly distributed across the included health professions: physical therapy (n=11), occupational therapy (n=11), mixed physical and occupational therapy programs (n=11), and speech-language pathology (n=7). Table 1 provides more information. Three major themes emerged as the authors discussed their findings: types of measures used, type and context of librarian involvement, and skills and outcomes described.

Table 1 Included studies

OT=occupational therapy.

PT= physical therapy.

SLP=speech-language pathology.

Theme: Types of measures used

Thirty-four measures of attitude found in 30 studies were collated into the following categories: self-report surveys (n=25), focus groups (n=2), informal feedback (n=2), and interviews (n=1) [40]. Most of the self-report surveys were developed locally. Eight studies used one of the following published instruments: evidence-based practice profile (EBP2) [63, 67, 68]; knowledge, attitude, and behavior (KAB) questionnaire [80]; practice-scholar self-efficacy survey [52]; EBP competency scale [75]; self-evaluation in EBP [8]; and EBP self-efficacy and task value [77].

Thirty-seven measures of performance in 34 studies were collated into the following categories: course products (n=24); non-IL standardized tests, defined as tests developed for EBP contexts (n= 6, using 4 tests and 1 rubric); IL standardized tests, defined as tests developed for IL contexts (n=2, using the research readiness self-assessment [RRSA] [73]); and locally developed tests (n=5). The course products varied widely; for example, there were 7 searching assignments, where the search strategy was the main focus; 5 case-based inquiry [86] assignments, where students applied the 5 EBP steps to a patient scenario; 5 critically appraised topic assignments, where students summarized the evidence on a given topic; and 4 research project assignments. One study adapted the Valid Assessment of Learning in Undergraduate Education (VALUE) rubric to assess an EBP assignment [82].

The non-IL standardized tests were all originally tested for reliability and validity in separate published studies: the Adapted Fresno Test (for occupational therapy), the Dutch Modified Fresno Test (for speech-language pathology, in Dutch only), the knowledge of research evidence competencies (K-REC) test, and the California critical thinking skills test (CCTST). Table 2 provides more details on all of the published measures.

Table 2 Published assessment measures

Theme: Type and context of librarian involvement

Librarian involvement was nearly equally divided, with almost half (n=19, 47.5%) of the studies involving a librarian. In these studies, the librarian was most frequently involved as an instructor (n=15), but the librarian was also involved as an author (n=14), as an assessor (n=12), and as a course designer (n=2). Though librarian involvement was not explicitly stated in the other half (n=21, 52.5%) of the studies, the students received instruction and/or assessment of searching and other IL skills. All but one of the included studies involving librarians used some measure of performance. The full breakdown of librarian involvement can be seen in Table 3.

Table 3 Librarian role in the study and assessment measures used

The majority of the studies (n=33, 82.5%) had IL knowledge and skills being taught in a course-integrated context by either a librarian or the instructor, though there were studies with one-shot instruction (n=2) and studies where the delivery context was unclear (n=2) or where no course was directly associated (n=3). While not all studies had clearly stated learning outcomes, 9 studies that did not describe librarian involvement included explicit statements regarding knowledge and skills often taught by a librarian, such as “effectively search research literature and judiciously select relevant evidence” [48] and “enable students to develop skills in formulating answerable clinical questions and finding and evaluating the research evidence” [70]. Of the 3 studies that used the Adapted Fresno Test or the Modified Dutch Fresno Test as the assessment for EBP knowledge and skills, both of which include a section on assessing the “finding the evidence” portion of the EBP cycle, none of the studies described the involvement of a librarian.

Theme: Skills and outcomes described

Eighteen of the studies described the instructional content of the lectures, workshops, and course materials, and another 8 studies did not describe the content but had learning outcomes that described EBP knowledge and skills. For the purpose of summarizing the knowledge and skills taught, we merged the descriptions of instructional content and of learning outcomes, even though stating in an outcome that a course will teach certain concepts does not necessarily mean that the stated knowledge or skill has been taught. We have reported the specific knowledge and skills (as described in the methods) in Table 4. Most of these 26 studies (n=23, 88.5%) described database searching knowledge or skills, including 9 studies that did not involve a librarian. The second most common content topic of these 26 studies was question formulation (n=21, 80.8%), with librarians involved in 10 of the studies.

Table 4 Librarian role in the study and assessment measures used

* Librarian involvement.  

DISCUSSION

The forty included studies revealed a wide variety of attitude and performance measures being used in rehabilitation sciences student contexts. The published self-report surveys are worth investigating for adaptation in library contexts; however, a deeper investigation and discussion of these measures is beyond the intended scope of this review. These measures may provide guidance for librarians who want valid and reliable ways of understanding students’ perceptions of IL teaching and learning.

Of the validated tests of performance, the Adapted Fresno Test [51] and the Modified Fresno Test [90] have the most potential for instructional librarians to incorporate into their own assessments, as they have already been adapted to occupational therapy and physical therapy and have components for evaluating the first two steps of the EBP cycle.

This review did not find an example study using the Modified Fresno Test (other than the validation study), but it was included here as a suggested assessment measure for those teaching physical therapy students. Unfortunately, there is no validated English version for speech-language pathology. The benefit of the adapted or Modified Fresno Test is that it evaluates a real-life application of knowledge and skills [91], testing the student’s ability to take a case through the five EBP steps. One drawback of the adapted or Modified Fresno Test is that these measures were originally designed to be taken as a stand-alone exam, where the student is asked to describe how to search instead of providing an actual search history. As such, the adapted or Modified Fresno Test equates describing how to search to actually using a database and does not test the student’s ability to apply the knowledge and skills that librarians teach in IL instruction.

The K-REC test [65], while based on the Fresno Test, primarily tests knowledge and does not cover IL knowledge well. The CCTST [42] is a commercially available test and does not directly test EBP knowledge. The only IL standardized test included in our review, the RRSA [73], was not used in a course-integrated instruction setting, and it is unclear if it tests the effectiveness of IL instruction in EBP contexts.

The large number of assignments that focused on IL knowledge and skills provide many examples for librarians designing their own assignments for use in courses. Course assignments are an excellent method for librarians to become involved in the assessment cycle, especially when class time is limited. While time-consuming to develop, a valid and reliable grading rubric is one way for librarians to give more rigor to assignments used as assessment measures [92].

Our review did not find any examples of grading rubrics that measure skills in a meaningful way and are ready for use by interested librarians. Thomas et al. used a comprehensive and instructive EBP reference model [93] as a grading rubric for their EBP case-inquiry, but it would be impractical for librarians to use for grading assignments due to its complexity [10]. Grant and Brettle as well as Evenson used simple yes/no rubrics [56, 57]. The adaptation of the VALUE rubric to the graduate physical therapy student context by Turbow and Evener might be the most useful [82], as it assessed multiple skills in the EBP cycle and could be widely adopted by librarians who are grading case-based inquiries, critically appraised topics, or research project assignments. However, librarians using the VALUE rubric would most likely need to create more detailed requirements for the points awarded for each category in order to make it an effective assessment tool. It is our opinion that more studies are needed to verify the VALUE rubric’s reliability and validity in health sciences and graduate student contexts.

The fact that all but one of the included studies involving librarians used some measure of performance suggests that health sciences librarians are doing performance-based assessment. Most of these assessment measures were course assignments; therefore, their impact on learning can be more difficult than tests to quantify, unless they are graded in a systematic manner. The use of grading rubrics, as discussed above, and clearly defined learning outcomes aligned with these rubrics would allow better demonstration of the impact of assignments.

The fact that the majority of the included studies described course-integrated instruction suggests that a place has already been made in these curricula for EBP instruction and the accompanying IL knowledge and skills. We had hoped to make a connection between the learning outcomes of a course, the knowledge and skills taught by the instructional activities, and the assessment measures used in order to better understand how the knowledge and skills were assessed. However, most of the studies did not describe in sufficient detail all three components to make these connections. Instead, as it emerged that IL skills were often taught or stated as outcomes in courses where librarians were not mentioned, we collated what knowledge and skills were taught and by whom.

It was striking to discover the number of courses that reported learning outcomes that addressed searching and database use but that did not have a librarian involved in the course. This trend suggests that there are still barriers to instructors recognizing the value and expertise of librarians in EBP instruction. As experts in IL, librarians can model the proper knowledge and skills, particularly in the first three EBP steps, to help future clinicians reach eventual expertise and behavior in clinical practice [15].

With course learning outcomes already aligned with IL outcomes, there are many opportunities for librarians to work with instructors to develop course content and assessments to better teach to these outcomes, if instructors are open to this collaboration. In cases where the librarian is not involved at all, the time has already been made in the curriculum and course instructors may be open to assistance from the librarian expert in designing and teaching the content. In cases where the librarian is involved in the instruction, but not the assessment, there may be an assignment or other assessment measure that the librarian could use to assess IL knowledge and skills. With some modifications, portions of the adapted or Modified Fresno Test could be used by librarians to evaluate question formulation and database searching more effectively. Librarians could work with instructors to integrate the entire adapted or Modified Fresno Test into the course assessment. Another option would be for librarians to use the VALUE rubric as modified by Turbow and Evener [82] with existing assignments.

The limitations of our study were as follows: It was possible that we missed relevant studies due to the large and ever-changing body of literature. We also did not contact authors for additional data in cases where studies lacked detail about such points as the involvement of a librarian or specific learning objectives.

This review confirmed for us that useful assessment tools in EBP instruction take time to develop and implement, on the part of the librarian and in the context of larger course programs. Assessment measures that test knowledge can be useful, especially if that is the only measure that a librarian can integrate into a course. However, knowledge tests will not evaluate whether instructional content such as hands-on practice with question formulation and database searching is effective in skill development. Assessments that measure skills, such as grading rubrics or the adapted or Modified Fresno Test, better indicate the effectiveness of instruction on student learning. Future areas of research could focus on validating modifications of the adapted or Modified Fresno Test that better assess searching skills and on further validating the VALUE rubric in EBP contexts.

Health sciences librarians spend a great deal of time developing and delivering instruction, and it is important to find measures that assess this instruction in an authentic way. More studies need to be done on the long-term retention of IL knowledge and skills. Given the lifelong learning characteristics of rehabilitation sciences professional programs, measuring long-term knowledge retention and skill development is essential to proving the value of IL instruction.

Supplemental File

REFERENCES

1 Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ. 1996 Jan 13;312(7023):71–2.
      

2 Canadian Association of Speech-Language Pathologists and Audiologists. Assessing and certifying clinical competency: foundations of clinical practice for audiology and speech-language pathology 2004 [Internet]. The Association [cited 9 Oct 2017]. <http://www.sac-oac.ca/professional-resources/resource-library/assessing-and-certifying-clinical-competency-foundations>.

3 National Physical Therapy Advisory Group. Essential competency profile for physiotherapists in Canada (2009) [Internet]. The Association [cited 9 Oct 2017]. <http://npag.ca/English/joint.html>.

4 Canadian Association of Occupational Therapists. Profile of practice of occupational therapists in Canada (2012) [Internet]. The Association [cited 9 Oct 2017]. <http://npag.ca/PDFs/Joint%20Initiatives/PTA%20profile%202012%20English.pdf>.

5 Franzen S, Bannon CM. Merging information literacy and evidence-based practice in an undergraduate health sciences curriculum map. Commun Inf Lit. 2016;10(2):245–63.

6 Adams NE. A comparison of evidence-based practice and the ACRL information literacy standards: implications for information literacy practice. Coll Res Libr. 2014 Mar;75(2):232–48.
  

7 Boruff JT, Thomas A. Integrating evidence-based practice and information literacy skills in teaching physical and occupational therapy students. Health Inf Libr J. 2011 Dec;28(4):264–72.
  

8 Straus SE, Gasziou P, Richardson WS, Haynes RB. Evidence-based medicine: how to practice and teach it. Edinburgh, Scotland, UK: Elsevier Churchill Livingstone; 2011.

9 Rosenberg W, Donald A. Evidence based medicine: an approach to clinical problem-solving. BMJ. 1995 Apr 29;310(6987):1122–6.
      

10 Thomas A, Saroyan A, Snider LM. Evidence-based practice behaviours: a comparison amongst occupational therapy students and clinicians. Can J Occup Ther. 2012 Apr;79(2):96–107.
    

11 Coomarasamy A, Khan KS. What is the evidence that postgraduate teaching in evidence based medicine changes anything? a systematic review. BMJ. 2004 Oct 30;329(7473):1017.
      

12 Shaneyfelt T, Baum KD, Bell D, Feldstein D, Houston TK, Kaatz S, Whelan C, Green M. Instruments for evaluating education in evidence-based practice - a systematic review. JAMA. 2006 Sep 6;296(9):1116–27.
    

13 Glegg SMN, Holsti L. Measures of knowledge and skills for evidence-based practice: a systematic review. Can J Occup Ther. 2010 Oct;77(4):219–32.
    

14 Schilling K, Applegate R. Best methods for evaluating educational impact: a comparison of the efficacy of commonly used measures of library instruction. J Med Libr Assoc. 2012 Oct;100(4):258–69. DOI: http://dx.doi.org/10.3163/1536-5050.100.4.007.
      

15 Thomas A, Saroyan A, Dauphinee WD. Evidence-based practice: a review of theoretical assumptions and effectiveness of teaching and assessment interventions in health professions. Adv Health Sci Educ. 2011 May;16(2):253–76.
  

16 Gilchrist D, Oakleaf M. An essential partner: the librarian’s role in student learning assessment 2012 [Internet]. National Institute for Learning Outcomes Assessment [cited 9 Oct 2017]. <http://www.learningoutcomeassessment.org/occasionalpaperfourteen.htm>.

17 Oakleaf M. Dangers and opportunities: a conceptual map of information literacy assessment approaches. Portal Libr Acad. 2008 Jul;8(3):233–53.
  

18 Warner DA. Programmatic assessment: turning process into practice by teaching for learning. J Acad Librariansh. 2003 May;29(3):169–76.
  

19 Cull BW. Voices in the wilderness: a report on academic information literacy instruction in Atlantic Canada. Can J Inf Libr Sci. 2005 Mar;29(1):1–26.

20 Crawford J. The use of electronic information services and information literacy. J Librariansh Inf Sci. 2006;38(1):33–44.
  

21 Schilling K, Applegate R. Evaluating library instruction: measures for assessing educational quality and impact. Proceedings of the Thirteenth National Conference of the Association of College and Research Libraries. Chicago, IL: Association of College and Research Libraries; 2007. p. 206–14.

22 Molteni VE, Chan EK. Student confidence/overconfidence in the research process. J Acad Librariansh. 2015 Jan;41(1):2–8.
  

23 Swartz B, Ratcliff A, Ivanitskaya L. Correlation of attitudes and beliefs with actual abilities of speech-language pathology students regarding aspects of information literacy. Contemp Issues Commun Sci Disord. 2015 Fall;42:215–25.

24 Stevenson P. Evaluating educational interventions for information literacy. Health Inf Libr J. 2012 Mar;29(1):81–6.
  

25 Brettle A. Evaluating information skills training in health libraries: a systematic review. Health Inf Libr J. 2007 Dec;24(supp 1):18–37.
  

26 Walsh A. Information literacy assessment. where do we start? J Librariansh Inf Sci. 2009;41(1):19–28.
  

27 Maggio LA, Kung JY. How are medical students trained to locate biomedical information to practice evidence-based medicine? a review of the 2007–2012 literature. J Med Libr Assoc. 2014 Jul;102(3):184–91. DOI: http://dx.doi.org/10.3163/1536-5050.102.3.008.
      

28 Just ML. Is literature search training for medical students and residents effective? a literature review. J Med Libr Assoc. 2012 Oct;100(4):270–6. DOI: http://dx.doi.org/10.3163/1536-5050.100.4.008.
      

29 Swanberg SM, Dennison CC, Farrell A, Machel V, Marton C, O’Brien KK, Pannabecker V, Thuna M, Holyoke AN. Instructional methods used by health sciences librarians to teach evidence-based practice (EBP): a systematic review. J Med Libr Assoc. 2016 Jul;104(3):197–208. DOI: http://dx.doi.org/10.3163/1536-5050.104.3.004.
      

30 Dizon JMR, Grimmer-Somers KA, Kumar S. Current evidence on evidence-based practice training in allied health: a systematic review of the literature. Int J Evid Based Healthc. 2012 Dec;10(4):347–60.
    

31 Buchanan H, Siegfried N, Jelsma J. Survey instruments for knowledge, skills, attitudes and behaviour related to evidence-based practice in occupational therapy: a systematic review. Occup Ther Int. 2016;23(2):59–90.
  

32 Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.
  

33 McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016 Jul;75:40–6.
    

34 Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009 Jul 21;339:b2535.
      

35 Alverson KL. Evidence-based practice in early training of speech-language pathologists [master’s of arts thesis]. Ann Arbor, MI: Wayne State University; 2009.

36 Zipoli JRP, Kennedy M. Evidence-based practice among speech-language pathologists attitudes, utilization, and barriers. Am J Speech Lang Pathol. 2005 Aug;14(3):208–20.
    

37 Bennett S, Hoffmann T, Arkins M. A multi-professional evidence-based practice course improved allied health students’ confidence and knowledge. J Eval Clin Pract. 2011 Aug;17(4):635–9.
  

38 Bennett S, Tooth L, McKenna K, Rodger S, Strong J, Ziviani J, Mickan S, Gibson L. Perceptions of evidence-based practice: a survey of Australian occupational therapists. Aust Occup Ther J. 2003 Mar;50(1):13–22.
  

39 Young JM, Glasziou P, Ward JE. General practitioners’ self ratings of skills in evidence based medicine: validation study. BMJ. 2002 Apr 20;324(7343):950–1.
      

40 Bortone JM. Critical thinking and evidence-based practice in problem-based learning tutorial groups: a critical case study [doctor of education dissertation]. Ann Arbor, MI: Fordham University; 2007.

41 Straus SE. Evidence-based medicine: how to practice and teach EBM. Edinburgh, Scotland, UK; New York, NY: Elsevier/Churchill Livingstone; 2005.

42 Facione PA. The California critical thinking skills test—college level. Technical report #1. Experimental validation and content validity. 1990. Document no.: TM 015 818. [Accessed from ERIC.]

43 Bozzolan M, Simoni G, Balboni M, Fiorini F, Bombardi S, Bertin N, Da Roit M. Undergraduate physiotherapy students’ competencies, attitudes and perceptions after integrated educational pathways in evidence-based practice: a mixed methods study. Physiother Theory Pract. 2014 Nov;30(8):557–71.
    

44 Bozzolan M, Pavanello A, Barbieri R, Spada M, Del Giovane C, Gaiani R. The adapted Fresno test to assess knowledge and skills in evidence based practice: the Italian version/Il test di Fresno adattato per valutare conoscenze ed abilita’ in evidence based practice: la versione italiana. Scienza Riabilitativa. 2011;13:12+.

45 Brooks SV, Bigelow S. Preparing students for research: faculty/librarian collaboration in a pre-doctoral physical therapy research course. Health Inf Libr J. 2015 Dec;32(4):332–8.
  

46 Burnham JF. Information management education for students in the health care professions: a coordinated, integrated plan. Med Ref Serv Q. 1994;13(2):45–62.
  

47 Cobus-Kuo L, Waller J. Teaching information literacy and evidence-based practice in an undergraduate speech-language pathology program: a student reflection. Contemp Issues Commun Sci Disord. 2016 Spring;43:35–49.

48 Cohn E, Coster W, Kramer J. Facilitated learning model to teach habits of evidence-based reasoning across an integrated master of science in occupational therapy curriculum. Am J Occup Ther. 2014 Sep–Oct;68(suppl 2):S73–82.
    

49 Crabtree J, Justiss M, Swinehart S. Occupational therapy master-level students’ evidence-based practice knowledge and skills before and after fieldwork. Occup Ther Health Care. 2012 Apr–Jul;26(2–3):138–49.
    

50 McCluskey A, Lovarini M. Providing education on evidence-based practice improved knowledge but did not change behaviour: a before and after study. BMC Med Educ. 2005 Dec 19;5:40.
      

51 McCluskey A, Bishop B. The adapted Fresno test of competence in evidence-based practice. J Contin Educ Health Prof. 2009 Spring;29(2):119–26.
    

52 Crist P. Adapting research instruction to support the scholarship of practice: practice-scholar partnerships. Occup Ther Health Care. 2010 Jan;24(1):39–55.
    

53 Bieschke KJ, Bishop RM, Garcia VL. The utility of the research self-efficacy scale. J Career Assess. 1996;4(1):59–75.
  

54 Cusack T, O’Donoghue G. The introduction of an interprofessional education module: students’ perceptions. Qual Prim Care. 2012;20(3):231–8.
  

55 Durando P, Oakley P. Developing information literacy skills in nursing and rehabilitation therapy students. J Can Health Libr Assoc. 2005;26(1):7–11.
  

56 Evenson ME. Preparing for fieldwork: students’ perceptions of their readiness to provide evidence-based practice. Work. 2013;44(3):297–306.
  

57 Grant MJ, Brettle AJ. Developing and evaluating an interactive information skills tutorial. Health Inf Libr J. 2006 Jun;23(2):79–88.
  

58 Rosenberg WM, Deeks J, Lusher A, Snowball R, Dooley G, Sackett D. Improving searching skills and evidence retrieval. J R Coll Physicians Lond. 1998 Nov–Dec;32(6):557–63.

59 Griffin N, Schumm RW. Instructing occupational therapy students in information retrieval. Am J Occup Ther. 1992 Feb;46(2):158–61.
    

60 Hoberecht T, Randall K, Schweikhard AJ. Library tutorials in an allied health evidence-based practice class. Med Ref Serv Q. 2015;34(2):240–8.
    

61 Lechner DL. Graduate student research instruction: testing an interactive web-based library tutorial for a health sciences database. Res Strateg. 2007;20(4):469–81.
  

62 Lederer JM. Evidence for occupational therapy interventions: a student educational assignment. Occup Ther Health Care. 2004;18(4):29–40.
  

63 Lewis LK, Wong SC, Wiles LK, McEvoy MP. Diminishing effect sizes with repeated exposure to evidence-based practice training in entry-level health professional students: a longitudinal study. Physiother Can. 2016;68(1):73–80.
      

64 McEvoy MP, Williams MT, Olds TS. Development and psychometric testing of a trans-professional evidence-based practice profile questionnaire. Med Teach. 2010;32(9):e373–80.
    

65 Lewis L, Williams MT, Olds TS. Development and psychometric testing of an instrument to evaluate cognitive skills of evidence based practice in student health professionals. BMC Med Educ. 2011 Oct 3;11:77.
      

66 Maritz CA, Thielman G, Campolo M. Using a capstone project to prepare students to become evidence-based practitioners. J Fac Dev. 2011 May;25(2):12–7.

67 McEvoy M, Williams MT, Olds TS. Evidence based practice profiles: differences among allied health professions. BMC Med Educ. 2010 Oct 12;10:69.
      

68 McEvoy M, Williams MT, Olds TS, Lewis LK, Petkov J. Evidence-based practice profiles of physiotherapists transitioning into the workforce: a study of two cohorts. BMC Med Educ. 2011 Nov 29;11:100.
      

69 Olsen N, Lygren H, Espehaug B, Nortvedt MW, Bradley P, Bjordal JM. Evidence-based practice exposure and physiotherapy students’ behaviour during clinical placements: a survey. Physiother Res Int. 2014 Dec;19(4):238–47.
    

70 Portney LG. Evidence-based practice and clinical decision making: it’s not just the research course anymore. J Phys Ther Educ. 2004 Winter;18(3):46–51.

71 Raghavendra P. Teaching evidence-based practice in a problem-based learning course in speech-language pathology. Evid Based Commun Assess Interv. 2009;3(4):232–7.
  

72 Ratcliff A, Swartz B, Ivanitskaya L. Information literacy skills in speech-language pathology students: skill differences across academic levels. Contemp Issues Commun Sci Disord. 2013 Spring;40:31–9.

73 Ivanitskaya L, Laus R, Casey AM. Research readiness self-assessment. J Libr Adm. 2004;41(1–2):167–83.

74 Reynolds S. Teaching evidence-based practice in a distance education occupational therapy doctoral program: strategies for professional growth and advancing the profession. Occup Ther Health Care. 2010 Jan;24(1):56–67.
    

75 Sabus C. The effects of modeling evidence-based practice during the clinical internship. J Phys Ther Educ. 2008;22(3):74–84.

76 Scott P, Altenburger PA, Kean J. A collaborative teaching strategy for enhancing learning of evidence-based clinical decision-making. J Allied Health. 2011 Fall;40(3):120–7.
  

77 Spek B, Wieringa-de Waard M, Lucas C, van Dijk N. Teaching evidence-based practice (EBP) to speech-language therapy students: are students competent and confident EBP users? Int J Lang Commun Disord. 2013 Jul–Aug;48(4):444–52.
    

78 Spek B, Wieringa-de Waard M, Lucas C, Dijk N. Competent in evidence-based practice (EBP): validation of a measurement tool that measures EBP self-efficacy and task value in speech-language therapy students. Int J Lang Commun Disord. 2013 Jul–Aug;48(4):453–7.
    

79 Spek B, de Wolf GS, van Dijk N, Lucas C. Development and validation of an assessment instrument for teaching evidence-based practice to students in allied health care: the Dutch modified Fresno. J Allied Health. 2012 Summer;41(2):77–82.
  

80 Stronge M, Cahill M. Self-reported knowledge, attitudes and behaviour towards evidence-based practice of occupational therapy students in Ireland. Occup Ther Int. 2012 Mar;19(1):7–16.
  

81 Johnston JM, Leung GM, Fielding R, Tin KY, Ho LM. The development and validation of a knowledge, attitude and behaviour questionnaire to assess undergraduate evidence-based practice teaching and learning. Med Educ. 2003 Nov;37(11):992–1000.
    

82 Turbow DJ, Evener J. Norming a VALUE rubric to assess graduate information literacy skills. J Med Libr Assoc. 2016 Jul;104(3):209–14. DOI: http://dx.doi.org/10.3163/1536-5050.104.3.005.
      

83 Finley AP. How reliable are the VALUE rubrics? Peer Rev. 2011 Fall/Winter;13/14(4/1):31–3.

84 Rhodes TL, Finley AP. Using the VALUE rubrics for improvement of learning and authentic assessment. Washington, DC: Association of American Colleges and Universities; 2013. 44 p.

85 Van Moorsel G. Library-sponsored instruction improves core informatics competencies among allied health students: a research-based case study. J Allied Health. 2005 Fall;34(3):145–52.
  

86 Villeneuve M, Maranda S. Preparing entry-level practitioners for evidence-based practice. J Can Health Libr Assoc. 2005;26(1):13–21.
  

87 Vogel K. Librarians and occupational therapy faculty: a collaboration for teaching evidence-based practice. J Allied Health. 2012 Spring;41(1):e15–20.
  

88 Wolter JA, Corbin-Lewis K, Self T, Elsweiler A. An evidence-based practice model across the academic and clinical settings. Commun Disord Q. 2011;32(2):118–32.
  

89 Ramos KD, Schafer S, Tracz SM. Validation of the Fresno test of competence in evidence based medicine. BMJ (Clinical research ed). 2003 Feb 8;326(7384):319–21.
  

90 Tilson JK. Validation of the modified Fresno test: assessing physical therapists’ evidence based practice knowledge and skills. BMC Med Educ. 2010 May 25;10:38.
  

91 Oakleaf M. The information literacy instruction assessment cycle: a guide for increasing student learning and improving librarian instructional skills. J Doc. 2009;65(4):539–60.
  

92 Oakleaf M. Using rubrics to assess information literacy: an examination of methodology and interrater reliability. J Assoc Inf Sci Technol. 2009 May;60(5):969–83.
  

93 Thomas A, Saroyan A, Lajoie SP. Creation of an evidence-based practice reference model in falls prevention: findings from occupational therapy. Disabil Rehabil. 2012;34(4):311–28.
  

(Return to Top)

Jill T. Boruff, AHIP, jill.boruff@mcgill.ca, orcid.org/0000-0002-0338-7322, Associate Librarian, Schulich Library of Physical Sciences, Life Sciences, and Engineering, McGill University, Montreal, QC, Canada

Pamela Harrison, pamela.harrison@albertahealthservices.ca, Librarian, Knowledge Resource Service, Rockyview General Hospital, Calgary, AB, Canada

Articles in this journal are licensed under a Creative Commons Attribution 4.0 International License.

This journal is published by the University Library System of the University of Pittsburgh as part of its D-Scribe Digital Publishing Program and is cosponsored by the University of Pittsburgh Press.

Journal of the Medical Library Association, VOLUME 106, NUMBER 1, January 2018