Instructional methods used by health sciences librarians to teach evidence-based practice ( EBP ) : a systematic review * † ‡

Methods: A systematic review of the literature was conducted searching CINAHL, EMBASE, ERIC, LISTA, PubMed, Scopus, and others. Searches were completed through December 2014. No limits were applied. Hand searching of Medical Library Association annual meeting abstracts from 2009– 2014 was also completed. Studies must be about EBP instruction by a librarian within undergraduate or graduate health sciences curricula and include skills assessment. Studies with no assessment, letters and comments, and veterinary education studies were excluded. Data extraction and critical appraisal were performed to determine the risk of bias of each study.

Guyatt first introduced the phrase ''evidence-based medicine'' (EBM) in 1991 [1].This five-step process became the standard for how physicians would incorporate evidence into clinical practice: (1) asking an answerable question based on an information need; (2) acquiring the best evidence to answer the question; (3) appraising the evidence; (4) applying the evidence along with one's clinical expertise and the patient's or population's preferences, values, and circumstances in practice; and (5) assessing how efficiently and effectively the previous steps were done [2].This approach has since been utilized in many other health-related fields-including nursing, social work, and public health-and is commonly referred to by the broader descriptor of evidencebased practice (EBP) [3].
Many curricula for health professional education include EBP components to fulfill accreditation requirements and follow national recommendations, such as those from the Association of American Medical Colleges (AAMC) and Royal College of Physicians and Surgeons of Canada [4,5].As many EBP skills are closely tied to information literacy and research has demonstrated the effectiveness of information literacy instruction in other disciplines [6][7][8], librarians are natural candidates for teaching EBP [9,10].This systematic review aims to determine the methods employed by librarians for teaching EBP within health sciences curricula and which instructional method is most effective in promoting successful learning.

METHODS
The research methodology used for this study was a systematic review based on guidelines outlined in the ''PRISMA Statement'' [11].A systematic review published by Koufogiannakis and Wiebe was used as a model for conducting a systematic review in the field of library and information science [12].

Eligibility criteria
The following criteria were used to screen and select studies about instruction to be included in the systematic review.
Inclusion criteria: n Instruction must be led by a librarian.n Instruction may be in any format, including faceto-face, online, or a hybrid of both.
n Instruction must be related to EBP including concepts such as population, intervention, comparison, outcomes (PICO); the evidence-based medicine cycle; searching of the literature to apply to clinical practice; or critical appraisal.n Instruction may be integrated into the curriculum or be a standalone course.n Study population must be health sciences-related at the diploma, undergraduate, or graduate level.n Instruction must include assessments of learner skills.
Exclusion criteria: n Instruction was focused solely on information literacy.
n Instruction was a one-shot session and/or library orientation.n Instruction was geared to a study population of practicing professionals in health sciences-related fields or in veterinary studies.n Instruction was assessed using only feedback forms or measures of learners' self-perceptions of improvement.n Letters, editorials, and comments were excluded.

Search strategy
A base search was developed and tested in PubMed using relevant search terms, both Medical Subject Headings (MeSH) and keywords, for the three key concepts: instructional methods, librarians, and EBP (Appendix A, online only).This base search was then customized for each database search to reflect database-specific subject headings.No limits were applied.Seventeen different databases and Google Scholar were searched through November 2013.An updated search of all databases was conducted in December 2014.The following databases were searched:

Study selection
In selecting studies to include in the systematic review, search results were divided into five sets, and each set was assigned to two members of the research team.Each pair independently reviewed their sets of results against the inclusion and exclusion criteria, using the title and abstract of each item.Each pair compared their results and removed all items where both had noted to exclude.For records that the pair did not agree, a third reviewer made the final decision.This smaller set was then examined using the full text of each article.Once again, for records that were not agreed upon by the pair, a third reviewer was used to make the final decision of which articles to include in the systematic review.The pairs also identified any duplicates that had not previously been located, and these were removed.The results from the five groups were combined to make the set of items for data extraction.This set included forty-nine items (thirty-eight from the initial searches conducted in November 2013 and eleven items from the updated searches conducted in December 2014).

Data extraction
Data extraction was performed on the forty-nine items, modeled after the method described by Koufogiannakis and Wiebe [12] (Appendix B, online only).Nineteen data elements were extracted from each study by two members of the research team and recorded in an Excel spreadsheet.Each pair independently extracted the data, compared their results, and removed all of the articles where both had noted a criterion-based reason to exclude.For any items that were not agreed upon by the pair, a third reviewer made the final decision.

Critical appraisal
Critical appraisal was performed concurrently with data extraction on the same set of forty-nine articles to determine the quality, validity, and applicability of each study.The nine-item Glasgow checklist was used for the appraisal [13].A different pair of team members performed the critical appraisal on the articles.

Study selection
A total of 49,795 bibliographic records were identified through database searching (45,157 in November 2013, plus another 4,638 in December 2014).An additional 456 records were identified through the MLA annual meeting abstracts (448 in November 2013 and an additional 8 in December 2014).After removing duplicates, 30,043 remained for screening.In selecting studies to include in the systematic review, the research team was divided into pairs, and each pair independently reviewed approximately 5,000 citations using only the title and abstract.Following title and abstract review, 29,406 studies were excluded leaving 637 studies for fulltext review.After full-text review, 49 studies were selected for data extraction and critical appraisal.During data extraction and critical appraisal, 22 studies were excluded for not meeting 1 or more inclusion criterion; therefore, 27 studies were included in the final set for analysis.Figure 1 shows the PRISMA flow diagram for these results.For a summary of key findings, refer to online only Table 1.

Study characteristics
Publication information.Of the twenty-seven studies selected for inclusion, twenty-six were journal articles and one was an academic conference proceeding [14].Sixteen articles, representing more than half of the publications in this set, were published in library and information science journals: seven in the Journal of the Medical Library Association [15][16][17][18][19][20][21], four in Medical Reference Services Quarterly [22][23][24][25], four in Health Information & Libraries Journal [26][27][28][29], and one in the Journal of the Canadian Health Libraries Association/Journal de l'Association des bibliothèques de la santé du Canada [30].Ten articles were published in health sciences journals, including two in BMC Medical Education [31,32], three in the Journal of General Internal Medicine [33][34][35], and five in other health sciences journals [36][37][38][39][40].
Country of origin.The majority of studies were conducted in the United States, although studies from Canada [26,28,30], the United Kingdom [27], and Italy [29] were included.
Study participants.Upper-year medical students and residents accounted for the majority of participants in these studies.However, undergraduate programs in nursing [40], dentistry [26], occupational therapy and physical therapy [28,39], respiratory therapy and physician assistant [39], and graduate-degree students in physician assistant [24] and health sciences research [27] were also represented.The number of participants and their assignment to groups (e.g., intervention and control groups) varied considerably, from 10 [16] to more than 300 [25], with an estimated average of 50 participants per training session.
With regard to instruction of the EBP process as a whole, one study [18] included the full EBP process.Five studies [17,22,32,34,36] included instruction in four out of five skills (all except assessment), and another seven studies [20,25,31,33,35,38,39] addressed three EBP skills, though which three of the five varied.For details on EBP steps covered in each study, refer to online only Table 1.
tion as well as the corresponding study numbers.The Cochrane Library followed MEDLINE as the second most frequently taught resource in 10 studies.The remaining resources represented a variety of clinical medicine tools, EBM publications, and other health sciences journal literature databases, such as CINAHL and PsycINFO.The latter were taught less often, which is consistent with the target discipline for the instruction (medicine) and for the majority of the studies selected for this review.EMBASE was not taught in any of these studies on EBP instruction.
Methods of instruction.The majority of studies used two or more teaching methods, with only seven studies relying solely on one method [16,17,23,24,29,32,38].However, some studies were not clear if only one teaching method was employed because teaching methods were not consistently described.Figure 3 depicts the instructional methods used in the studies and corresponding study numbers.The EBP instruction predominantly included lectures, also referred to in the studies as formal presentations or seminars, with twenty studies using this method.Small group discussions and/or one-onone instruction followed, with sixteen studies using this method.Hands-on practice for online database searching in a computer lab was used in fifteen studies.Online instruction was much less com-monly employed, with only six studies using this method, two of which used online teaching as their only method of instruction.
While this systematic review focused on determining the best teaching methods in librarianled instruction on EBP, only one study actually compared more than one teaching method.Schilling's presentation in 2011 at an e-learning conference compared traditional in-person EBM instruction to online EBM instruction.Students who received the online intervention had marginally better search scores than those who received traditional instruction.This difference approached but did not achieve statistical significance [14].
the researchers in individual studies or taken from earlier studies and modified were more commonly employed.Self-reporting on attitude, confidence, or skill level was also commonly utilized as an assessment method [14, 16-20, 23, 26, 27, 32-35, 38, 39], primarily pre-and post-surveys of participants' attitudes and confidence in their EBM skills and course evaluations.Only those studies that employed these self-reports in tandem with other skills data collection methods were included in the review.

DISCUSSION
Based on the results of this systematic review, it was not possible to determine which EBP teaching method was most effective.No previously published systematic reviews focused specifically on the effectiveness of instructional methods for teaching EBP with librarian involvement in health sciences curricula.There were systematic reviews that analyzed general information literacy skills training in the health sciences and the most effective methods for teaching information literacy skills to undergraduate students [12,44,45], but these studies did not address EBP instruction or directly focus on the role of the librarian in teaching EBP.Other literature reviews summarized EBM instruction in undergraduate medical education but excluded other health professional students and did not discuss the role of the librarian in teaching EBM [46,47].

Publication information
It is worth noting that sixteen of the twenty-seven articles in this review were published in library and information science journals.The academic conference proceedings article was written by a library school faculty member.While the designations of the authors did not always clearly identify librarian involvement, only three articles were written solely by librarians (who were not also clinicians) with no clinician or researcher coauthors [14,20,24].The remaining articles, regardless of the journal, involved librarians, clinicians, and/or researchers, or did not clearly indicate if a librarian was involved as a coauthor.

Methods of instruction
Studies have shown that teaching EBP skills to health professional students results in an increase in knowledge and skills [48][49][50].Library instructors often utilized a combination of teaching formats, usually lecture combined with lab practice and/or small group discussion (18 out of 27 studies), suggesting that the instructors felt that the use of a combination of teaching methods was the best approach.In fact, there were indications that the most effective form of EBP instruction was when content was delivered over multiple sessions utilizing several teaching methods [40] and that oneshot single sessions might not be the best option to teach information literacy skills [51].
Koufogiannakis and Wiebe sought to determine the most effective methods for teaching information literacy skills to students at the undergraduate college level.Using fifty-five studies that either compared two teaching methods or instruction to no instruction, they found that though each of the teaching methods was more effective than no instruction, no particular method was more effective than another [12].
Only one study in this systematic review compared more than one teaching method.Schilling's presentation in 2011 at an e-learning conference compared traditional in-person EBM instruction to online EBM instruction.Study participants were first-year medical students (n¼128).The control group (n¼63) received traditional, instructor-led training in skills in information and MEDLINE searching, while the intervention group (n¼65) participated in the same instruction via a webbased tutorial.The e-learning module was structured to exactly duplicate the classroom instruction.Students who received the latter intervention had marginally better search scores than those who received traditional instruction.This difference approached but did not achieve statistical significance [14].
The findings suggest that search performance improved regardless of instructional method, but due to the large variability in study sample, sample size, measurement tools, and statistical tests employed, the research team was unable to quantify the improvement.In a systematic review where the study included all patrons of academic libraries, nine out of ten studies did not show any difference in learning between computer-assisted instruction and traditional lecture-type instruction [52].With no consensus in the literature about which instructional methods are most effective for teaching EBP, future studies are needed to investigate and compare teaching methods in this area.

Assessment of EBP instruction
In this systematic review, all twenty-seven studies used a performance-based skills assessment or evaluation of student learning, as this was one of the inclusion criteria.Only four studies [18,19,29,31] used a standardized, validated assessment tool for skills performance assessment: the Berlin Questionnaire [41], used by one study [18], and the Fresno test [42], used by three studies [19,29,31].The EBPQ [43] was used by one study [19] for a selfreported assessment of skills, attitude, or comfort.The most popular method of assessment was found to be the pre/post format of giving tests before and after an intervention.Instructor-developed rubrics, modifications of standardized assessment tools, and intervention-control skills testing were also used.In a 2006 systematic review, Shanyefelt et al. summarized and classified assessment instruments used to evaluate EBP skills [53].A key criteria for inclusion in their study was validity of the test, which was lacking in the majority of papers in this study.It is unclear whether the lack of use of validated instruments was because there was no assessment tool that was sufficiently robust to meet the needs of health librarians' EBP teaching methods or program curricula or if awareness of the availability and applicability of these instruments was not sufficient.

Role of librarians in EBP
In the studies included in this review, librarians held an instructor role in all twenty-seven studies.Librarians were also involved in other aspects of EBP instruction from developing curricula, planning and teaching EBP material, to assessing student progress and contributing to the scholarly literature.
The nature and extent of the librarian role seems to be influenced by several factors.For example, authorship in this set of included articles seems to correlate with the role of the librarian in the study in question.In general, those authors who published in library science journals usually describe in detail the role of the librarian in all aspects of instruction.The majority of these authors are academic health sciences librarians.In contrast, similar studies published in medical journals are primarily authored by medical faculty, with one or two librarian coauthors, and the librarian role is primarily that of EBP instructor; although librarians are also involved to some extent in curriculum planning and student performance assessment in these studies as well.
Depending on the circumstances and opportunities, librarians can do well in any role and contribute at all levels in the EBP teaching process [49].Teaching information skills seems to be the most common role for librarians in EBP instruction, and research has indicated that librarians may have better learning outcomes for one-shot, face-to-face information literacy sessions than nonlibrary instructors [51].As information skills are very important for EBP, librarians should be encouraged to participate, at the very least, in the information skills step of any EBP instruction.Faculty involved with health-related curricula should be strongly encouraged by administrators to invite librarians to be part of the EBP teaching team.
Finding opportunities to include library EBP instruction in already busy school curricula, medical curricula in particular, may present a challenge [51].Depending on circumstances, web-based online courses could be considered as an alternative, because a well-designed online course may be successfully taught and result in better learning outcomes compared to one-shot lecture-type instructions [14,51].

Study design and statistical methods
For this set of twenty-seven studies, findings were generally weakly positive, showing an improvement in search performance.Several studies demonstrated robust positive findings for better search performance or positive attitudes toward EBP skills training or both, while others obtained mixed findings.Large variability in study sample, sample size, measurement tools, and statistical tests employed made quantifying the amount of positive change in search performance and other measures of EBP skills by meta-analysis impossible.Almost all studies examined whether instruction on EBP provided by librarians (and clinicians or clinical faculty) led to a measurable change in participants' skills and/or attitudes toward EBP.Only one study addressed the topic of this systematic review-which instructional method is most effective for librarians teaching EBP-and the findings showed a trend toward significance for the intervention group, which received online instruction, versus the control group, which received traditional instruction [14].

Limitations
The decisions regarding the search process may have introduced some bias in the studies found and selected for this review.Although a few non-English language databases were searched (Spanish, Portuguese, Japanese), many others were not included, potentially creating language bias.Hand searching of journals related to the research topic was not conducted, and therefore, some relevant studies may have been overlooked.Meeting posters and presentations were searched, but they were ultimately excluded for lack of complete data on quantitative learning assessment, which was one of the inclusion criteria.Also, in cases where information was unclear or not included in the actual article, authors were not contacted for additional information, which could have affected the data analysis.For example, details on the grading of assignments not included in the actual article, but retrieved from authors, could have clarified or improved the analysis of learning assessment.

FUNDING
Research was performed with no external funding.

ACKNOWLEDGMENTS
Genevieve Gore, MLIS, a team member during early project development, trained the team in the systematic review process.The authors also thank