Educational interventions to improve literature searching skills in the health sciences: a scoping review

Objective: The authors reviewed educational interventions for improving literature searching skills in the health sciences. Methods: We performed a scoping review of experimental and quasi-experimental studies published in English and German, irrespective of publication year. Targeted outcomes were objectively measurable literature searching skills (e.g., quality of search strategy, study retrieval, precision). The search methods consisted of searching databases (CINAHL, Embase, MEDLINE, PsycINFO, Web of Science), tracking citations, free web searching, and contacting experts. Two reviewers performed screening and data extraction. To evaluate the completeness of reporting, the Template for Intervention Description and Replication (TIDieR) was applied. Results: We included 6 controlled trials and 8 pre-post trials from the 8,484 references that we screened. Study participants were students in various health professions and physicians. The educational formats of the interventions varied. Outcomes clustered into 2 categories: (1) developing search strategies (e.g., identifying search concepts, selecting databases, applying Boolean operators) and (2) database searching skills (e.g., searching PubMed, MEDLINE, or CINAHL). In addition to baseline and post-intervention measurement, 5 studies reported follow-up. Almost all studies adequately described their intervention procedures and delivery but did not provide access to the educational material. The expertise of the intervention facilitators was described in only 3 studies. Conclusions: The results showed a wide range of study populations, interventions, and outcomes. Studies often lacked information about educational material and facilitators' expertise. Further research should focus on intervention effectiveness using controlled study designs and long-term follow-up. To ensure transparency, replication, and comparability, studies should rigorously describe their intervention.


INTRODUCTION
Identifying and evaluating available scientific evidence by means of literature review is one component of evidence-based practice (EBP) in health care [1]. Systematic reviews often are the bases of wide-ranging, important decisions in clinical practice [2]. Therefore, literature searches of high methodological quality have considerable scientific, practical, and ethical relevance. Constructing appropriate search strategies to retrieve relevant literature for answering clinical questions is fundamental to systematic reviews. Indeed, effective searching is a core competency of EBP for health sciences professionals, researchers, and librarians [3][4][5].
In practice, however, many systematic reviews do not employ high-quality literature searches. Several analyses demonstrate shortcomings of literature searches described by published systematic reviews. For example, 7-9 out of 10 search strategies in Cochrane reviews or other systematic reviews contain at least 1 error, such as missing Medical Subject Headings (MeSH), unwarranted explosion of MeSH terms, irrelevant MeSH or free-text terms, missed spelling variants, failure to tailor the search strategy for other databases, and misuse of logical operators [6,7]. Furthermore, 50%-80% of these errors potentially lower the recall of relevant studies and may impact the overall results of the review [6][7][8].
Education in EBP should integrate training on literature searching [3]. A recent systematic review indicates that training can improve evidence-based knowledge and skills among health care professionals [9]. However, the actual impact of such training on EBP remains unclear, largely because its implementation and patient-relevant outcomes are rarely examined or are affected by risk of bias [9,10]. Although training in EBP might improve the skills of health professionals [9], the complexity of these interventions precludes drawing conclusions about their impact on literature searching skills. EBP training can contain several intervention components, such as (1) formulating a research question, (2) developing a search strategy, (3) critically appraising the evidence, and (4) communicating research results to patients. These educational interventions and their components are characterized by varying durations and intensities of training as well as differing materials and methods of delivery [9].
Literature searching skills are, therefore, one outcome of interest among others, and the complexity of EBP training makes it challenging to isolate components and outcomes specifically related to literature searching skills. An older review published in 2003 found some evidence for a positive impact of training on health professionals' skill levels in literature searching; however, the included studies suffered from methodological shortcomings and were underpowered [11]. To scope educational interventions for improving literature searching skills in the health sciences, an updated review is warranted. Therefore, the objective of the current review was to answer the following research questions: What is known about educational interventions to improve literature searching skills in the health sciences? Which outcomes were measured? How completely are these interventions reported in the published literature?

Eligibility criteria
We included journal articles that described experimental or quasi-experimental studies in English or German, irrespective of publication year, as we were interested in all available studies focusing on educational interventions with outcome evaluation. The populations of interest were researchers, students, and librarians in the health sciences and health professionals such as nurses, physicians, and pharmacists, regardless of their searching experience or expertise. We included any type of educational intervention (e.g., training, instruction, course, information, peer review) aimed at improving health sciences-related literature searching skills.
We were interested in outcomes that were objectively measurable in terms of improvements in literature searching skills (e.g., proficiency, quality or correctness of search strategies, study retrieval). We excluded studies in which literature searching was one skill or intervention component among others (e.g., EBP courses, courses on reviewing the literature) and those concerning the performance or effectiveness of search filters, hand searching, or citation tracking. We also excluded studies addressing subjective outcomes (e.g., self-perceived knowledge and confidence in literature searching skills), as there is evidence of a weak correlation between self-perceived and objectively assessed literature searching skills [19]. In summary, we were interested in outcomes indicating an impact on literature searching performance.

Information sources
We searched CINAHL, Embase, MEDLINE via PubMed, Web of Science Core Collection, and PsycINFO via Ovid. Additionally, we performed free web searching via Google Scholar and hand searching in reviews that were identified as relevant. We also contacted expert health sciences librarians through four email lists (Netzwerk Fachbibliotheken Gesundheit, Canadian Medical Libraries, Expertsearch, and MEDBIB-L/German-speaking medical librarians). By forward and backward citation tracking of the included studies, we retrieved additional references. For citation tracking, we used Scopus because this database seemed to cover the largest number of relevant citations for the purpose of this review [20]. Based on already included studies, we searched for more references using the "cited by" and "citing" buttons in Scopus. We exported the retrieved references and, after deduplication, performed the study selection process as described below. We repeated this process using "cited by" and "citing" buttons in Scopus, if newly identified studies were eligible, until no further studies were included.

Search
The search strategy was based on database-specific controlled vocabulary and free-text terms. We identified initial search terms on the basis of our experience, an orienting search, and familiar literature. To identify relevant search terms and synonyms, we used the MeSH Browser, COREMINE Medical, and a thesaurus. We analyzed keywords in relevant publications and similar articles that we identified via PubMed to determine frequently occurring terms to include in the search strategy. Considering the presence of delays in keyword indexing, we also searched for controlled vocabulary terms in title and abstract search fields [21]. The search strategy is provided in supplemental Appendix A and Appendix B.

Selection of sources of evidence
Two authors (Hirt, Nordhausen) designed the search strategy after consultation with the senior authors (Zeller, Meyer). An information specialist (Braun) reviewed the search strategy by means of the Peer Review of Electronic Search Strategy (PRESS) [5]. One author (Hirt) conducted the search. Two independent authors (Hirt, Nordhausen) screened titles, abstracts, and full texts for inclusion using Rayyan and Citavi. We discussed conflicts until reaching consensus.

Data charting process
Two authors (Hirt, Nordhausen) developed a standardized data extraction sheet. One author (Meichlinger) extracted data on study design, country, setting, participants (i.e., number, gender, age), intervention and control characteristics, outcome measurement, time of measurement, and main results. A second author (Hirt or Nordhausen) checked data extraction. We clarified uncertainties by consulting with the senior authors (Zeller, Meyer). To determine the completeness of reporting in included studies, we applied the Template for Intervention Description and Replication (TIDieR) [22]. For this purpose, two authors (Hirt, Nordhausen) defined a set of minimum required information to meet a TIDieR criterion (supplemental Appendix C). One author (Meichlinger) assessed the reporting of interventions, which was checked by a second author (Hirt or Nordhausen).

Synthesis of results
One author (Hirt) narratively summarized study characteristics, interventions, outcomes, results, and the reporting of interventions.

Search and study selection
The systematic database and supplementary search yielded 8,484 references. After removal of duplicates, we screened 7,017 titles and abstracts and 144 full texts, resulting in the inclusion of 14 studies ( Figure 1). jmla.mlanet.org 108 (4) October 2020 Journal of the Medical Library Association Figure 1 Search and selection process

Study outcomes
Outcomes clustered into two groups: (1) search strategy development and (2)    Outcomes related to developing search strategies included using appropriate keywords and appropriate search strategies [31]; developing search strategies in general [24]; identifying search concepts [24,30]; selecting databases and where to find evidence [24,34,36]; applying Boolean operators, indexing terms, and search limits [27,30,33]; and having search quality as well as errors [26].
Outcomes related to database searching skills included database searching skills in general or searching more than one database [24,27]; creating sensitive and specific search strategies using Ovid [29]; retrieving a manageable and relevant number of references in MEDLINE [33]; having recall and precision in MEDLINE [23]; searching PubMed [36], MEDLINE [28,32], CINAHL [25,34]; and searching a library catalog [35].

Study results
Most of the included studies-which consisted of pre-post trials, RCTs, and controlled trials-reported positive outcomes (i.e., an improvement in literature searching skills). Some pre-post trials reported significant improvements in the development of search strategies [31,33,36] and database searching skills [33,36]. Other pre-post trials reported nonsignificant improvements in the development of search strategies [24,27,30] and database searching skills [24,27,29]. One study reported a significant improvement without reporting descriptive data concerning database searching skills [28].
Concerning RCTs and controlled trials, one reported a significant improvement in database searching skills [34], whereas others reported nonsignificant improvements in database searching skills [25,32,35]. Two studies reported no improvements in database searching skills [23,34]. In one study, the intervention group had a significantly higher search quality score and significantly lower number of search errors [26].

Reporting of interventions
In six studies, the educational materials that were used were not clearly described [24][25][26][34][35][36]. Almost all studies adequately reported the procedures of the interventions, except for two [31,32]. Only three studies characterized the persons who delivered the intervention in terms of their professional background, expertise, and/or specific training [23,31,32]. All studies reported how the intervention was delivered. Four studies did not report the location of the intervention [23,29,31,34]. Three studies did not report the number or frequency of educational sessions [26,31,35]. Most studies did not report tailoring, except for three studies [26,31,33], or modifications of the intervention or its planned and actual intervention fidelity, except for one [34] (Table 3).

DISCUSSION
This scoping review focused on intervention studies aiming to improve literature searching skills in the health sciences. The results showed a wide range of study populations, educational interventions and their components, and outcomes. Overall, the reporting of these studies lacked essential details about educational materials and intervention deliveries to allow a detailed understanding of how the educational interventions were applied.
Database searching is a key element of systematic literature searches [37]. Systematic reviews require a high methodological standard of literature searching using several databases and highly sensitive search strategies [38]. Eight of the fourteen included studies tested database-specific educational interventions to improve searching skills in CINAHL, MEDLINE, and PubMed [23,25,28,29,[32][33][34]36]. Since the advent of electronic databases in the late 1970s, technological progress and usability requirements resulted in fundamental changes in database functionalities, capabilities, and layouts [39]. These changes were paralleled by changes in search features and user interfaces. Recent examples are the launch of "new PubMed" as well as updated search functions and layout of the Cochrane Library. These evolutions should be taken into account when designing contemporary educational interventions to improve literature searching skills. Only two studies with databasespecific educational interventions were published within the last ten years [34,36]; thus, the timeliness of these interventions is questionable. jmla.mlanet.org 108 (4) October 2020 Journal of the Medical Library Association Abbreviations: Y=Yes, reported; N=No, not reported; -=Unclear whether it was conducted.
Journal of the Medical Library Association 108 (4) October 2020 jmla.mlanet.org Furthermore, there was general lack of complete reporting among the included studies, including detailed descriptions of database-specific educational interventions and outcomes. While the intervention procedure, format, and intensity (i.e., when and how much) were often reported, most studies lacked information about the educational material used and the expertise of the persons delivering the intervention. These two aspects-timeliness and transparent reporting of interventions-might hinder the transfer of educational interventions into current practice [40]. To better describe the context of implementation of educational interventions and ensure study transparency, replicability, adaptation, and comparability, authors should thoroughly describe their interventions [9] using TIDieR [22] or CReDECI-2 [41] guidelines.
Participants in the included studies consisted of students or physicians. Thus, these studies mainly addressed university education and clinical practice. One finding of our review was that experimental and quasi-experimental studies on objectively measurable literature searching skills of librarians as study participants were lacking, despite the increasing demand to involve skilled librarians and information specialists as methodological peer reviewers or as part of systematic review teams [42,43]. However, there is evidence that underlines the importance of the active involvement of expert librarians in systematic reviews. A recently published analysis showed that systematic reviews coauthored by librarians had less risk of bias than reviews in which librarians' contributions were only mentioned in the acknowledgments or were unclear [44]. A recently developed competency framework for librarians who are involved in systematic reviews contains indicators that are helpful for determining whether the tasks performed by librarians could be applied independently [45].
Furthermore, a discussion-based framework presenting propositions for planning, developing, and evaluating training interventions for expert librarians to participate effectively in systematic review teams was recently published [46]. These two contextual propositions might underpin the theoretical framework for effective involvement of expert librarians in systematic review teams. However, based on the results of our scoping review, it remains unclear how librarians' searching skills might be further developed or improved. Given their increasingly important role in systematic reviewing, the effectiveness of literature search skill training for librarians should be considered in future research [46].
The development of feasible and effective interventions should be accompanied by the selection of relevant outcomes that are measured at appropriate times [47]. The included studies involved different outcomes, but their times of measurement were predominantly baseline and post-intervention. Follow-up measurement was conducted only in five studies, with a wide range between one and eleven months. Therefore, it is difficult to draw conclusions regarding long-term effects of educational interventions. Studies indicate that literature searching skills can worsen over time [48]. Future research should, therefore, employ longterm evaluations to ensure that the outcomes of interventions demonstrate continuity and long-term effectiveness. This might also promote the development of enduring information material such as manuals that guide students, researchers, and librarians in systematic literature searching.
Literature search recall and precision can be improved by other potential interventions including the development of search filters, supplementary search methods such as hand searching and citation tracking, structured search forms, and objectively derived search terms [37,[49][50][51][52]. These interventions were excluded due to our focus on educational interventions. However, such interventions need to be considered in systematic literature searching effectiveness [53].
The included studies were not critically appraised [18]. Nevertheless, significant limitations of internal and external validity were evident. Most studies were neither controlled nor randomized and had small sample sizes. The presentation of results was insufficient in some studies (e.g., confidence intervals and/or estimated effect sizes were not reported). Furthermore, the included studies used different types of outcomes, impeding comparison of their results. To enhance further research, authors should ensure objective, valid, and reliable outcome measurement [54], which could lead to greater internal validity and comparability of results.
A strength of our scoping review was our comprehensive search using a peer-reviewed search strategy [5]. To transform our research question into a search strategy, we intended to use two search components: education and literature search.
jmla.mlanet.org 108 (4) October 2020 Journal of the Medical Library Association However, this resulted in a large number of search results that we could not feasibly work through considering our resources. To further limit the search results, we added three concepts under the guidance of a medical expert librarian (Braun): (1) aim of the intervention (e.g., competence, improvement, success); (2) health sciences-related participants (e.g., clinician, librarian, nurse, physician); and (3) study design (e.g., randomized, trial, quasi-experimental). Adding these search components yielded a lower number of results and increased the risk of excluding relevant studies [55]. Therefore, to avoid missing studies that might contribute to the body of evidence, we applied several supplementary search methods-including citation tracking, free web searching, hand searching in relevant reviews, and contacting experts [37,56]resulting in the identification of four additional studies.
However, a limitation of our search approach was that we did not search trial registries or conference proceedings as well as specialized educational and library-related databases such as Education Resources Information Center (ERIC); Library, Information Science & Technology Abstracts (LISTA); and Library and Information Science Abstracts (LISA). Instead, we focused on databases covering multiple disciplines such as CINAHL, PsycINFO, and Web of Science Core Collection. The non-use of specialized libraryrelated databases was due to limited subscriptions at our institution but may be a reason why no studies describing educational interventions for librarians were identified.
Another strength of our review was that we ensured high data quality by using two independent reviewers for study selection and a second reviewer to check data extraction and assessment of reporting. The analysis using TIDieR enabled a comprehensive assessment of the reporting of educational interventions to improve literature searching skills in health sciences [57]. To increase the reliability of the data extraction, two reviewers (Hirt, Nordhausen) developed minimal requirements for each TIDieR item criterion.
Our research question was limited to experimental and quasi-experimental studies employing search-specific educational interventions to improve literature searching skills. This scoping review did not address the effectiveness of interventions and quality of studies, which was appropriate for scoping review methodology [12,17]. Conclusions concerning the positive impact of the identified educational interventions are, therefore, limited. The small number of rigorous study designs that we identified has led us to conclude that a systematic review to determine the effectiveness of interventions and internal validity of studies might not be necessary at the moment, as the last RCT was published in 2013 [34]. Further primary research is needed and should focus on the longterm effectiveness of educational interventions to improve literature searching skills using controlled study designs and follow-up outcome measurement.
To understand which interventions are effective for whom and why, the development of educational interventions should follow a systematic process based on available evidence and the needs of end users and librarians. This review indicates that there is still a need for high-quality research employing well-developed educational interventions and contemporary search methods using rigorous study designs [11]. jmla.mlanet.org