Flipping one-shot library instruction: using Canvas and Pecha Kucha for peer teaching*†

In the short term, health sciences students utilize the information resources that librarians highlight during instruction. In the years after graduation, students demonstrate low levels of information literacy skills [1–3]. These low levels of retention likely result from the limited amount of time dedicated to guided practice during library sessions, which has been shown to be critical to learning [4]. In addition, few undergraduate curricula feature scaffolded instruction on evidence-based practice, where guidance of student learning is deployed progressively to promote stronger understanding of concepts. This raises the concern that many students may be unprepared for the rigors and expectations of graduate-level research. Consequently, health sciences librarians and educators have tested numerous methods for improving the effectiveness of information literacy and evidencebased practice instruction [5–7].

In the short term, health sciences students utilize the information resources that librarians highlight during instruction. In the years after graduation, students demonstrate low levels of information literacy skills [1][2][3]. These low levels of retention likely result from the limited amount of time dedicated to guided practice during library sessions, which has been shown to be critical to learning [4]. In addition, few undergraduate curricula feature scaffolded instruction on evidence-based practice, where guidance of student learning is deployed progressively to promote stronger understanding of concepts. This raises the concern that many students may be unprepared for the rigors and expectations of graduate-level research. Consequently, health sciences librarians and educators have tested numerous methods for improving the effectiveness of information literacy and evidencebased practice instruction [5][6][7].
Higher education has also seen a rise in the use of instructional technologies, such as increased use of video tutorials, web-conferencing tools, and learning management systems (LMSs). Such tools can enhance the effectiveness of face-to-face teaching by delivering time-of-need instruction that is available for future reference during the entire semester [8][9][10][11]. LMSs, such as Canvas or Blackboard, provide a virtual space where students take ownership of the learning process and create meaningful learning experiences for themselves and their peer learners [12]. Educators increasingly use these technologies to enable a flipped classroom model of instruction, in which the lecture and homework are reversed [13][14][15]. In older teaching models, a lecture occurs in the classroom, followed by exercises and applications of the information by students as homework. In the flipped model, students get the lecture-type information at home, then do exercises and applications in the classroom. This model allows instructors to move away from the traditional one-time lecture-based instruction and to incorporate active learning, which leads to better student performance for science, technology, engineering, and mathematics (STEM) undergraduate students [16][17][18]. The flipped classroom model also accommodates the needs of diverse populations of learners by integrating peer learning and assessment [19,20]. In contrast, traditional lecture-based approaches to information literacy have been shown to have limited effectiveness for promoting skill development and retention, and often rely on shallow, quantitative assessments to measure student learning [21]. Integrating library instruction strategically into the curriculum and using the flipped classroom model creates opportunities to use a greater variety of assessment tools at various points throughout the semester in order to conduct more meaningful assessment of student learning.
Flipped classroom models have not been widely studied for health sciences students. This study aimed to determine whether a flipped classroom approach for upper-level undergraduate students in the health sciences would improve their abilities to find, evaluate, and utilize appropriate evidence for research assignments.

Participant selection and recruitment
The University of Maryland, College Park (UMD), is a major public research university near Washington, DC. The authors recruited via email (Appendix A, online only) faculty and undergraduate students in UMD's School of Public Health and College of Agriculture and Natural Resources. These schools have similar learning outcomes and require students to utilize many of the same information resources to complete research assignments. Five classes, with a total of 175 students, were included in this pilot. Faculty and students participants were compensated with Amazon gift cards, paid for by funding from a Mid-Atlantic Chapter of the Medical Library Association (MAC-MLA) Research & Assessment Grant and the UMD Libraries' Library Research Fund.

Course development
Often, librarians develop research guides, static websites with information resources, for students to consult during the semester in addition to providing one-time lectures per faculty's requests. This approach offers a passive learning experience and achieves a low level of retention of information literacy skills. To increase student engagement, the authors designed information literacy modules in Canvas, the university's LMS, during the spring of 2014 (Appendix B, online only, item #1). Canvas is software that allows faculty to set up an online presence for their courses, where they can post readings, video tutorials, discussion boards, assignments, and quizzes. Instructors can also track student participation and performance with tools built into the software. By creating Canvas modules, we were able to directly import content into a faculty member's course space. We were added as co-instructors so that we could also view students' performance.
The Canvas course content was designed to address all five standards from the Association of College & Research Libraries (ACRL) Information Literacy Standards for Higher Education. The course was grouped into thematic modules mapped to one or more of the ACRL standards [22].
Each session of the course was modified to meet the individual needs of each of the instructors and their courses. For example, specific resources in the ''Topic development and basic resources'' module differed between a public health and an animal health management course. The modules presented short textual information, supplemented with static images and streaming videos that illustrated various information literacy concepts.

Flipped classroom implementation
Starting in the fall semester of 2014, we piloted a flipped information literacy program. A week prior to class, students were enrolled into the Canvas library course or at an instructor's request, modules were exported into the instructor's class space. Working as a team, students were assigned a specific module in advance of class to present to their peers, but were also required to complete all online modules and quizzes.
During subsequent in-class sessions, each team received a laptop, a flash drive, and approximately fifteen minutes to create an electronic presentation on their assigned modules, using a slightly modified Pecha Kucha format. Pecha Kucha is a Japanese presentation style used to show twenty images, each for twenty seconds, but students were instructed to use as many images as needed within their fiveminute allotted presentation time. With its heavy use of images, this approach required students to focus on finding images that enhanced their presentations rather than creating text-heavy slides. This resulted in more engaging and humorous presentations. The students led the sessions as peer educators, delivering their brief presentations, while the librarians acted as facilitators.

Program evaluation and assessment
The outcomes of this pilot program were evaluated using several methods. We designed rubrics (Appendix B, online only, item #2) for assessing students' performance on quizzes. Students peerassessed the Pecha Kucha presentations of their classmates using a rubric (Appendix B, online only, item #3). The members of the first-and second-place teams received $30 and $10 Amazon gift cards, respectively. At the end of each class, students identified on a notecard a concept, skill, or tool that they had learned of during the session, as well as one that they were still unclear about. At the end of each semester, we conducted semi-structured interviews (Appendix B, online only, item #4) with faculty participants to assess their perceptions of the program. The approximately 30-minute interviews took place at faculty offices. Faculty participants provided access to a least 1 substantive research-based assignment that students completed subsequent to the session facilitated by the librarians. We evaluated students' performance on these assignments, using an adapted version of the Association of American College & Universities (AACU) Information Literacy VALUE Rubric [23] (Appendix B, online only, item #5).

Quizzes
The quizzes provided us with two valuable indicators: whether students completed the modules prior to class and how well students learned basic information literacy concepts and skills from self-directed learning. Each quiz featured open-response questions, which were graded by two librarians using a rubric (Appendix B, online only, item #6).
Students performed best on basic information literacy concepts such as topic development, resource evaluation, and citing sources. Students scored notably lower in more advanced concepts and skills specifically relevant for conducting upper-level health sciences research-such as using cited reference searching or the patient problem or population, intervention, comparison, and outcomes (PICO) method-to develop a searchable clinical question and perform cited reference searches.

Student presentations
While watching students' presentations in class, the audience assessed the presenters by using a rubric (Appendix B, item #3). Student presenters generally abided by the limitations of the Pecha Kucha method through the use of a wide variety of images, often drawn from popular culture, to engage their classmates with humor. Presentations also were free of library jargon, making them more accessible to their peers.

Notecards
Notecards proved useful in classes in which we did not have the opportunity to clarify issues that were still unclear or to conduct an in-class assignment to test students' competencies. The most commonly reported response was learning about where and how to search the literature for relevant evidence, including which journal databases to consult and how to use Medical Subject Headings (MeSH), the PICO method, and Boolean operators to improve the precision and comprehensiveness of searches.
Students also mentioned learning about library services (e.g., interlibrary loan), how to cite sources, how to identify more recent articles through cited reference searching, the difference between a literature review and an annotated bibliography, and how to save their findings using citation tools (Figure 1, online only).
However, a considerable number of students reported still being unclear about how to use the search techniques that they were introduced to, the libraries' services, issues of academic integrity, and how to cite sources in American Psychological Association (APA) style. Remarkably, given the students' relatively poor performance in conducting cited reference searches during the pre-session quizzes, only one student reported being unclear on how to perform a cited reference search (Figure 1, online only).

Student research assignments
While the details of student research assignments varied from class to class, all five classes required students, independently or in groups, to submit an assignment that required significant use of the literature. Unless specifically required in the assignment description to exclusively use peerreviewed journal articles, students tended to use free web resources and secondary sources more often. Students, however, were able to successfully search for and find refereed journal articles that were recent, with the vast majority published within the last five years. Some students cited commercial or industry publications from websites of questionable validity, but the majority of students cited government publications from federal agencies or university extension services.
The majority of student assignments demonstrated capstone or milestone three level of competency in our version of the AACU Information Literacy VALUE rubric, ''Use information effectively to accomplish a specific purpose.'' In their assignments, students critically evaluated the resources that they found, used many recent peerreviewed journal articles, and synthesized the information they found with clarity and purpose. In terms of citing sources, the majority of assignments reached at least milestone three for the AACU standard, ''Access and use information ethically and legally,'' by consistently providing in-text citations or reference lists, but not both. The remainder of student assignments demonstrated milestone two, by consistently providing relatively complete citations that did not meet the exact particulars of APA style. Some students also struggled at times with differentiating between common knowledge and statements that required a citation.

Semi-structured faculty interviews
Four faculty instructors were interviewed. The faculty participants who had previously worked with a librarian all responded that the flipped classroom model increased student engagement in information literacy. Because the students were required to deliver a presentation to their peers, they became more invested in the content than in a traditional, lecturebased setting. Faculty also commented that students often have inflated views of their ability to use the literature in research assignments, so they fail to perceive the usefulness of instruction on scholarly research. This instructional model helped counter student overconfidence by allowing students to tease out what they did not actually know and reinforce skills with extra practice.
All four faculty respondents indicated that they would like to collaborate with a librarian on information literacy instruction in the future, wanted further integration of a librarian and evidence-based practice instruction into their future classes, and preferred the flipped classroom model for facilitating student learning.
They also provided suggestions for further improvement. Faculty noted that many students still demonstrated significant deficiencies in their abilities to search the literature and properly cite evidence, and concluded that a single flipped classroom session was insufficient for addressing this content. They suggested that graded integration of information literacy instruction could increase student engagement in the content, as well as address continued student skill deficiencies. Better coordination between the librarians and faculty with sequencing the content could also mitigate issues of duplicate instruction in multiple courses.

DISCUSSION
As flipped classrooms become more pervasive in health sciences curricula, librarians can adopt similar approaches to ensure that their instruction aligns with the expectations of their user communities. Use of the flipped classroom also enables students to engage in self-directed instruction on a greater number of topics than can be addressed in a single in-person session. Follow-up quizzes gave students additional hands-on practice applying the skills and concepts addressed in each lesson. While in class, students discussed an information literacy topic in small groups and then presented it to their classmates, enabling students to reach higher levels of Bloom's taxonomy, a commonly used framework to assess student learning [24,25]. Furthermore, the presentations provided each student in the audience with a review of the highlights of each section, further reinforcing the topics addressed in the modules. These student presentations can be assessed in real time by a course instructor or librarian, providing real-time feedback on how well students understand complex, abstract information literacy concepts and skills.
In addition to increasing student engagement and making information literacy more relevant to students, this pilot program also led to stronger partnerships between disciplinary and library faculty. Faculty began to perceive librarians as capable instructional designers, creating opportunities for further collaboration. In the interviews, faculty articulated the need for further integration of information literacy instruction and assessment into future courses. They suggested collaborating with librarians to develop learning outcomes, design assignments, and create credit-based courses. Faculty participants also expressed concern with the ineffectiveness of one-time guest lectures and have begun advocating locally for embedding research methods into the undergraduate curriculum.
There are scalability and feasibility concerns. While the Canvas course was an effective means of delivering information literacy content to students, the time commitment required for administering, facilitating, and assessing this flipped model can be considerable, especially for librarians accustomed to delivering one-time guest lectures.
This pilot project provided the opportunity for developing stronger partnerships among librarians. Whereas one-time sessions were conducted by liaison librarians working individually, a shared tutorial course in Canvas allowed librarians to learn from one another on topics such as planning lessons and assessing student learning. These are areas of librarianship that are often under-addressed in master's of library science curricula, which suggests that early career librarians could especially benefit from a role in collaborative teaching projects such as this one [26].

Limitations
For the purpose of subject recruitment, we compensated student and faculty participants. Providing winning student teams with gift cards probably increased student engagement, but whether such engagement is sustainable without compensation is worth considering. Gift cards did not work, however, to entice uninterested faculty, because participating faculty were recruited based on preexisting relationships with librarians. While student engagement could be addressed by integrating library assignments into the grading scheme for a course, attracting uninterested faculty instructors remains a challenge.
While this study provides a model for information literacy instruction that can enhance student engagement and librarian-faculty collaborations, its use of a convenience sample limits the ability to draw causal or generalizable relationships between this flipped classroom intervention and improved student achievement in research-based assignments. Working with different faculty participants in different courses presents several potentially confounding variables, including the level of instruction devoted to these topics outside of the purview of a librarian and the design of research assignments.