Case Study

Adapting data management education to support clinical research projects in an academic medical center

Kevin B. Read

doi: http://dx.doi.org/10.5195/jmla.2019.580

Received 01 August 2018: Accepted 01 September 2018

ABSTRACT

Background

Librarians and researchers alike have long identified research data management (RDM) training as a need in biomedical research. Despite the wealth of libraries offering RDM education to their communities, clinical research is an area that has not been targeted. Clinical RDM (CRDM) is seen by its community as an essential part of the research process where established guidelines exist, yet educational initiatives in this area are unknown.

Case Presentation

Leveraging my academic library’s experience supporting CRDM through informationist grants and REDCap training in our medical center, I developed a 1.5 hour CRDM workshop. This workshop was designed to use established CRDM guidelines in clinical research and address common questions asked by our community through the library’s existing data support program. The workshop was offered to the entire medical center 4 times between November 2017 and July 2018. This case study describes the development, implementation, and evaluation of this workshop.

Conclusions

The 4 workshops were well attended and well received by the medical center community, with 99% stating that they would recommend the class to others and 98% stating that they would use what they learned in their work. Attendees also articulated how they would implement the main competencies they learned from the workshop into their work. For the library, the effort to support CRDM has led to the coordination of a larger institutional collaborative training series to educate researchers on best practices with data, as well as the formation of institution-wide policy groups to address researcher challenges with CRDM, data transfer, and data sharing.

BACKGROUND

For over ten years, data management training has been identified as a need by the biomedical research community and librarians alike. From the perspective of biomedical researchers, the lack of good quality information management for research data [1, 2] and an absence of training for researchers to improve their data management skills are recurring issues cited in the literature and a cause for concern for research overall [1, 3, 4]. Similarly, librarians practicing data management have identified that researchers generally receive no formal training in data management [5] yet have a desire to learn [6] because they lack confidence in their skills.

To address this need, librarians from academic institutions have been working to provide data management education and support to their communities. By developing specific approaches to creating data management education, libraries have found successful avenues in implementing stand-alone courses and one-shot workshops [7], integrating research data management into an existing curriculum [8], and offering domain-specific training [9]. Libraries have offered these training programs by providing general data management training to undergraduate and graduate students [10–12], doctoral scholars [13], and the general research community [14–20], whereas domain-specific data management can be seen most prominently in the life sciences [21], earth and environmental sciences [22, 23], social sciences [24], and the digital humanities [25].

While it is clear that libraries have made inroads into domain-specific areas to provide training in data management, the clinical research community—clinical faculty, project and research coordinators, postdoctoral scholars, medical residents and fellows, data analysts, and medical or doctoral degree (MD/PhD) students—is one that has not received much attention. Clinical research data management (CRDM), an integral part of the clinical research process, differs from the broader concept of research data management because it involves rigorous procedures for the standardized collection and careful management of patient data to protect patient privacy and ensure quality and accuracy in medical care. The clinical research community understands the importance of data standardization [26–29], data quality [30–33], and data collection [28, 34–36] and has established good clinical data management practices (GCDMP) [37] to ensure that CRDM is conducted at the highest level of excellence.

Despite this community-driven goal toward CRDM excellence, there is a dearth of literature about data management training for clinical research, with the only evidence coming from nursing training programs [35, 38], whose research practices are further afield in that they focus on quality improvement rather than clinical investigations. This lack of evidence is surprising considering that the need for CRDM training has been communicated [1, 3, 4, 6].

My library, located in an academic medical center, has supported CRDM through National Library of Medicine informationist projects by collaborating with clinical research teams to improve data management practices [39] and, more recently, by serving as the front line of support for REDCap (an electronic data capture system for storing research data) by offering consultations and comprehensive training [40]. Through REDCap training, I identified a need to expand my knowledge of CRDM to better support the needs of our research community. While REDCap is a tool to help researchers collect data for their studies, the majority of issues that our clinical research community encountered were related to data management. These issues included developing data collection plans, assigning and managing roles and responsibilities throughout the research process, ensuring that the quality of data remains intact throughout the course of the study, and creating data collection instruments. As this recurring thread of issues expanded the learning needs of our community beyond those provided via our REDCap training, I decided to expand my knowledge to address the questions that our researchers asked, to develop a curriculum to support CRDM, and to offer and evaluate CRDM training for our community.

STUDY PURPOSE

This case study will discuss (a) the development and implementation of a 1.5-hour CRDM workshop for the medical center research community, (b) the results and outcomes from teaching the CRDM workshop, and (c) the next steps for the library in this area.

CASE PRESENTATION

Workshop development

Gaining skills

Beyond the experience I gained from working closely with researchers on their clinical research projects and through REDCap support, I took two particularly valuable training opportunities that improved my skills in CRDM: the “Data Management for Clinical Research” Coursera course [41] and “Developing Data Management Plans” course [42] offered through the online educational program sponsored by the Society for Clinical Data Management. These two courses provided me with the knowledge that I needed to teach a CRDM workshop but more importantly gave me the confidence to teach it because they provided a depth of knowledge I did not have before. These courses also served to reinforce that the issues and challenges encountered at my own institution were common data management concerns across the broader clinical research community.

Identifying core competencies and building workshop content

The primary focus for developing a 1.5-hour CRDM workshop was to use the GCDMP core guidelines [37] as the baseline structure for the workshop. The core guidelines are separated into chapters in the GCDMP, which were used as the foundation for the core competencies of the workshop. Once this baseline structure was established, my goal was to weave in answers to the common questions that our clinical research community has asked through our existing REDCap training. These questions related to how to create codebooks and data dictionaries for research projects, how to structure roles in a research team, how to use best practices for building data collection instruments, how to protect their data according to Health Insurance Portability and Accountability Act (HIPAA) regulations that they should be aware of, how to improve the quality of their data throughout a study, and how to best document procedures throughout a study.

The goal of the workshop was to tie as many examples back to REDCap as possible, because the use of REDCap was written into institutional policy as the recommended tool for research data collection, which made it essential to highlight its data management capabilities. The core competencies combined with the questions mentioned above served as the foundation for developing the learning objectives and interactive learning activities for the workshop (Table 1).

Table 1 Clinical research data management workshop core competencies

The core competencies and learning objectives were designed to make the workshop as practical as possible. While the theoretical components of CRDM are important and are emphasized in the workshop, the main focus was to consistently incorporate interactive learning throughout so that attendees could both apply and contextualize what they learned to their own research. Another goal of this workshop was to encourage communication between attendees to highlight common CRDM errors and provide avenues for attendees to learn about successful and unsuccessful approaches from their peers. To this end, after each core competency was taught, the workshop was designed to have attendees discuss their own experiences.

In addition to the core competencies listed in Table 1, the overarching theme and intention applied across the workshop was the importance of maintaining good documentation throughout a clinical research project (e.g., data collection plan, roles and responsibilities documents, statistical analysis plan). By stressing the importance of documentation for each competency, I hoped that attendees would understand the value of and be able to develop their own detailed documentation at each stage of the research process. The time dedicated to developing this workshop—which included reviewing the GCDMP core competencies, outlining commonly asked questions from the research community, establishing learning objectives, building the slide deck, and creating the workshop activities—took between 80 and 100 hours to complete.

Workshop implementation

The CRDM workshop was offered broadly throughout the medical center three separate times in November 2017, January 2018, and February 2018. These workshops were promoted using our library’s email discussion list of attendees from previous data classes and the Office of Science and Research and Clinical and Translational Science Institute’s announcements emails. Direct outreach was also extended to residency directors and research coordinators, both of whom regularly attend the library’s REDCap training. A fourth workshop was offered in July 2018 as part of the library’s established Data Day to Day series [43], which the library has substantially marketed through posters, write-ups in institutional newsletters, and broadcast emails.

Workshop evaluation

The CRDM workshop evaluation consisted of both quantitative and qualitative methods using a questionnaire administered at the conclusion of each workshop (supplemental Appendix). This study was deemed exempt by our institutional review board (IRB). Using Likert scales, questions asked attendees to evaluate the difficulty level of the material presented in the workshop, their willingness to recommend the workshop to others, and their intention to use what they had learned in their work. Free-text questions asked attendees to specify how they would use what they learned in their current roles in the institution and what other course topics they would be interested in learning about. For the question that asked attendees to describe how they would use what they learned in their current roles, I hand coded responses in a spreadsheet using the emergent coding technique [44] to identify the competencies that attendees stated as the most applicable to their work.

Workshop results

Of the 145 attendees at the 4 workshops, 113 provided fully or partially completed evaluation forms. Overall registration to and attendance at all 4 workshops was very high, with substantial waitlists accumulating for each class offered (Figure 1). In fact, the workshop offered in February 2018 was a direct result of having 60 people on the waitlist from the January session. Waitlists were useful for identifying communities that I had not reached through training to date as well as for understanding the popularity of the topic for the research community. If the waitlist was high in number, it provided another opportunity to offer the workshop or reach out to attendees to see if there was an opportunity to teach a smaller class in their departments.

There was a wide range of attendees at these workshops (Figure 2), as there were no restrictions on who could attend. Project/research coordinators (n=38), faculty (n=18), and managers (n=13) were prominent attendees at the workshop, and their comments in the evaluation form reflected its value and the importance of someone from the library teaching this material.

Research coordinators and project managers specifically indicated that the CRDM workshop was helpful in multiple ways for their roles, including how to set up the organization of their data collection procedures, how to establish and clarify roles in a research team, and how to develop documentation for both data collection and the roles and responsibilities of their staff. Research coordinators also indicated that no other stakeholders in the institution taught this kind of material and that this type of training was essential for their work.

Faculty indicated that the workshop was beneficial for developing project management skills, gaining an awareness of the benefits of using REDCap to both collect and manage data, and clarifying the roles and responsibilities of statisticians on their team. They also mentioned the benefits of their study team taking a workshop of this kind at the beginning of a study.

Attendees more generally described the value of the resources presented in the workshop, specifically stating that using REDCap, locating resources for identifying relevant data collection standards, gaining awareness of institutional data storage options, and using the workshop slide deck to guide their CRDM processes were particularly helpful.

Overall, the evaluation data indicated positive results, with the majority of those who responded (94%) indicating the level of material was just right and almost all who responded stating they would recommend the class to others (99%) and would use what they learned in their work (98%). Additionally, responses from attendees who indicated how they would use what they learned and apply it to their current role helped provide additional context for the benefits of the CRDM workshop (Figure 3) with improving documentation (37%), planning work flows (34%), using REDCap (22%), and assigning roles and responsibilities (17%) being the most prominent applications of the core competencies learned.

Finally, attendees expressed interest in many additional topics that they would like to see taught in future classes. These topics included statistics, research compliance, the legal implication of data sharing, and IRB best practices for study design. It is important to mention that attendees indicated that they would like to see these additional topics taught in tandem with the CRDM workshop so that they could gain a better understanding of CRDM from the perspective of an established institutional work flow for clinical research projects.

DISCUSSION

Considering that this was the first time that I had offered CRDM training to our research community, the overall attendance, high waitlist numbers, and percentage of attendees who said the course content was at the appropriate level validated the educational approach that I used. One major concern during the workshop development phase was that the content would be too rudimentary for our research community; however, the evaluations suggested that this was not the case. Furthermore, since one of the central goals of the CRDM workshop was to emphasize the importance of documentation for each core competency, the fact that this was the most commonly cited application of what attendees learned was further validation of the CRDM workshop’s course content.

While my approach was to utilize REDCap as a resource to demonstrate good CRDM practices because it served a direct purpose for our research community, this workshop can be taught without reference to it. The core competencies of this workshop (Table 1) are based on fundamental guidelines of good CRDM practice, and these competencies and skills are applicable to any stakeholder who participates in clinical research, no matter what tool or format they decide to use to collect their data.

The positive reviews of the four broadly offered courses led to seven additional CRDM training sessions that were requested by specific departments and research teams, indicating a strong need from our research community for this material. Evaluation forms were not distributed during these seven sessions due to the consult-like nature of these requests. During these sessions, several research coordinators indicated that the CRDM workshop should be required for all clinical research teams before their studies begin. This call for additional training presents an opportunity for our library to incorporate CRDM education into existing institutional initiatives. Specifically, I identified our institutional education and training management system, residency research blocks, and principal investigator training as logical next steps for integrating CRDM education into institutional research work flows.

The evaluation data initiated the development of partnerships with other institutional stakeholders to better support clinical research training efforts. Our library has begun conversations with stakeholders from research compliance, general counsel, the IRB, the Office of Science and Research, and information technology (IT) to identify ways to better address the needs of clinical researchers. The CRDM workshop highlighted a level of uncertainty on the part of clinical researchers about how best to conduct research in the medical center and whom to contact when faced with certain questions or issues.

Subsequent discussions with the aforementioned stakeholders have emphasized a need to provide more clarity to our community about the research process. To this end, our library is leading the coordination of these groups to offer a comprehensive clinical data education series with representatives from each major department providing their own training to complement the library’s existing REDCap and CRDM workshops. This training series will likely be offered through our library’s existing “Data Day to Day” series so that the research community can take all of the classes within a short time span.

The lack of institutional clarity that attendees and the aforementioned stakeholders identified has also led to policy discussions related to data transfer, sharing, and compliance, as our current institutional procedures are unclear and poorly utilized. Through the development of new standard operating procedures and increased educational initiatives, our library is driving awareness of institutional best practices with the hopes of improving clinical research efficiency. Members from our library now sit on institutional policy working groups that are working to improve institutional data transfer and data sharing work flows.

Just as librarians at the University of Washington carved out a role for themselves in supporting clinical research efforts [45], we seized the opportunity to do the same by offering CRDM education. As the first line of defense for teaching researchers, identifying their data management issues, and hearing their concerns, our library is serving as the conduit for ensuring clinical research is conducted according to GCDM practices at our institution. Establishing partnerships with research compliance, general counsel, the Office of Science and Research, and IT provides us with additional knowledge of their institutional roles and subsequently enables us to send researchers in the right direction to receive the necessary expertise and support. As this service model develops, our library plans to monitor and assess referrals to these other departments to demonstrate the value of increasing compliance in the institution and to integrate CRDM education services into any newly developed policy (which we were successful in doing for the new institutional data storage policy and REDCap). With our library serving as the driving force behind the improvement of CRDM support, the ultimate goal is that these new partnerships will result in our research community being better trained, more compliant, and increasingly aware of established institutional work flows for clinical research

DATA AVAILABILITY STATEMENT

The workshop evaluation form, resulting data, and slide deck from the “Clinical Research Data Management” workshop are available in Figshare at DOI: http://dx.doi.org/10.6084/m9.figshare.7105817.v1.

SUPPLEMENTAL FILE

REFERENCES

1 Anderson NR, Lee ES, Brockenbrough JS, Minie ME, Fuller S, Brinkley J, Tarczy-Hornoch P. Issues in biomedical research data management and analysis: needs and barriers. J Am Med Inform Assoc. 2007 Jul–Aug;14(4):478–88.
      

2 Wang X, Williams C, Liu ZH, Croghan J. Big data management challenges in health research—a literature review. Briefings Bioinform. 2017 Aug 7.
  

3 Barone L, Williams J, Micklos D. Unmet needs for analyzing biological big data: a survey of 704 NSF principal investigators. PLoS Comput Biol. 2017 Oct 19;13(10):e1005755. Corrected: PLoS Comput Biol. 2017 Nov 13;13(11):e1005858.
      

4 Johansson B, Fogelberg-Dahm M, Wadensten B. Evidence-based practice: the importance of education and leadership. J Nurs Manag. 2010 Jan;18(1):70–7.
    

5 Federer LM, Lu YL, Joubert DJ. Data literacy training needs of biomedical researchers. J Med Libr Assoc. 2016 Jan;104(1):52–7. DOI: http://dx.doi.org/10.3163/1536-5050.104.1.008.
      

6 Scaramozzino JM, Ramírez ML, McGaughey KJ. A study of faculty data curation behaviors and attitudes at a teaching-centered university. Coll Res Libr. 2012 Jul;73(4):349–65.
  

7 Carlson J, Johnston L, Westra B, Nichols M. Developing an approach for data management education: a report from the data information literacy project. Int J Digit Curation. 2013;8(1):204–17.
  

8 Macmillan D. Developing data literacy competencies to enhance faculty collaborations. LIBER Q. 2015;24(3):140–60.
  

9 Wittenberg J, Elings M. Building a research data management service at the University of California, Berkeley: a tale of collaboration. IFLA J. 2017 Mar;43(1):89–97.
  

10 Piorun ME, Kafel D, Leger-Hornby T, Najafi S, Martin ER, Colombo P, LaPelle N. Teaching research data management: an undergraduate/graduate curriculum. J eSci Libr. 2012;1(1):8.

11 Reisner BA, Vaughan KTL, Shorish YL. Making data management accessible in the undergraduate chemistry curriculum. J Chem Educ. 2014;91(11):1943–6.
  

12 Adamick J, Reznik-Zellen RC, Sheridan M. Data management training for graduate students at a large research university. J eSci Libr. 2013;1(3):8.

13 Fransson J, Lagunas PT, Kjellberg S, Toit MD. Developing integrated research data management support in close relation to doctoral students’ research practices. Proc Assoc Inf Sci Technol. 2016;53(1):1–4.
  

14 Clement R, Blau A, Abbaspour P, Gandour-Rood E. Team-based data management instruction at small liberal arts colleges. IFLA J. 2017 Mar;43(1):105–18.
  

15 Johnston L, Jeffryes J. Steal this idea: a library instructors’ guide to educating students in data management skills. Coll Res Libr News. 2014 Sep;75(8):431–4.
  

16 Johnston L, Lafferty M, Petsan B. Training researchers on data management: a scalable, cross-disciplinary approach. J eSci Libr. 2012;1(2):2.

17 Muilenburg J, Lebow M, Rich J. Lessons learned from a research data management pilot course at an academic library. J eSci Libr. 2014;3(1):8.

18 Southall J, Scutt C. Training for research data management at the Bodleian Libraries: national contexts and local implementation for researchers and librarians. New Rev Acad Libr. 2017;23(2–3):303–22.
  

19 Tammaro AM, Casarosa V, eds. Research data management in the curriculum: an interdisciplinary approach. Procedia Computer Science; 2014.
  

20 Verbakel E, Grootveld M. ‘Essentials 4 Data Support’: five years’ experience with data management training. IFLA J. 2016 Dec;42(4):278–83.
  

21 DeBose KG, Haugen I, Miller RK. Information literacy instruction programs: supporting the college of agriculture and life sciences community at Virginia Tech. Libr Trends. 2017 Winter;65(3):316–38.
  

22 Fong BL, Wang M. Required data management training for graduate students in an earth and environmental sciences department. J eSci Libr. 2015;4(1):3.

23 Hou CY. Meeting the needs of data management training: the federation of Earth Science Information Partners (ESIP) data management for scientists short course. Issues Sci Technol Libr. 2015 Spring;2015(80).

24 Thielen J, Hess AN. Advancing research data management in the social sciences: implementing instruction for education graduate students into a doctoral curriculum. Behav Soc Sci Libr. 2018:1–15.

25 Dressel WF. Research data management instruction for digital humanities. J eSci Libr. 2017;6(2):5.

26 Bruland P, Breil B, Fritz F, Dugas M. Interoperability in clinical research: from metadata registries to semantically annotated CDISC ODM. Studies Health Technol Inform. 2012;180:564–8.

27 Gaddale JR. Clinical data acquisition standards harmonization importance and benefits in clinical data management. Perspect Clin Res. 2015 Oct–Dec;6(4):179–83.
      

28 Krishnankutty B, Bellary S, Kumar NB, Moodahadu LS. Data management in clinical research: an overview. Indian J Pharm. 2012 Mar;44(2):168–72.
  

29 Leroux H, Metke-Jimenez A, Lawley MJ. Towards achieving semantic interoperability of clinical study data with FHIR. J Biomed Semantics. 2017 Sep 19;8(1):41.
      

30 Arthofer K, Girardi D. Data quality- and master data management—a hospital case. Stud Health Technol Inform. 2017;236:259–66.

31 Callahan T, Barnard J, Helmkamp L, Maertens J, Kahn M. Reporting data quality assessment results: identifying individual and organizational barriers and solutions. EGEMS (Washington, DC). 2017 Sep 4;5(1):16.

32 Houston L, Probst Y, Yu P, Martin A. Exploring data quality management within clinical trials. Appl Clin Inform. 2018 Jan;9(1):72–81.
      

33 Teunenbroek TV, Baker J, Dijkzeul A. Towards a more effective and efficient governance and regulation of nanomaterials. Particle Fibre Toxicol. 2017 Dec 19;14(1):54.
  

34 Ohmann C, Banzi R, Canham S, Battaglia S, Matei M, Ariyo C, Becnel L, Bierer B, Bowers S, Clivio L, Dias M, Druml C, Faure H, Fenner M, Galvez J, Ghersi D, Gluud C, Groves T, Houston P, Karam G, Kalra D, Knowles RL, Krleža-Jerić K, Kubiak C, Kuchinke W, Kush R, Lukkarinen A, Marques PS, Newbigging A, O’Callaghan J, Ravaud P, Schlünder I, Shanahan D, Sitter H, Spalding D, Tudur-Smith C, van Reusel P, van Veen EB, Visser GR, Wilson J, Demotes-Mainard J. Sharing and reuse of individual participant data from clinical trials: principles and recommendations. BMJ Open. 2017 Dec 14;7(12):e018647.
      

35 Polancich S, James DH, Miltner RS, Smith GL, Moneyham L. Building DNP essential skills in clinical data management and analysis. Nurse Educ. 2018 Jan/Feb;43(1):37–41.
  

36 Sirgo G, Esteban F, Gomez J, Moreno G, Rodriguez A, Blanch L, Guardiola J7, Gracia R, De Haro L, Bodí M. Validation of the ICU-DaMa tool for automatically extracting variables for minimum dataset and quality indicators: the importance of data quality assessment. Int J Med Inform. 2018 Apr;112:166–72.
    

37 Society for Clinical Data Management. Good clinical data management practices. The Society; 2017.

38 Sylvia M, Terhaar M. An approach to clinical data management for the doctor of nursing practice curriculum. J Prof Nurs. 2014 Jan–Feb;30(1):56–62.
    

39 Read KB, LaPolla FWZ, Tolea MI, Galvin JE, Surkis A. Improving data collection, documentation, and workflow in a dementia screening study. J Med Libr Assoc. 2017 Apr;105(2):160–66. DOI: http://dx.doi.org/10.5195/jmla.2017.221.
    

40 Read K, LaPolla FWZ. A new hat for librarians: providing REDCap support to establish the library as a central data hub. J Med Libr Assoc. 2018 Jan;106(1):120–6. DOI: http://dx.doi.org/10.5195/jmla.2018.327.
      

41 Data management for clinical research [course]. Coursera, Vanderbilt University; 2016.

42 Society for Clinical Data Management. Developing data management plans [course]. The Society; 2017.

43 Surkis A, LaPolla FWZ, Contaxis N, Read KB. Data Day to Day: building a community of expertise to address data skills gaps in an academic medical center. J Med Libr Assoc. 2017 Apr;105(2):185–91. DOI: http://dx.doi.org/10.5195/jmla.2017.35.
    

44 Stuckey H. The second step in data analysis: coding qualitative research data. J Soc Health Diabetes. 2015;3(1):7–10.
  

45 Bardyn TP, Patridge EF, Moore MT, Koh JJ. Health sciences libraries advancing collaborative clinical research data management in universities. J eSci Libr. 2018;7(2):4.

(Return to Top)

Kevin B. Read, kevin.read@nyumc.org, https://orcid.org/0000-0002-7511-9036, Data Services Librarian and Data Discovery Lead, NYU Health Sciences Library, New York University School of Medicine, 577 First Avenue, New York, NY 10016

Articles in this journal are licensed under a Creative Commons Attribution 4.0 International License.

This journal is published by the University Library System of the University of Pittsburgh as part of its D-Scribe Digital Publishing Program and is cosponsored by the University of Pittsburgh Press.

Journal of the Medical Library Association, VOLUME 107, NUMBER 1, January 2019