Implementing and assessing a service to demonstrate public impact of faculty research in news and policy sources
DOI:
https://doi.org/10.5195/jmla.2019.709Keywords:
Research Impact, Bibliometrics, AltmetricsAbstract
Background: As the need to demonstrate research impact increases, faculty are looking for new ways to show funders, departments, and institutions that their work is making a difference. While traditional metrics such as citation counts can tell one part of this story, these metrics are focused on the academic sphere and often miss the wide-ranging public impact that research can have in areas such as the news or policy documents.
Case Presentation: This case report describes how one library piloted and established the Policy & News Media Impact Service, where librarians generate reports for faculty members of the University of Minnesota Academic Health Center that tracks citations of their research in governmental and organizational policies as well as local, national, and international news media. Workflows of, resources used in, and faculty feedback on the service are described.
Conclusions: This Policy & News Media Impact Service pilot was successful and resulted in the establishment of a permanent service that is available to all departments in the Academic Health Center. Faculty feedback indicated that the service was valuable in demonstrating the public impact of their research.
This article has been approved for the Medical Library Association’s Independent Reading Program.
References
UK Research and Innovation. Pathways to impact [Internet]. Swindon, UK: UK Research and Innovation [cited 23 Mar 2019]. <https://www.ukri.org/innovation/excellence-with-impact/pathways-to-impact/>.
National Science Foundation. Chapter III - NSF proposal processing and review. In: Proposal & award policies & procedures guide [Internet]. Arlington, VA: The Foundation; 2018 [cited 23 Mar 2019]. <https://www.nsf.gov/pubs/policydocs/pappg18_1/pappg_3.jsp#IIIA2b>.
Jaques H. Get your research reported well in the news. BMJ. 2011 Jan 19;342:c6921. DOI: http://dx.doi.org/10.1136/bmj.c6921.
Brownell KD, Roberto CA. Strategic science with policy impact. Lancet. 2015 Jun 20;385(9986):2445–6. DOI: http://dx.doi.org/10.1016/S0140-6736(14)62397-7.
Greenhalgh T, Raftery J, Hanney S, Glover M. Research impact: a narrative review. BMC Med. 2016 May 23;14(1):78. DOI: http://dx.doi.org/10.1186/s12916-016-0620-8.
Fredriksson M, Tiainen A, Hanning M. Regional media coverage influences the public’s negative attitudes to policy implementation success in Sweden. Heal Expect. 2015 Dec;18(6)2731–41.
Gafson AR, Giovannoni G. CCSVI-A. a call to clinicians and scientists to vocalise in an Internet age. Mult Scler Relat Disord. 2014 Mar;3(2):143–6.
Almomani B, Hawwa AF, Goodfellow NA, Millership JS, McElnay JC. Pharmacogenetics and the print media: what is the public told? BMC Med Genet. 2015 May 9;16(1):32. DOI: http://dx.doi.org/10.1186/s12881-015-0172-3.
Previte J, Gurrieri L. Who is the biggest loser? fat news coverage is a barrier to healthy lifestyle promotion. Health Mark Q. 2015;32(4):330–49. DOI: http://dx.doi.org/10.1080/07359683.2015.1093881.
MacLean A, Sweeting H, Walker L, Patterson C, Räisänen U, Hunt K. “It’s not healthy and it’s decidedly not masculine”: a media analysis of UK newspaper representations of eating disorders in males. BMJ Open. 2015 May 29;5(5):e007468. DOI: http://dx.doi.org/10.1136/bmjopen-2014-007468.
Thrasher JF, Kim SH, Rose I, Craft MK. Media coverage of smoke-free policies after their innovation. J Health Commun. 2015;20(3):297–305. DOI: http://dx.doi.org/10.1080/10810730.2014.925017.
Dunn J, Koo J. Social media impact factor: the top ten dermatology journals on Facebook and Twitter. Dermatol Online J. 2014 Apr 16;20(4):1–19. (Available from: <http://escholarship.org/uc/item/4rb3w037>. [cited 26 Jul 2019].)
Roberts J. Measuring the social media impact of your headache article. Headache. 2014 Oct;54(9):1435–6.
Graham K, Chorzempa H, Valentine P, Magnan P. Evaluating health research impact: development and implementation of the Alberta Innovates - Health Solutions impact framework. Res Eval. 2012 Dec;21(5):354–67. DOI: http://dx.doi.org/10.1093/reseval/rvs027.
Rivera S, Kyte DG, Aiyegbusi OL, Calvert MJ. Assessing the impact of healthcare research: a systematic review of methodological frameworks. PLOS Med. 2017;14(8):e1002370. DOI: http://dx.doi.org/10.1371/journal.pmed.1002370.
Gutzman KE, Bales ME, Belter CW, Chambers T, Chan L, Holmes KL, Lu YL, Palmer LA, Reznik-Zellen RC, Sarli CC, Suiter AM, Wheeler TR. Research evaluation support services in biomedical libraries. J Med Libr Assoc. 2018 Jan;106(1):1–14. DOI: http://dx.doi.org/10.5195/jmla.2018.205.
Bakker C, McBurney J. Big splashes & ripple effects: research impact services across university libraries [Internet]. Presented at: American Library Association Conference; 2017. (Available from: <https://conservancy.umn.edu/handle/11299/188632>. [cited 26 Jul 2019].)
Sarli CC, Dubinsky EK, Holmes KL. Beyond citation analysis: a model for assessment of research impact. J Med Libr Assoc. 2010 Jan;98(1):17–23. DOI: http://dx.doi.org/10.3163/1536-5050.98.1.008.
Lewis R, Sarli CC, Suiter AM. SPEC kit 346: scholarly output assessment activities [Internet]. Washington, DC: Association of Research Libraries; 2015 [cited 26 Jul 2019]. <https://doi.org/10.29242/spec.346>.
Grant J. The nature, scale and beneficiaries of research impact: an initial analysis of Research Excellence Framework (REF) 2014 impact case studies [Internet]. 2015 [cited 26 Jul 2019]. <https://dera.ioe.ac.uk//22540/>.
Braun S. Manifold: a custom analytics platform to visualize research impact. Code4Lib [Internet]. 2015 Oct;30:10948. <http://journal.code4lib.org/articles/10948>.
Bakker C. Contextualizing impact: the University of Minnesota’s Manifold Initiative. Presented at: 2016 Bibliometrics and Research Assessment Symposium; Bethesda, MD; 2016.
Bakker C, Chew K. Evaluative bibliometrics meets the CTSI [Internet]. Presented at: MLA ’16, Medical Library Association 116th Annual Meeting; Toronto, ON, Canada; May 2016 [cited 23 Mar 2019]. <http://hdl.handle.net/11299/180871>.
Chew K, Bakker C. Adventures in bibliometrics: research impact and the CTSI [Internet]. Presented at: Midwest Chapter and Midcontinental Chapter of the Medical Library Association Annual Meeting; Des Moines, IA; Oct 2016 [cited 23 Mar 2019]. <http://hdl.handle.net/11299/194375>.
Sa CM, Kretz A, Sigurdson K. Accountability, performance assessment, and evaluation: policy pressures and responses from research councils. Res Eval. 2013 Jun;22(2):105–17. DOI: http://dx.doi.org/10.1093/reseval/rvs041.
Terama E, Smallman M, Lock S, Johnson C, Austwick MZ. Beyond academia – interrogating research impact in the research excellence framework. PLOS ONE. 2016;11(12):1–18. DOI: http://dx.doi.org/10.1371/journal.pone.0168533. Correction: 21 Feb 2017: Terama E, Smallman M, Lock SJ, Johnson C, Austwick MZ. Correction: beyond academia – interrogating research impact in the Research Excellence Framework. PLOS ONE. 2017;12(2):e0172817. DOI: http://dx.doi.org/10.1371/journal.pone.0172817.
Chubb J, Reed M. The politics of research impact: academic perceptions of the implications for research funding, motivation and quality. Br Politics. 2018 Sep;13(3):295–311. DOI: http://dx.doi.org/10.1057/s41293-018-0077-9.
Derrick GE, Haynes A, Chapman S, Hall WD. The association between four citation metrics and peer rankings of research influence of Australian researchers in six public health fields. PLOS ONE. 2011;6(4):e18521. DOI: http://dx.doi.org/10.1371/journal.pone.0018521.
Brownson RC, Eyler AA, Harris JK, Moore JB, Tabak RG. Getting the word out: new approaches for disseminating public health science. J Public Health Manag Pract. 2018 Mar;24(2):102–11. DOI: http://dx.doi.org/10.1097/PHH.0000000000000673.
Yu H. Context of altmetrics data matters: an investigation of count type and user category. Scientometrics. 2017 Apr;111(1):267–83. DOI: http://dx.doi.org/10.1007/s11192-017-2251-z.
White HD. Citation analysis and discourse analysis revisited. Appl Linguist. 2004 Mar;25(1):89–116.
Timilsina M, Khawaja W, Davis B, Taylor M, Hayes C. Social impact assessment of scientist from mainstream news and weblogs. Soc Netw Anal Min. 2017 Dec;7:48. DOI: http://dx.doi.org/10.1007/s13278-017-0466-x.
Bakker C, Chew K, McBurney J, Reed D, Aho M. Measuring impact with altmetrics: is there one tool to rule them all? [Internet]. Presented at: Midwest Chapter of Medical Library Association Annual Meeting; Cleveland, OH; Oct 2018 [cited 23 Mar 2019]. <http://hdl.handle.net/11299/200727>.
Stern N. Building on success and learning from experience: an independent review of the Research Excellence Framework [Internet]. UK: Crown; 2016 [cited 26 Jul 2019]. <https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/541338/ind-16-9-ref-stern-review.pdf>.
