Effect of librarian collaboration on otolaryngology systematic review and meta-analysis quality

Rachel Whitney; Michael C. Shih; Tamar Gordis; Shaun A. Nguyen; Ted A. Meyer; Emily A. Brennan

doi:10.5195/jmla.2024.1774

Authors

Rachel Whitney, AHIP Medical University of South Carolina Libraries https://orcid.org/0000-0002-9648-055X
Michael C. Shih Department of Internal Medicine, Tulane University https://orcid.org/0000-0002-2972-637X
Tamar Gordis Department of Otolaryngology – Head & Neck Surgery, Medical University of South Carolina https://orcid.org/0000-0002-1573-8257
Shaun A. Nguyen Department of Otolaryngology – Head & Neck Surgery, Medical University of South Carolina https://orcid.org/0000-0003-0664-4571
Ted A. Meyer Department of Otolaryngology – Head & Neck Surgery, Medical University of South Carolina
Emily A. Brennan Medical University of South Carolina https://orcid.org/0000-0001-8463-1499

DOI:

https://doi.org/10.5195/jmla.2024.1774

Keywords:

systematic reviews, meta-analyses, otolaryngology, librarians, reproducibility

Abstract

Objective: To determine if librarian collaboration was associated with improved database search quality, search reproducibility, and systematic review reporting in otolaryngology systematic reviews and meta-analyses.

Methods: In this retrospective cross-sectional study, PubMed was queried for systematic reviews and meta-analyses published in otolaryngology journals in 2010, 2015, and 2021. Two researchers independently extracted data. Two librarians independently rated search strategy reproducibility and quality for each article. The main outcomes include association of librarian involvement with study reporting quality, search quality, and publication metrics in otolaryngology systematic reviews and meta-analyses. Categorical data were compared with Chi-Squared tests or Fisher’s Exact tests. Continuous variables were compared via Mann Whitney U Tests for two groups, and Kruskal-Wallis Tests for three or more groups.

Results: Of 559 articles retrieved, 505 were analyzed. More studies indicated librarian involvement in 2021 (n=72, 20.7%) compared to 2015 (n=14, 10.4%) and 2010 (n=2, 9.0%) (p=0.04). 2021 studies showed improvements in properly using a reporting tool (p<0.001), number of databases queried (p<0.001), describing date of database searches (p<0.001), and including a flow diagram (p<0.001). Librarian involvement was associated with using reporting tools (p<0.001), increased number of databases queried (p<0.001), describing date of database search (p=0.002), mentioning search peer reviewer (p=0.02), and reproducibility of search strategies (p<0.001). For search strategy quality, librarian involvement was associated with greater use of “Boolean & proximity operators” (p=0.004), “subject headings” (p<0.001), “text word searching” (p<0.001), and “spelling/syntax/line numbers” (p<0.001). Studies with librarian involvement were associated with publication in journals with higher impact factors for 2015 (p=0.003) and 2021 (p<0.001).

Conclusion: Librarian involvement was associated with improved reporting quality and search strategy quality. Our study supports the inclusion of librarians in review teams, and journal editing and peer reviewing teams.

References

Schellinger J, Sewell K, Bloss JE, Ebron T, Forbes C. The effect of librarian involvement on the quality of systematic reviews in dental medicine. PLoS One. 2021 16(9):e0256833. DOI:10.1371/journal.pone.0256833.

Faggion CM, Jr., Huivin R, Aranda L, Pandis N, Alarcon M. The search and selection for primary studies in systematic reviews published in dental journals indexed in medline was not fully reproducible. J Clin Epidemiol. 2018 Jun;98:53-61. DOI:10.1016/j.jclinepi.2018.02.011.

Pussegoda K, Turner L, Garritty C, Mayhew A, Skidmore B, Stevens A, Boutron I, Sarkis-Onofre R, Bjerre LM, Hróbjartsson A, Altman DG, Moher D. Identifying approaches for assessing methodological and reporting quality of systematic reviews: A descriptive study. Syst Rev. 2017 Jun 19;6(1):117. DOI:10.1186/s13643-017-0507-6.

Golder S, Loke Y, McIntosh HM. Poor reporting and inadequate searches were apparent in systematic reviews of adverse effects. J Clin Epidemiol. 2008 May;61(5):440-8. DOI:10.1016/j.jclinepi.2007.06.005.

McKeown S, Ross-White A. Building capacity for librarian support and addressing collaboration challenges by formalizing library systematic review services. J Med Libr Assoc. 2019 Jul;107(3):411-9. DOI:10.5195/jmla.2019.443.

Koffel JB. Use of recommended search strategies in systematic reviews and the impact of librarian involvement: A cross-sectional survey of recent authors. PLoS One. 2015 10(5):e0125931. DOI:10.1371/journal.pone.0125931.

Spencer AJ, Eldredge JD. Roles for librarians in systematic reviews: A scoping review. J Med Libr Assoc. 2018 Jan;106(1):46-56. DOI:10.5195/jmla.2018.82.

Bramer WM, Rethlefsen ML, Mast F, Kleijnen J. Evaluation of a new method for librarian-mediated literature searches for systematic reviews. Res Synth Methods. 2018 Dec;9(4):510-20. DOI:10.1002/jrsm.1279.

Ross-White A. Search is a verb: Systematic review searching as invisible labor. J Med Libr Assoc. 2021 Jul 1;109(3):505-6. DOI:10.5195/jmla.2021.1226.

Institute of Medicine. Finding what works in health care: Standards for systematic reviews. Jill Eden, Laura Levit, Alfred Berg, and Sally Morton, editors. Washington, DC: The National Academies Press; 2011. 340 p. ISBN 978-0-309-16425-2.

Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VAe. Cochrane handbook for systematic reviews of interventions (version 6.3): Cochrane; 2022. Available from: www.training.cochrane.org/handbook.

Holmes HN, Brennan EA. MLAConnect [Internet]: Medical Library Association (MLA). 2022. Available from: https://www.mlanet.org/blog/advocating-for-authorship-librarians-and-information-professionals-as-authors-on-evidence-synthesis-publications.

Rethlefsen ML, Farrell AM, Osterhaus Trzasko LC, Brigham TJ. Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. J Clin Epidemiol. 2015 Jun;68(6):617-26. DOI:10.1016/j.jclinepi.2014.11.025.

Meert D, Torabi N, Costella J. Impact of librarians on reporting of the literature searching component of pediatric systematic reviews. J Med Libr Assoc. 2016 Oct;104(4):267-77. DOI:10.3163/1536-5050.104.4.004.

Aamodt M, Huurdeman H, Strømme H. Librarian co-authored systematic reviews are associated with lower risk of bias compared to systematic reviews with acknowledgement of librarians or no participation by librarians. Evidence Based Library and Information Practice. 2019 Dec 13;14(4):103-27. DOI:10.18438/eblip29601.

Koffel JB, Rethlefsen ML. Reproducibility of search strategies is poor in systematic reviews published in high-impact pediatrics, cardiology and surgery journals: A cross-sectional study. PLoS One. 2016 11(9):e0163309. DOI:10.1371/journal.pone.0163309.

Journal Citation Reports. Otorhinolaryngology – science category [Internet]. Clarivate Analytics. 2022.

National Library of Medicine. Journal selection for medline [Internet]. [rev. 6 June 2023; cited 28 Nov 2023]. https://www.nlm.nih.gov/medline/medline_journal_selection.html.

Covidence. Veritas Health Innovation, Melbourne, Australia; https://www.covidence.org.

McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. Press peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016 Jul;75:40-6. DOI:10.1016/j.jclinepi.2016.01.021.

Clark LA, Watson D. Constructing validity: New developments in creating objective measuring instruments. Psychol Assess. 2019 Dec;31(12):1412-27. DOI:10.1037/pas0000626.

Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977 33(1):159-74. DOI:10.2307/2529310.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, Chou R, Glanville J, Grimshaw JM, Hróbjartsson A, Lalu MM, Li T, Loder EW, Mayo-Wilson E, McDonald S, McGuinness LA, Stewart LA, Thomas J, Tricco AC, Welch VA, Whiting P, Moher D. The prisma 2020 statement: An updated guideline for reporting systematic reviews. BMJ. 2021 Mar 29;372:n71. DOI:10.1136/bmj.n71.

Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: The prisma statement. BMJ. 2009 Jul 21;339:b2535. DOI:10.1136/bmj.b2535.

Rethlefsen ML, Kirtley S, Waffenschmidt S, Ayala AP, Moher D, Page MJ, Koffel JB, Group P-S. Prisma-s: An extension to the prisma statement for reporting literature searches in systematic reviews. Syst Rev. 2021 Jan 26;10(1):39. DOI:10.1186/s13643-020-01542-z.

Grossetta Nardini HK, Batten J, Funaro MC, Garcia-Milian R, Nyhan K, Spak JM, Wang L, Glover JG. Librarians as methodological peer reviewers for systematic reviews: Results of an online survey. Res Integr Peer Rev. 2019 Nov 27;4:23. DOI:10.1186/s41073-019-0083-5.

Nyhan K, Haugh D, Grossetta Nardini H, Rethlefsen M. Librarian peer reviewer database [Internet]. [cited 13 Dec 2023]. https://sites.google.com/view/mlprdatabase/home/about-this-site.

Iverson S, Seta MD, Lefebvre C, Ritchie A, Traditi L, Baliozian K. International health library associations urge the icmje to seek information specialists as peer reviewers for knowledge synthesis publications. J Med Libr Assoc. 2020 109(3):503-4. DOI:10.5195/jmla.2021.1301.

Effect of librarian collaboration on otolaryngology systematic review and meta-analysis quality

Authors

DOI:

Keywords:

Abstract

References

Downloads

Additional Files

Published

Issue

Section

License

Current Issue