Assessing journal author guidelines for systematic reviews and meta-analyses: findings from an institutional sample
DOI:
https://doi.org/10.5195/jmla.2022.1273Keywords:
systematic review, meta-analysis, author instructions, journal requirements, research methodology, publishingAbstract
Objective: Systematic reviews and meta-analyses (SRs/MAs) are designed to be rigorous research methodologies that synthesize information and inform practice. An increase in their publication runs parallel to quality concerns and a movement toward standards to improve reporting and methodology. With the goal of informing the guidance librarians provide to SR/MA teams, this study assesses online journal author guidelines from an institutional sample to determine whether these author guidelines address SR/MA methodological quality.
Methods: A Web of Science Core Collection (Clarivate) search identified SRs/MAs published in 2014–2019 by authors affiliated with a single institution. The AMSTAR 2 checklist was used to develop an assessment tool of closed questions specific to measures for SR/MA methodological quality in author guidelines, with questions added about author guidelines in general. Multiple reviewers completed the assessment.
Results: The author guidelines of 141 journals were evaluated. Less than 20% addressed at least one of the assessed measures specific to SR/MA methodological quality. There was wide variation in author guidelines between journals from the same publisher apart from the American Medical Association, which consistently offered in-depth author guidelines. Normalized Eigenfactor and Article Influence Scores did not indicate author guideline breadth.
Conclusions: Most author guidelines in the institutional sample did not address SR/MA methodological quality. When consulting with teams embarking on SRs/MAs, librarians should not expect author guidelines to provide details about the requirements of the target journals. Librarians should advise teams to follow established SR/MA standards, contact journal staff, and review SRs/MAs previously published in the journal.
References
Murad MH, Asi N, Alsawas M, Alahdab F. New evidence pyramid. Evid Based Med. 2016 Jun 23;21(4):125–7.
Halevi G, Pinotti R. Systematic reviews: characteristics and impact. Pub Res Q. 2020 Dec;36(4):523–37.
Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol. 1991;44(11):1271–8.
Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet. 1999 Nov 27;354(9193):1896–1900.
Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009 Jul 21;6(7):e1000097.
Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, Moher D, Tugwell P, Welch V, Kristjansson E, Henry DA. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017 Sep 21;358:j4008.
Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007 Feb 15;7:10.
Kung J, Chiappelli F, Cajulis OO, Avezova R, Kossan G, Chew L, Maida CA. From systematic reviews to clinical recommendations for evidence-based health care: validation of revised assessment of multiple systematic reviews (R-AMSTAR) for grading of clinical relevance. Open Dent J. 2010 Jul 16;4:84–91.
Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, Moher D, Becker BJ, Sipe TA, Thacker SB. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA. 2000 Apr 19;283(15):2008–12.
Stevens A, Shamseer L, Weinstein E, Yazdi F, Turner L, Thielman J, Altman DG, Hirst A, Hoey J, Palepu A, Schulz KF, Moher D. Relation of completeness of reporting of health research to journals’ endorsement of reporting guidelines: systematic review. BMJ. 2014 Jun 25;348:g3804.
Tam WWS, Lo KKH, Khalechelvam P. Endorsement of PRISMA statement and quality of systematic reviews and meta-analyses published in nursing journals: a cross-sectional study. BMJ Open. 2017 Feb 7;7(2):e013905.
Wayant C, Smith C, Sims M, Vassar M. Hematology journals do not sufficiently adhere to reporting guidelines: a systematic review. J Thromb Haemost. 2017 Feb 27;15(4):608–17.
Agha RA, Barai I, Rajmohan S, Lee S, Anwar MO, Fowler AJ, Orgill DP, Altman DG. Support for reporting guidelines in surgical journals needs improvement: a systematic review. Int J Surg. 2017 Sep;45:14–17.
Van der Pol CB, McInnes MDF, Petrcich W, Tunis AS, Hanna R. Is quality and completeness of reporting of systematic reviews and meta-analyses published in high impact radiology journals associated with citation rates? PLoS One. 2015 Mar 16;10(3):e0119892.
Jayaraman J, Nagendrababu V, Pulikkotil SJ, Innes NP. Critical appraisal of methodological quality of systematic reviews and meta-analysis in paediatric dentistry journals. Int J Paediatr Dent. 2018 Nov;28(6):548–60.
Fleming PS, Koletsi D, Seehra J, Pandis N. Systematic reviews published in higher impact clinical journals were of higher quality. J Clin Epidemiol. 2014 Jul;67(7):754–9.
Tunis AS, McInnes MDF, Hanna R, Esmail K. Association of study quality with completeness of reporting: have completeness of reporting and quality of systematic reviews and meta-analyses in major radiology journals changed since publication of the PRISMA statement? Radiology. 2013 Nov;269(2):413–26.
Croitoru DO, Huang Y, Kurdina A, Chan AW, Drucker AM. Quality of reporting in systematic reviews published in dermatology journals. Br J Dermatol. 2019 Sep 16;182(6):1469–76.
O’Donohoe TJ, Dhillon R, Bridson TL, Tee J. Reporting quality of systematic review abstracts published in leading neurosurgical journals: a research on research study. Neurosurgery. 2019 Jul 1;85(1):1–10.
Nascimento DP, Gonzalez GZ, Araujo AC, Costa LOP. Journal impact factor is associated with PRISMA endorsement, but not with the methodological quality of low back pain systematic reviews: a methodological review. Eur Spine J. 2020;29(3):462–79.
Riado Minguez D, Kowalski M, Vallve Odena M, Longin Pontzen D, Jelicic Kadic A, Jeric M, Dosenovic S, Jakus D, Vrdoljak M, Poklepovic Pericic T, Sapunar D, Puljak L. Methodological and reporting quality of systematic reviews published in the highest ranking journals in the field of pain. Anesth Analg. 2017;125(4):1348–54.
Pölkki T, Kanste O, Kääriäinen M, Elo S, Kyngäs H. The methodological quality of systematic reviews published in high-impact nursing journals: a review of the literature. J Clin Nurs. 2014 Feb;23(3-4):315–32.
Butler E, Granholm A, Aneman A. Trustworthy systematic reviews-Can journals do more? Acta Anaesthesiol Scand. 2019 Jan 24;63(4):558–9.
Systematic Reviews Filter Strategy [Internet]. [cited 2021 Jun 8]. <https://www.nlm.nih.gov/bsd/pubmed_subsets/sysreviews_strategy.html>.
University of Washington. Eigenfactor: About [Internet]. Eigenfactor.org. [cited 2021 Jun 22]. <http://eigenfactor.org/about.php>.
Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA. Cochrane Handbook for Systematic Reviews of Interventions [Internet]. Cochrane; 2021 Feb [cited 2021 Mar 30]. <http://www.training.cochrane.org/handbook>.
Higgins J, Lasserson T, Chandler J, Tovey D, Thomas J, Flemyng E, Churchill R. Methodological Expectations of Cochrane Intervention Reviews (MECIR) [Internet]. London: Cochrane; 2021 Feb [cited 2021 Mar 30]. <https://community.cochrane.org/mecir-manual>.
Joanna Briggs Institute. Checklist for systematic reviews and research syntheses: critical appraisal tools for use in JBI systematic reviews [Internet]. Joanna Briggs Institute; 2017 [cited 2021 Mar 27]. <https://joannabriggs.org/sites/default/files/2019-06/JBI_Critical_Appraisal-Checklist_for_Systematic_Reviews2017.docx>.
Whiting P, Savović J, Higgins JPT, Caldwell DM, Reeves BC, Shea B, Davies P, Kleijnen J, Churchill R, ROBIS group. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016 Jan;69:225–34.
Rethlefsen ML, Farrell AM, Osterhaus Trzasko LC, Brigham TJ. Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. J Clin Epidemiol. 2015 Jun;68(6):617–26.
Committee on Publication Ethics (COPE), Directory of Open Access Journals (DOAJ), Open Access Scholarly Publishers Association (OASPA), World Association of Medical Editors (WAME). Principles of transparency and best practice in scholarly publishing [Internet]. 2018 [cited 2021 Mar 16]. <http://wame.org/principles-of-transparency-and-best-practice-in-scholarly-publishing>.
International Committee of Medical Journal Editors. Recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals. International Committee of Medical Journal Editors; 2019 Dec. <http://www.icmje.org/recommendations/>
Tao K, Li X, Zhou Q, Moher D, Ling C, Yu W. From QUOROM to PRISMA: a survey of high-impact medical journals’ instructions to authors and a review of systematic reviews in anesthesia literature. PLoS One. 2011 Nov 16;6(11):e27611.
Toews I, Binder N, Wolff RF, Toprak G, von Elm E, Meerpohl JJ. Guidance in author instructions of hematology and oncology journals: a cross sectional and longitudinal study. PLoS One. 2017 Apr 28;12(4):e0176489.
Checketts JX, Sims MT, Detweiler B, Middlemist K, Jones J, Vassar M. An evaluation of reporting guidelines and clinical trial registry requirements among orthopaedic surgery journals. J Bone Joint Surg Am. 2018 Feb 7;100(3):e15.
Sims MT, Checketts JX, Wayant C, Vassar M. Requirements for trial registration and adherence to reporting guidelines in critical care journals: a meta-epidemiological study of journals’ instructions for authors. Int J Evid Based Healthc. 2018 Mar;16(1):55–65.
Sims MT, Bowers AM, Fernan JM, Dormire KD, Herrington JM, Vassar M. Trial registration and adherence to reporting guidelines in cardiovascular journals. Heart. 2018;104(9):753–9.
Checketts JX, Cook C, Imani S, Duckett L, Vassar M. An evaluation of reporting guidelines and clinical trial registry requirements among plastic surgery journals. Ann Plast Surg. 2018;81(2):215–9.
Sims MT, Henning NM, Wayant CC, Vassar M. Do emergency medicine journals promote trial registration and adherence to reporting guidelines? A survey of “Instructions for Authors.” Scand J Trauma Resusc Emerg Med. 2016 Nov 24;24(1):137.
Published
Issue
Section
License
Copyright (c) 2022 Johanna Goldberg, Lindsay M. Boyce, Celine Soudant, Kendra Godwin
This work is licensed under a Creative Commons Attribution 4.0 International License.
