How ChatGPT writes scientific titles in medical research: structural and content differences compared to human authors
DOI:
https://doi.org/10.5195/jmla.2026.2266Keywords:
article title, artificial intelligence, AI, ChatGPT, language model, medical publishing, scientific writingAbstract
Objective: Scientific article titles play a central role in shaping research visibility, interpretation, and discoverability. With the rise of large language models like ChatGPT, there is growing interest in using AI tools to support title generation, yet little is known about how AI-generated titles differ from those written by human authors. This study compared titles written by original authors to those generated by ChatGPT-4.0 from the same abstracts, focusing on structural and content-level differences.
Methods: Fifty research articles published in 2000, before the advent of generative AI, were randomly selected from ten high-impact general internal medicine journals. For each, the structured abstract was submitted to ChatGPT-4.0 using a standardized prompt to generate a title. Human-written and AI-generated titles were then compared using quantitative measures (word and character counts, punctuation marks) and descriptive content analysis (study design, population descriptors, outcome emphasis, public health or clinical framing, temporal context). Paired statistical tests were applied to assess differences.
Results: ChatGPT-generated titles were significantly longer than human-written titles (median 16 vs. 12.5 words, p-value<0.001) and included more characters and punctuation marks (colons in 100% vs. 30%; p-value<0.001). AI titles more often specified or clarified study design, detailed populations, emphasized outcomes, framed findings in public health or clinical terms, and incorporated temporal context.
Conclusion: ChatGPT-4.0 produces more explicit and structured titles than human authors, emphasizing methodological clarity and content completeness. These findings raise important questions about norms in scientific communication and the need for further research and ethical guidance on AI-assisted writing.
References
1. Jacques TS, Sebire NJ. The Impact of Article Titles on Citation Hits: An Analysis of General and Specialist Medical Journals. JRSM Short Rep SAGE Publications; 2010 June 1;1(1):1–5. DOI: 10.1258/shorts.2009.100020.
2. Paiva CE, Lima JP da SN, Paiva BSR. Articles with short titles describing the results are cited more often. Clinics 2012 May 1;67(5):509–513. DOI: 10.6061/clinics/2012(05)17.
3. Subotic S, Mukherjee B. Short and amusing: The relationship between title characteristics, downloads, and citations in psychology articles. J Inf Sci SAGE Publications Ltd; 2014 Feb 1;40(1):115–124. DOI: 10.1177/0165551513511393.
4. Jamali HR, Nikzad M. Article title type and its relation with the number of downloads and citations. Scientometrics 2011 Aug 1;88(2):653–661. DOI: 10.1007/s11192-011-0412-z.
5. Nair LB, Gibbert M. What makes a ‘good’ title and (how) does it matter for citations? A review and general model of article title attributes in management science. Scientometrics 2016 June 1;107(3):1331–1359. DOI: 10.1007/s11192-016-1937-y.
6. Harmon JE, Gross AG. The Structure of Scientific Titles. J Tech Writ Commun SAGE Publications Inc; 2009 Oct 1;39(4):455–465. DOI: 10.2190/TW.39.4.g.
7. Guo F, Ma C, Shi Q, Zong Q. Succinct effect or informative effect: the relationship between title length and the number of citations. Scientometrics 2018 Sept 1;116(3):1531–1539. DOI: 10.1007/s11192-018-2805-8.
8. Braticevic MN, Babic I, Abramovic I, Jokic A, Horvat M. Title does matter: a cross-sectional study of 30 journals in the Medical Laboratory Technology category. Biochem Medica Croatian Society of Medical Biochemistry and Laboratory Medicine; 2020 Feb 15;30(1):0–0. DOI: 10.11613/BM.2020.010708.
9. Hartley J. To Attract or to Inform: What are Titles for? J Tech Writ Commun SAGE Publications Inc; 2005 Apr 1;35(2):203–213. DOI: 10.2190/NV6E-FN3N-7NGN-TWQT.
10. Yitzhaki M. Variation in informativity of titles of research papers in selected humanities journals: A comparative study. Scientometrics Akadémiai Kiadó, co-published with Springer Science+Business Media B.V., Formerly Kluwer Academic Publishers B.V.; 2006 July 26;38(2):219–229. DOI: 10.1007/bf02457410.
11. Medical Writing 4th Edition. Camb Univ Press Assess. Available from: https://www.cambridge.org/universitypress/subjects/medicine/medicine-general-interest/medical-writing-prescription-clarity-4th-edition [accessed June 7, 2025].
12. Haggan M. Research paper titles in literature, linguistics and science: dimensions of attraction. J Pragmat 2004 Feb 1;36(2):293–317. DOI: 10.1016/S0378-2166(03)00090-0.
13. Habibzadeh F, Yadollahie M. Are shorter article titles more attractive for citations? Cross-sectional study of 22 scientific journals. Croat Med J 2010 Apr;51(2):165–170. PMID:20401960.
14. Salvagno M, Taccone FS, Gerli AG. Can artificial intelligence help for scientific writing? Crit Care 2023 Feb 25;27(1):75. DOI: 10.1186/s13054-023-04380-2.
15. Korinek A. Language Models and Cognitive Automation for Economic Research. National Bureau of Economic Research; 2023. DOI: 10.3386/w30957.
16. Stokel-Walker C, Van Noorden R. What ChatGPT and generative AI mean for science. Nature 2023 Feb;614(7947):214–216. PMID:36747115.
17. Lund B, Ting W. Chatting about ChatGPT: How May AI and GPT Impact Academia and Libraries? Rochester, NY: Social Science Research Network; 2023. DOI: 10.2139/ssrn.4333415.
18. Gao CA, Howard FM, Markov NS, Dyer EC, Ramesh S, Luo Y, Pearson AT. Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers. NPJ Digit Med 2023 Apr 26;6(1):75. PMID:37100871.
19. Else H. Abstracts written by ChatGPT fool scientists. Nature 2023 Jan;613(7944):423. PMID:36635510.
20. Biswas S. ChatGPT and the Future of Medical Writing. Radiology 2023 Apr;307(2):e223312. PMID:36728748.
21. Sebo P. Can ChatGPT Recognize Its Own Writing in Scientific Abstracts? Cureus 2025 July;17(7):e88774. PMID:40861680.
22. Hosseini M, Horbach SPJM. Fighting reviewer fatigue or amplifying bias? Considerations and recommendations for use of ChatGPT and other Large Language Models in scholarly peer review. Res Sq 2023 Feb 20;rs.3.rs-2587766. PMID:36865238.
23. Robertson Z. GPT4 is Slightly Helpful for Peer-Review Assistance: A Pilot Study. arXiv; 2023. DOI: 10.48550/arXiv.2307.05492.
24. Stokel-Walker C, Van Noorden R. What ChatGPT and generative AI mean for science. Nature 2023 Feb;614(7947):214–216. PMID:36747115.
25. Thorp HH. ChatGPT is fun, but not an author. Science 2023 Jan 27;379(6630):313. PMID:36701446.
26. Sebo P. How accurate are the references generated by ChatGPT in internal medicine? Intern Emerg Med 2024 Jan;19(1):247–249. PMID:37979017.
27. Chen Y, Eger S. Transformers Go for the LOLs: Generating (Humourous) Titles from Scientific Abstracts End-to-End. arXiv; 2023. DOI: 10.48550/arXiv.2212.10522.
28. Rehman T, Sanyal DK, Chattopadhyay S. Can pre-trained language models generate titles for research papers? arXiv; 2024. DOI: 10.48550/arXiv.2409.14602.
29. Krippendorff K. Content analysis: an introduction to its methodology. Fourth edition. Los Angeles London New Delhi Singapore Washington DC Melbourne: SAGE; 2019. ISBN:978-1-5063-9566-1.
30. Neuendorf KA. The content analysis guidebook. Second edition. Los Angeles London New Delhi Singapore Washington, DC Melbourne: SAGE; 2017. ISBN:978-1-4129-7947-4.
31. Schilhan L, Kaier C, Lackner K. Increasing visibility and discoverability of scholarly publications with academic search engine optimization. Insights 2021; Available from: http://unipub.uni-graz.at/obvugrfodok/5992511 [accessed Nov 11, 2025].
32. Pottier P, Lagisz M, Burke S, Drobniak SM, Downing PA, Macartney EL, Martinig AR, Mizuno A, Morrison K, Pollo P, Ricolfi L, Tam J, Williams C, Yang Y, Nakagawa S. Title, abstract and keywords: a practical guide to maximize the visibility and impact of academic papers. Proc R Soc B Biol Sci Royal Society; 2024 July 31;291(2027):20241222. DOI: 10.1098/rspb.2024.1222.
33. Busse C, August E. How to Write and Publish a Research Paper for a Peer-Reviewed Journal. J Cancer Educ Off J Am Assoc Cancer Educ 2021 Oct;36(5):909–913. PMID:32356250.
34. Lang TA. Writing a better research article. J Public Health Emerg AME Publishing Company; 2017 Dec 18;1(12). DOI: 10.21037/jphe.2017.11.06.
35. Milojević S. The Length and Semantic Structure of Article Titles—Evolving Disciplinary Practices and Correlations with Impact. Front Res Metr Anal Frontiers; 2017 Apr 6;2. DOI: 10.3389/frma.2017.00002.
36. Ecarnot F, Seronde M-F, Chopard R, Schiele F, Meneveau N. Writing a scientific article: A step-by-step guide for beginners. Eur Geriatr Med 2015 Dec 1;6(6):573–579. DOI: 10.1016/j.eurger.2015.08.005.
37. Busse CE, Anderson EW, Endale T, Smith YR, Kaniecki M, Shannon C, August ET. Strengthening research capacity: a systematic review of manuscript writing and publishing interventions for researchers in low-income and middle-income countries. BMJ Glob Health BMJ Specialist Journals; 2022 Feb 1;7(2):e008059. PMID:35165096.
38. Rust P, Frings J, Meister S, Fehring L. Evaluation of a large language model to simplify discharge summaries and provide cardiological lifestyle recommendations. Commun Med Nature Publishing Group; 2025 May 29;5(1):208. DOI: 10.1038/s43856-025-00927-2.
39. Roster K, Kann RB, Farabi B, Gronbeck C, Brownstone N, Lipner SR. Readability and Health Literacy Scores for ChatGPT-Generated Dermatology Public Education Materials: Cross-Sectional Analysis of Sunscreen and Melanoma Questions. JMIR Dermatol 2024 Mar 6;7(1):e50163. DOI: 10.2196/50163.
40. Patel EA, Fleischer L, Filip P, Eggerstedt M, Hutz M, Michaelides E, Batra PS, Tajudeen BA. The Use of Artificial Intelligence to Improve Readability of Otolaryngology Patient Education Materials. Otolaryngol--Head Neck Surg Off J Am Acad Otolaryngol-Head Neck Surg 2024 Aug;171(2):603–608. PMID:38751109.
41. Luo X, Chen F, Zhu D, Wang L, Wang Z, Liu H, Lyu M, Wang Y, Wang Q, Chen Y. Potential Roles of Large Language Models in the Production of Systematic Reviews and Meta-Analyses. J Med Internet Res 2024 June 25;26(1):e56780. DOI: 10.2196/56780.
42. Wu S, Ma X, Luo D, Li L, Shi X, Chang X, Lin X, Luo R, Pei C, Du C, Zhao Z-J, Gong J. Automated literature research and review-generation method based on large language models. Natl Sci Rev 2025 June 1;12(6):nwaf169. DOI: 10.1093/nsr/nwaf169.
43. Robinson K, Bontekoe K, Muellenbach J. Integrating PICO principles into generative artificial intelligence prompt engineering to enhance information retrieval for medical librarians. J Med Libr Assoc JMLA 2025 Apr 18;113(2):184–188. PMID:40342302.
44. Eriksen MB, Frandsen TF. The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: a systematic review. J Med Libr Assoc JMLA 2018 Oct;106(4):420–431. PMID:30271283.
45. Manning CD, Raghavan P, Schütze H. Introduction to information retrieval. Cambridge: Cambridge university press; 2008. ISBN:978-0-521-86571-5.
46. Hosseini M, Rasmussen LM, Resnik DB. Using AI to write scholarly publications. Account Res 2024 Oct;31(7):715–723. PMID:36697395.
47. Science journals set new authorship guidelines for AI-generated text. Natl Inst Environ Health Sci. Available from: https://factor.niehs.nih.gov/2023/3/feature/2-artificial-intelligence-ethics [accessed June 13, 2025].
48. Resnik DB, and Hosseini M. Disclosing artificial intelligence use in scientific research and publication: When should disclosure be mandatory, optional, or unnecessary? Account Res Taylor & Francis; 0(0):1–13. PMID:40126451.
49. Lund BD, Naheem K t. Can ChatGPT be an author? A study of artificial intelligence authorship policies in top academic journals. Learn Publ 2024;37(1):13–21. DOI: 10.1002/leap.1582.
50. Lund B, Ting W, Mannuru NR, Nie B, Shimray S, Wang Z. ChatGPT and a New Academic Reality: Artificial Intelligence-Written Research Papers and the Ethics of the Large Language Models in Scholarly Publishing. Rochester, NY: Social Science Research Network; 2023. DOI: 10.2139/ssrn.4389887.
51. Letchford A, Moat HS, Preis T. The advantage of short paper titles. R Soc Open Sci Royal Society; 2015 Aug;2(8):150266. DOI: 10.1098/rsos.150266.
52. Rostami F, Mohammadpoorasl A, Hajizadeh M. The effect of characteristics of title on citation rates of articles. Scientometrics 2014 Mar 1;98(3):2007–2010. DOI: 10.1007/s11192-013-1118-1.
53. Del Giglio A, Pereira da Costa MU. The use of artificial intelligence to improve the scientific writing of non-native English speakers. Rev Assoc Med Bras (1992). 2023;69(9):e20230560. DOI: 10.1590/1806-9282.20230560.
54. Schilhan L, Kaier C, Lackner K. Increasing visibility and discoverability of scholarly publications with academic search engine optimization. Insights. 2021;34:6. DOI: 10.1629/uksg.534.
55. International Committee of Medical Journal Editors. Uniform requirements for manuscripts submitted to biomedical journals. N Engl J Med 1997 Jan 23;336(4):309–315. PMID:8995096.
56. Uniform requirements for manuscripts submitted to biomedical journals. International Committee of Medical Journal Editors. BMJ 1991 Feb 9;302(6772):338–341. PMID:2001512.https://doi.org/10.1177/096100062096773.
Published
Issue
Section
License
Copyright (c) 2026 Paul Sebo, Bing Nie, Ting Wang
This work is licensed under a Creative Commons Attribution 4.0 International License.
