Paper

Publication outcome of abstracts submitted to the American Academy of Ophthalmology meeting^*

Michael Mimouni, MD, Mark Krauthammer, MD, Hamza Abualhasan, MD, Hanan Badarni, MD, Kamal Imtanis, MD, Gilad Allon, MD, Liron Berkovitz, MD, Eytan Z. Blumenthal, MD, Francis B. Mimouni, MD, Gil Amarilyo, MD

doi: http://dx.doi.org/10.5195/jmla.2018.314

Received August 2017: Accepted September 2017

ABSTRACT

Objective

Abstracts submitted to meetings are subject to less rigorous peer review than full-text manuscripts. This study aimed to explore the publication outcome of abstracts presented at the American Academy of Ophthalmology (AAO) annual meeting.

Methods

Abstracts presented at the 2008 AAO meeting were analyzed. Each presented abstract was sought via PubMed to identify if it had been published as a full-text manuscript. The publication outcome, journal impact factor (IF), and time to publication were recorded.

Results

A total of 690 abstracts were reviewed, of which 39.1% were subsequently published. They were published in journals with a median IF of 2.9 (range 0–7.2) and a median publication time of 426 days (range 0–2,133 days). A quarter were published in the journal Ophthalmology, with a shorter time to publication (median 282 vs. 534 days, p=0.003). Oral presentations were more likely to be published than poster presentations (57.8% vs. 35.9%, p<0.001) and in journals with higher IFs (3.2 vs. 2.8, p=0.02). Abstracts describing rare diseases had higher publication rates (49.4% vs. 38.0%, p=0.04) and were published in higher IF journals (3.7 vs. 2.9, p=0.03), within a shorter period of time (358 vs. 428 days, p=0.03). In multivariate analysis, affiliation with an institute located in the United States (p=0.002), abstracts describing rare diseases (p=0.03), and funded studies (p=0.03) were associated with publication in higher IF journals.

Conclusions

Almost 40% of abstracts were published. Factors that correlated with publication in journals with higher IF were a focus on rare diseases, affiliation with a US institute, and funding.

INTRODUCTION

Scientific meetings are a platform for sharing developments in medical research. These meetings allow investigators to present current research data and findings, to exchange ideas, and to initiate future collaboration. Presentation of a study at a scientific conference is a method for rapidly disseminating research information that otherwise would take months or even years until it is made available to the colleagues through peer-reviewed journals [1].

Hundreds, and in some cases thousands, of abstracts are submitted to each scientific meeting. A subset is accepted as an oral presentation, a printed poster, or an electronic poster. The selection process is often not transparent from the author’s perspective. Generally, a program committee reviews the abstracts and decides upon their fates. Often, abstracts that are deemed more original, interesting, and/or of higher scientific value are presented orally, whereas the remaining accepted abstracts are presented as posters.

Not all meeting abstracts eventually get published in a peer-reviewed journal. In fact, previously reported publication rates in other fields of medicine (percentage of abstracts presented in a meeting that were eventually published) range from 20.5% to 68.9% [2, 3]. It is unclear which factors play a role, and to what extent, in the publication outcome of each abstract.

The purpose of this study was to determine the publication rate of abstracts presented at the American Academy of Ophthalmology (AAO) annual meeting and to identify factors associated with (a) higher publication rates, (b) publication in journals with greater impact factors (IFs), and (c) shorter time to publication.

METHODS

Abstract data collection

All of the 690 abstracts presented in the 2008 AAO meeting were identified and retrieved from the AAO meeting archive website. The 2008 AAO meeting was chosen to allow a sufficient interval between the time when the abstracts were presented (November 2008) and the time when this study was conducted (December 2014). Some journals consider data greater than 5 years old as obsolete and will not consider papers with old data for review. In addition, according to reports in other fields of medicine, few abstracts are published 5 years beyond their initial presentation [1, 4–6].

Abstracts were categorized by presentation format (poster or oral presentation), ophthalmic subspecialty, research scope (basic science or clinical research), methodology (prospective or retrospective, randomized or non-randomized), prevalence (common or rare disease), and use of a new technique. A rare disease was defined using the definitions specified in the Rare Disease Act of 2002.

Full-text manuscript search

To identify whether an abstract had been published as a full-text manuscript, a PubMed search was initially conducted. Only AAO abstracts matching full-text manuscripts or case reports appearing in PubMed were categorized as published. AAO abstracts appearing as letters to the editor were not categorized as published. To maximize accuracy, the search protocol included several phases. In the first phase, two independent investigators manually searched the PubMed database for each AAO abstract using a predefined search algorithm composed of a series of queries (Figure 1). The result of each query was manually reviewed for a matching full-text manuscript. A third reviewer adjudicated if disagreement arose.

In the second phase, AAO abstracts that were not found during the first phase were addressed by two coauthors who repeated the search algorithm to verify correct categorization. In the third and final phase, a random sample of AAO abstracts (20%) that had been identified as not published was selected, and the authors were contacted via email (with a second email sent 10–14 days following the first email if it was not answered). The authors were asked to provide information regarding the publication status of their AAO abstracts and any reasons why, in their opinion, it had not yet been published (supplemental appendix).

Full-text manuscript data collection

For each AAO abstract categorized as published, the following information was collected from the matching full-text manuscript: number of subjects, journal title, journal IF, author’s affiliation, number of authors, funding, and publication date.

Statistical analysis

Data were analyzed using Minitab software (version 16, Minitab, State College, Pennsylvania). A chi-square test was used for categorical variables. Normality of continuous data was assessed by the Kolmogorov-Smirnov test. The Student t-test or one-way analysis of variance (ANOVA) was used for normally distributed data, and the Kruskal-Wallis test for non-normally distributed data. Stepwise regression analysis included independent variables that reached a significance level of <0.05 in univariate analysis. A p-value <0.05 was considered statistically significant.

RESULTS

A total of 690 AAO abstracts were reviewed and analyzed, of which 14.8% (n=102) were oral presentations and 85.2% (n=588) were posters. There was 99.0% agreement (n=683) between the 2 independent investigators regarding the publication status of each abstract. The overall publication rate was 39.1% (n=270).

An email was sent to the authors of 20% (randomly sampled) of the AAO abstracts identified as not having been published (n=84), and a response was received regarding 26 AAO abstracts (31% response rate). Twelve of the authors replied that the full-text manuscript was rejected and resubmission abandoned due to low likelihood of acceptance (n=6), they had no time to resubmit (n=2), or other reasons (n=4). Seven authors replied that a different version of the data was published following rejection of the original study data. Four authors replied that the full-text manuscript was never submitted because they did not have time (n=2) or because the study was ongoing (n=2). Two authors replied that they had insufficient recollection of the abstract and its publishing outcome. One author replied that the full-text manuscript was published based on data presented in the meeting. Therefore, 1 out of 26 abstracts was found to be incorrectly categorized as not being published (3.8%), following the authors’ responses.

The number of AAO abstracts presented in each subspecialty and the corresponding publication rate are presented in Table 1. Briefly, there was a significant difference in publication rate between subspecialties (χ²=23.85, df=10, p=0.008), with intraocular inflammation and uveitis demonstrating the highest publication rate and refractive surgery the lowest publication rate.

Table 1 Distribution of abstracts presented and publication rate depending on subspecialty

The univariate analysis of factors associated with higher publication rates of AAO abstracts is depicted in Table 2. Oral presentations had a significantly higher publication rate than poster presentations. Similarly, abstracts describing rare diseases had a significantly higher publication rate than abstracts describing common diseases.

Table 2 Differences in publication rates of abstracts based on various factors

* Chi-square.

Table 3 depicts the results of stepwise binary logistic regression. Briefly, AAO abstracts were more likely to be published if they were orally presented, and the subspecialty of the presented abstract remained a significant predictor of whether it would be subsequently published.

Table 3 Multivariate binary logistic regression analysis of factors predicting publication

The median IF of journals in which abstracts were published was 2.9 (range 0–7.2). Of those published, 25.2% were published in the journal Ophthalmology (n=68), which is the official journal of the AAO. Table 4 shows the median IFs of the journals in which AAO abstracts were published, based on the factors examined. Briefly, oral presentations, abstracts describing rare diseases, studies with more than 1,000 subjects, funded studies, and studies in which the first author was affiliated with an institution in the United States were published in journals with higher IFs than those of their counterparts.

Table 4 Univariate analysis of factors and the impact factor (IF) of the journal in which they were published

* Kruskal-Wallis.

† Pediatric and glaucoma AAO abstracts were published in journals with the highest (5.27) and lowest (1.98) median IF, respectively.

Ranked stepwise multiple regression analysis showed that abstracts with a first author affiliated with an institution in the United States (R 2=4.2, p=0.002), those that described rare diseases (R 2=2.3, p=0.03), and funded studies (R 2=2.1, p=0.03) were associated with publication in journals with higher IFs.

The median publication time was 426 days (range 0–2,133 days). Meeting abstracts that were published in Ophthalmology were published nearly twice as fast as those that were published elsewhere (median 282 vs. 534 days, p=0.003). Table 5 depicts the time to publication depending on different factors. Briefly, studies describing rare diseases were published faster than those describing common diseases, and studies in which the number of subjects in the full-text manuscript was larger than that in the abstract (“subjects increased”) had longer times to publication. Both factors remained significant in ranked stepwise multiple regression analysis (subjects increased, R²=2.0, p=0.02; rare disease, R²=1.8, p=0.04).

Table 5 Univariate analysis of factors and their publication times

DISCUSSION

This study analyzed the publication outcomes of 690 abstracts presented at the AAO 2008 annual meeting. The overall publication rate of 39.1% was comparable to the reported publication rates of scientific meetings in other fields of medicine [2, 3, 7, 8] and the Canadian Ophthalmological Society annual meeting (45.7%) [9], while higher than that of the Royal College of Ophthalmologist annual congress (26.6%) [10]. However, it is strikingly lower than a study published more than 2 decades ago reporting that 57% of the AAO meeting abstracts reached full-text manuscript publication [6].

In the current study, most abstracts presented in the AAO meeting were not subsequently published in a peer-reviewed journal, either because they were never submitted to a journal or did not withstand the peer-review process. Interestingly, Saldanha et al. analyzed randomized controlled trials that had been presented at the 2001–2004 Association for Research in Vision and Ophthalmology conferences and reported that more than half the examined publications exhibited some amount of discordance in the main outcome results when compared with presented abstracts, calling into question the dependability of conference abstracts [11]. Together, these findings suggest that researchers should interpret preliminary data presented at scientific meetings with caution.

In this study, oral presentations were more likely to be published than poster presentations, a finding that was supported by a previous study in the fields of rheumatology [7] and veterinary ophthalmology [12]. Only a subset of AAO abstracts (14.8% in 2008) were selected to be presented in oral format. According to the AAO website, the Annual Meeting Program Committee chooses submissions to be presented as papers (i.e., oral presentations) based on originality, clinical relevance, comprehensiveness, and potential as a stimulus for prepared discussion, giving greater priority to novel work [13]. As such, it was not surprising that oral presentations go on to be published more often than poster presentations.

The median IF of journals in which AAO abstracts were published was 2.9, well above that of the 1.7 median IF of ophthalmology journals at the time this study was conducted (Journal Citation Reports 2013). Although IF has limitations, it is still considered a legitimate indicator of journal quality [14, 15]. The relatively high IF of the published AAO abstracts might be, as previously mentioned, the result of a filtering process of the AAO program committees in which the finest abstracts were chosen.

Interestingly, AAO abstracts in which the first author was affiliated with an institution in the United States were published in journals with higher IFs. This was supported by a previous study analyzing abstracts from the European College of Veterinary Ophthalmologists meeting, where author nationality and academic association were found to be significant factors of publication [12]. This might be the result of US study groups being based in tertiary centers with higher funding and more abundant resources available for conducting high-quality research. In addition, the English language serves as the main language for a majority of the scientific journals in general and ophthalmology journals specifically. Therefore, native English speakers may have an advantage when it comes to scientific writing and publishing. Indeed, non-native-English-speaking academicians have been reported to be less satisfied with the peer-review process [16] and may benefit from scientific writing training programs [17].

Funded studies were published in journals with a higher IF than non-funded studies. This may be the result of a filtering process performed by the parties providing the funding, in which only the studies deemed worth investigating are granted financial support. In addition, the funding of a study provides the researcher with ampler resources than those that do not receive funding. Finally, a study that has received funding is generally supervised by the funding party, who would have a vested interest in verifying that the study and its findings are indeed published. The financial relationship between pharmaceutical companies and researchers has increased dramatically and has been the subject of much debate [18, 19]. It has been shown that regardless of the main outcome results, conflicts of interests of first authors of abstracts are associated with whether they go on to be published [11].

Studies of rare diseases were published in journals with higher IFs and with shorter times to publication. This finding is encouraging considering that for many rare diseases, there are no effective treatments [20]. We must note that nearly one-third of “transformative drugs,” defined as pharmaceuticals that are innovative and have groundbreaking effects on patient care, that were approved between 1984 and 2009 were originally developed for rare diseases before broader applicability was found [21]. Therefore, rare diseases are not only more interesting for journal editors, reviewers, and readers, they at times constitute a bridge for scientific breakthroughs that later apply to more common diseases.

Median publication time was 426 days, similar to that of abstracts presented in a previous study in the field of rheumatology [7]. Those published in Ophthalmology were published in half of the time of those published elsewhere. This may be a byproduct of the AAO meeting and Ophthalmology journal guidelines stating that Ophthalmology has right of first refusal on these manuscripts. This may have also been the result of the relatively short time to publication of Ophthalmology, which was recently reported to be 297.5 days (range 266.3–353.0 days) [22], similar to the median of 282.0 days in this study. Also, studies in which the number of participants increased between the time of the AAO abstract and the full-text manuscript had a longer time to publication. This can be explained by the additional amount of time and resources that would be required to include more subjects in a study.

One limitation of this study is that meeting abstract data and findings may be partial or preliminary and the results may vary greatly before being submitted for publication. For instance, this study has shown that the number of study participants might increase. As such, many additional factors could change that we may have overlooked, such as diagnostic accuracy [23] and author conflict of interest [11]. In addition, the multiple tests performed in this study could potentially have led to capitalizing on chance (i.e., type I error).

Another limitation of this study relates to the fact that the data from a single meeting were analyzed and, therefore, might not apply to AAO meetings from other years or to other ophthalmic scientific meetings. Further studies could be performed to compare the publication rates of AAO meetings with those of other international ophthalmology meetings. An additional limitation is that we were unable to follow-up with abstract proposals that were not accepted for the conference, as it was possible that some of them were ultimately published as well. Lastly, despite the rigorous attempt to correctly identify the publication outcome of each abstract, the questionnaire sent to the authors showed that 4% were falsely identified as not published.

This study shows that less than half of the abstracts accepted for presentation at an AAO annual meeting were eventually published. Those that were published appeared in peer-reviewed journals with relatively high IFs. This may reflect that abstracts accepted for presentation by the AAO program committees, particularly as oral presentations, are a well-selected group of research studies.

Supplemental File

CONFLICT OF INTEREST

No conflicting relationship exists for any author.

REFERENCES

1 Scherer RW, Langenberg P, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev. 2007 Apr 18;(2):MR000005.
  

2 Autorino R, Quarto G, Sio MD, Lima E, Quarto E, Damiano R, Oliviero R, Osorio L, Marcelo F, D’Armiento M. Fate of abstracts presented at the World Congress of Endourology: are they followed by publication in peer-reviewed journals? J Endourol. 2006 Dec;20(12):996–1001.
  

3 Housri N, Cheung MC, Gutierrez JC, Zimmers TA, Koniaris LG. SUS/AAS abstracts: what is the scientific impact? Surgery. 2008 Aug;144(2):322–31.
    

4 Smollin CG, Nelson LS. Publication of abstracts presented at 2001 NAACT. J Med Toxicology. 2006 Sep;2(3):97–100.
  

5 Donegan DJ, Kim TW, Lee GC. Publication rates of presentations at an annual meeting of the American Academy of Orthopaedic Surgeons. Clin Orthop Relat Res. 2010 May;468(5):1428–35.
    

6 Juzych MS, Shin DH, Coffey JB, Parrow KA, Tsai CS, Briggs KS. Pattern of publication of ophthalmic abstracts in peer-reviewed journals. Ophthalmology. 1991 Apr;98(4):553–6.
    

7 Amarilyo G, Woo JM, Furst DE, Hoffman OL, Eyal R, Piao C, Parker DS, McCurdy DK. Publication outcomes of abstracts presented at an American College of Rheumatology/Association of Rheumatology Health Professionals annual scientific meeting. Arthritis Care Res (Hoboken). 2013 Apr;65(4):622–9.
  

8 Wong CX, Wong MX, Wong NX, Sun MT, Brooks AG, Stiles MK, Lau DH, Nelson AJ, De Sciscio P, Shipp NJ, Sanders P. Impact of research presentations at the annual scientific sessions of the Heart Rhythm Society. Heart Rhythm. 2009 Sep;6(9):1345–8.
    

9 Basilious A, Benavides Vargas AM, Buys YM. Publication rate of abstracts presented at the 2010 Canadian Ophthalmological Society annual meeting. Can J Ophthalmol. 2017 Aug;52(4):343–8.
    

10 Okonkwo AC, Hogg HD, Figueiredo FC. An 8-year longitudinal analysis of UK ophthalmic publication rates. Eye (Lond). 2016 Nov;30(11):1433–8.
  

11 Saldanha IJ, Scherer RW, Rodriguez-Barraquer I, Jampel HD, Dickersin K. Dependability of results in conference abstracts of randomized controlled trials in ophthalmology and author financial conflicts of interest as a factor associated with full publication. Trials. 2016 Apr 26;17(1):213.
      

12 Ofri R, Bdolah-Abram T, Yair N. Factors affecting peer-reviewed publication of abstracts presented at meetings of the European College of Veterinary Ophthalmologists (2008–2012). Vet Ophthalmol. 2017 Mar 1:533–8.
  

13 American Academy of Ophthalmology. Submission instructions [Internet]. The Academy [cited 20 Sep 2017]. <https://www.aao.org/annual-meeting/presenter/paper-submission-instructions>.

14 Elliott DB. The impact factor: a useful indicator of journal quality or fatally flawed? Ophthalmic Physiol Opt. 2014 Jan;34(1):4–7.
  

15 Cartwright VA, Savino PJ. Ophthalmology journals and the ether: considering journal impact factor and citation analysis in context. Clin Exp Ophthalmol. 2009 Dec;37(9):833–5.
  

16 Ho RC, Mak KK, Tao R, Lu Y, Day JR, Pan F. Views on the peer review system of biomedical journals: an online survey of academics from high-ranking universities. BMC Med Res Methodol. 2013 Jun 7;13:74.
      

17 Cameron C, Deming SP, Notzon B, Cantor SB, Broglio KR, Pagel W. Scientific writing training for academic physicians of diverse language backgrounds. Acad Med. 2009 Apr;84(4):505–10.
    

18 Patsopoulos NA, Ioannidis JP, Analatos AA. Origin and funding of the most frequently cited papers in medicine: database analysis. BMJ. 2006 May 6;332(7549):1061–4.
      

19 Buchkowsky SS, Jewesson PJ. Industry sponsorship and authorship of clinical trials over 20 years. Ann Pharmacother. 2004 Apr;38(4):579–85.
    

20 Luzzatto L, Hollak CE, Cox TM, Schieppati A, Licht C, Kääriäinen H, Merlini G, Schaefer F, Simoens S, Pani L, Garattini S, Remuzzi G. Rare diseases and effective treatments: are we delivering? Lancet. 2015 Feb 28;385(9970):750–2.
    

21 Kesselheim AS, Tan YT, Avorn J. The roles of academia, rare diseases, and repurposing in the development of the most transformative drugs. Health Aff (Millwood). 2015 Feb;34(2):286–93.
  

22 Chen H, Chen CH, Jhanji V. Publication times, impact factors, and advance online publication in ophthalmology journals. Ophthalmology. 2013 Aug;120(8):1697–701.
    

23 Korevaar DA, Cohen JF, Spijker R, Saldanha IJ, Dickersin K, Virgili G, Hooft L, Bossuyt PM. Reported estimates of diagnostic accuracy in ophthalmology conference abstracts were not associated with full-text publication. J Clin Epidemiol. 2016 Nov;79:96–103.
      

(Return to Top)

Michael Mimouni, MD† (corresponding author), michael@intername.co.il, Department of Ophthalmology, Rambam Health Care Campus, affiliated with the Technion Israel Institute of Technology, Haifa, Israel

Mark Krauthammer, MD,† krautmarik@gmail.com, Department of Ophthalmology, Sourasky Medical Center, Tel Aviv, Israel

Hamza Abualhasan, MD, hamzawaleed@gmail.com, Department of Ophthalmology, Rambam Health Care Campus, affiliated with the Technion Israel Institute of Technology, Haifa, Israel

Hanan Badarni, MD, dr.hanan.badarni@gmail.com, Department of Ophthalmology, Rambam Health Care Campus, affiliated with the Technion Israel Institute of Technology, Haifa, Israel

Kamal Imtanis, MD, ka.imtanis@gmail.com, Department of Ophthalmology, Rambam Health Care Campus, affiliated with the Technion Israel Institute of Technology, Haifa, Israel

Gilad Allon, MD, drgiladallon@gmail.com, Department of Ophthalmology, Rambam Health Care Campus, affiliated with the Technion Israel Institute of Technology, Haifa, Israel

Liron Berkovitz, MD, lironberkovitz@gmail.com, Department of Ophthalmology, Rambam Health Care Campus, affiliated with the Technion Israel Institute of Technology, Haifa, Israel

Eytan Z. Blumenthal, MD, Eytan@blumenthal.org.il, Department of Ophthalmology, Rambam Health Care Campus, affiliated with the Technion Israel Institute of Technology, Haifa, Israel

Francis B. Mimouni, MD, fbmimouni@gmail.com, Department of Pediatrics, Shaare Zedek Medical Center, Jerusalem, Israel

Gil Amarilyo, MD, gamarilyo@tlvmc.gov.il, Schneider Children’s Medical Center of Israel and Tel Aviv University, Tel Aviv, Israel

^*Data from this study were presented at the annual American Academy of Ophthalmology meeting; Las Vegas, NV; 2015. ( Return to Text )

FINANCIAL SUPPORT

None. ( Return to Text )

Articles in this journal are licensed under a Creative Commons Attribution 4.0 International License.

This journal is published by the University Library System of the University of Pittsburgh as part of its D-Scribe Digital Publishing Program and is cosponsored by the University of Pittsburgh Press.

Journal of the Medical Library Association, VOLUME 106, NUMBER 1, January 2018