The Proportion of Abstracts Presented at the 2010 American Academy of Orthopaedic Surgeons Annual Meeting Ultimately Published

Materials and Methods

The final program of the 77th Annual Meeting of the AAOS, March 2010, was used to review the 1289 abstracts listed in the proceedings (Figure 1). The abstracts were divided by presentation format into podium and poster groups. The 2010 AAOS Annual Meeting deadline for abstract submission was June 1, 2009; abstracts with a journal publication before this date were excluded from this study (18 podium and 16 poster abstracts), and publications that occurred after the submission date but prior to the conference date were included. Four podium abstracts were withdrawn before the annual meeting, leaving 698 abstracts eligible for analysis in the podium group. Four poster abstracts listed as allied health recruitment and 1 that was published in the proceedings as a duplicate were excluded, leaving 548 abstracts in the poster group.

Figure 1: Flow diagram of the included and excluded podium and poster abstracts. Abbreviation: AAOS, American Academy of Orthopaedic Surgeons.

The 2010 Annual Meeting was selected to allow for a 7-year follow-up window for journal publication because prior studies have shown that proportions of abstracts published plateau at 5 years.^3,8,9,13 For each abstract, 2 of 3 authors (B.R.W., W.G.F., T.J.R.) independently searched the PubMed, Embase, and Google Scholar databases to determine whether there was a subsequent journal publication. The search method was similar to those used in prior studies.^3,8,9,11 A search was performed in each database with keywords from the abstract title and each author individually until a publication resulted. Journal article abstracts were initially reviewed, and if there was any question on a match, the full-text journal article was retrieved and reviewed. If the search resulted in no publication, an abstract was considered not published. If there was disagreement on whether a journal publication matched, the third author reviewing abstracts conducted a search as well, breaking the tie. Disagreements were limited to fewer than 30 of the total 1246 abstracts presented.

From each abstract, the authors extracted the number of authors, author country of origin, and orthopedic subspecialty topic. Author origin was based on the site of AAOS registration noted in the final proceedings. If a journal publication was matched, the journal and the time from meeting presentation to publication were noted. The authors also calculated the number of abstracts published in journals within 3 and 4 years from presentation. If an electronic publication (e-pub) occurred before a hardcopy publication, that date was recorded as the date of publication. If no specific day of the month was noted, the authors recorded the first of the month as the publication day. The sub-specialty topic of the abstract was tracked, along with any changes in the number of authors, first author, and number of study subjects between abstract and journal publication. The authors compared the conclusions from meeting abstracts with the conclusions made in the final journal publications, and they recorded those that contradicted each other.

Results

Eight (1.7%) podium abstracts and 5 (1.7%) poster abstracts had conclusions that contradicted their subsequent journal publications. For podium abstracts that proceeded to journal publication, 70% had no change in the number of study subjects, whereas 17% reported an increase and 13% reported a decrease (Table 1). For poster abstracts that went on to journal publication, 56% had no change in the number of study subjects, 24% reported an increase, and 18% reported a decrease. The mean percent change in study size reported from abstract to publication was 39% for podium and 44% for poster (P=.68). The mean number of authors increased between podium abstracts (4.7±1.9) and their subsequent journal publication (5.4±2.3) (∆+ 0.6; 95% CI, 0.35–0.83; P<.001), as well as for poster abstracts (4.9±1.8) and their subsequent journal publication (5.6±2.4) (∆+ 0.6; 95% CI, 0.34–0.90; P<.001) (Table 1). Authorship changes were seen in 63% of the podium abstracts that went on to journal publication. Of those changes, 39% were the addition of at least 1 author and 17% were the removal of at least 1 author. For the poster abstracts, 66% had an authorship change, with 45% adding at least 1 author and 16% removing at least 1 author. A change in the first author was present in 28% of podium abstracts and 35% of poster abstracts (P=.033).

Table 1: Characteristics of Published Podium and Poster Abstracts

The overall proportion of all abstracts presented at the 2010 AAOS Annual Meeting that went on to journal publication was 61% (766 of 1246) (Table 2). The percent published was 68% (476 of 698) for podium presentations and 53% (290 of 548) for poster presentations. Podium presentations were more likely to proceed to journal publication than were poster presentations (OR, 1.90; 95% CI, 1.5–2.4; P<.001).

Table 2: Characteristics of Podium and Poster Abstracts of the 2010 American Academy of Orthopaedic Surgeons Annual Meeting

The mean time to publication for podium abstracts was 1.45±1.4 years. Eighty-six percent of podium abstracts that went on to journal publication did so within 3 years, and 94% by 4 years (Figure 2). For poster abstracts, the mean time to publication was 1.4±1.3 years. Within 3 years, 87% went on to journal publication, and 95% by 4 years.

Figure 2: Time to publication for podium and poster abstracts. A publication time of −0.75 years indicated the abstract was published after it was submitted to the American Academy of Orthopaedic Surgeons Annual Meeting but prior to the meeting date.

Adult hip reconstruction and adult knee reconstruction were the most common meeting subtopics (Table 3). The most common podium abstract topics to proceed to journal publication were shoulder/elbow (78%) and sports medicine/arthroscopy (76%), whereas pediatrics (78%) and spine (64%) were the most common for poster abstracts (Table 4). US authors were listed on 68% of podium abstracts, followed by European authors on 14% (Table 2). More than 2 countries were listed on 10.7% of podium abstracts. US authors were listed on 63% of poster abstracts, followed by European authors on 16%. More than 2 countries were listed on 12% of poster abstracts.

Table 3: Podium and Poster Abstract Subtopics From the 2010 American Academy of Orthopaedic Surgeons Annual Meeting

Table 4: Percentage of Journal Publications of Podium and Poster Abstract Subtopics From the 2010 American Academy of Orthopaedic Surgeons Annual Meeting

The most common journals of publication, from 101 separate journals, for podium abstracts were The Journal of Bone & Joint Surgery (11.8%) and The American Journal of Sports Medicine (8.0%) (Table 5). Of the podium abstracts that were published, 89% had an impact factor above 1.0 and 9% were published in journals without an impact factor. Only 2.9% of the abstracts were published in journals that had been established less than 6 years from the 2010 Annual Meeting. The most common journals of publication, from 87 separate journals, for poster abstracts were Clinical Orthopaedics and Related Research (10.0%) and The Journal of Arthroplasty (9.3%). Of the poster abstracts that were published, 91% had an impact factor above 1.0 and 6.2% were published in journals without an impact factor. Only 3.8% were published in journals that had been established less than 6 years from the 2010 Annual Meeting.

Table 5: Top 10 Journals of Publication for Presented Abstracts

Discussion

Annual orthopedic meetings are a major avenue for sharing novel and cutting-edge research. Clinicians in attendance can gather new developments and synthesize that information into their knowledge base. As they do, it is helpful to consider the quality of the new information they are consuming. Understanding when and where the final disposition of presented abstracts is may aid in these considerations. This study found that only 1.7% of abstract conclusions were contradicted in the subsequent journal publication. From the 2010 AAOS Annual Meeting, 61% of presented abstracts went on to journal publication, compared with 49% in 2001⁸ and 34% in 1996.³ Attendees of the orthopedic meetings can better assess the consistency of the presented abstracts and the likelihood that the presented data will be published and eventually become part of orthopedic practice.

Of concern in this study is that 8 (1.7%) podium abstracts and 5 (1.7%) poster abstracts that went on to subsequent journal publication had different conclusions between presentation at the meeting and publication. Although these numbers are small, they speak to the fact that some studies presented at meetings are ongoing or may be altered prior to publication. These concerns are most notable in the study population changes. The authors found that 30% of podium and 44% of poster abstracts had a change in study population size by journal publication. A review of the 1996 Annual Meeting found that 18% of abstracts had a change in study population size,³ suggesting that this may be an expanding trend. The number of studies with increased numbers of subjects may indicate that studies are being submitted to meetings while they are ongoing. Therefore, the presented abstract may not represent the definitive patient population and, more importantly, the definitive study findings or conclusion. However, this study found that 13% of podium and 18% of poster abstracts had a subsequent decrease in study populations. Although this could be due to something as innocent as misrepresentation of inclusion and exclusion criteria, it could also be due to more concerning things such as P value hacking. P value hacking is the effect of altering the inclusion criteria or statistical analysis until nonsignificant data become statistically significant.¹⁴ This phenomenon is a growing concern and has the potential to be harmful to patients.¹⁴

Of additional concern is the increased authorship on studies as they moved from abstract to journal publication. This trend in expansion of authorship has been observed in the orthopedic literature during the past few decades.^15–18 It may be due to increased pressure from institutions for their faculty to publish.¹⁹ The “gifting” of authorship undermines the trust in an author's contribution.¹⁵

Recent studies from orthopedic sub-specialty meetings show that the proportions of published abstracts have also been trending upward since 2000 (range, 49%–67%).^{6–8,10–12,20–24} This may be due to an increasing volume of orthopedic articles being published,^25–28 higher quality research being done,²⁹ or increased academic institutional pressure to publish.^15,19,30 Knowing which meetings and which presentation modalities result in a higher likelihood of being published can better direct meeting attendees as to where to spend their time. Studies with a lower likelihood of being published in a journal, and therefore a lower likelihood of becoming part of orthopedic clinical care, are potentially not worth the time of clinicians and clinician-scientists.

Similar to the study of the 2001 AAOS Annual Meeting,⁸ in this study, the subtopics of sports, arthroscopy, and spine had the highest proportion of journal publications. This is consistent with the subspecialty meetings: 67% for the 2006–2010 American Orthopaedic Society for Sports Medicine meeting,⁹ 67.3% for the 2004–2012 Arthroscopy Association of North America meeting,¹⁰ and 66% for the 2007–2011 Cervical Spine Research Society meeting.²¹ Subtopics with higher publication percentages are potentially fields with more unanswered questions or are of greater interest to researchers in orthopedics and/or industry. Further research at different time points is needed to better elucidate the factors behind these differences. Being aware of the country of authorship is important as well because it indicates where the current research is being conducted. Globally, there has been a downward trend in publications from the United States and an upward trend in publications from countries such as China and Korea.²⁵ This trend introduces new patient populations and different orthopedic care treatments that could eventually become part of and guide US orthopedic clinical practice.

This study had several limitations. Although the authors used a methodology similar to that previously described,^3,8,9,20 their searches were limited to PubMed, Embase, and Google Scholar. A review of 738 articles used in 29 systematic reviews published in The Journal of the American Medical Association or Cochrane Database found that Google Scholar was able to retrieve all of them.³¹ In an analysis of 4795 references from 120 systematic reviews, Google Scholar returned 97.2% of the citations, whereas Embase and PubMed returned 97.5%.³² Given these studies, there is a possibility that the authors were not able to retrieve all journal publications, although their approach was consistent with all of the other prior studies that they found. This study's follow-up on abstract publication time was limited to 7 years from the conference. It has been previously noted that most publications occur within 5 years.^2,3,8,11

Conclusion

This study found a 61% rate of subsequent journal publication of abstracts presented at the 2010 AAOS Annual Meeting, which is an increase from prior years. It is essential that the usefulness and clinical applicability of meetings continue to be evaluated. Knowing the final disposition of conference abstracts, where they are coming from, and when they will be in final print format are components of this evaluation. As conference attendees glean new information from presented abstracts, these data may help them to gauge the potential for that information to become part of accepted orthopedic care and therefore be useful as a basis for future clinical care or research endeavors.