Orthopedics

Feature Article 

In-depth Analysis of Adult Reconstruction Fellowship Websites

Shaan A. Ahmed, BA; William Elnemer; Adam E. M. Eltorai, PhD; Michael Mariorenzi, MD; Alan H. Daniels, MD; Eric M. Cohen, MD

Abstract

This investigation assessed the availability and comprehensiveness of adult reconstruction fellowship websites. A list of adult reconstruction fellowship programs was compiled using 4 online directories: American Association of Hip and Knee Surgeons, San Francisco Match, Fellowship and Residency Electronic Interactive Database Access, and American Academy of Orthopaedic Surgeons. These directories and Google (Alphabet Inc, Menlo Park, California) searches were used to assess for the presence and functionality of websites for each program. Each website was reviewed for the presence of 21 variables related to education and recruitment. The relationship between several program characteristics and presence of a website or website comprehensiveness was evaluated. In total, 81 programs were identified, of which 80.2% (65 of 81) had a functional website and 19.8% (16 of 81) did not. The mean±SD number of variables present on a website was 11.3±3.8 of 21 total variables, equating to mean comprehensiveness of 54.0%. Fellowships affiliated with an orthopedic residency were more likely to have a website than those that were not (91.8% vs 62.5%, P=.001). Fellowships that were accredited by the Accreditation Council for Graduate Medical Education were more likely to have a website than those that were not (100.0% vs 73.8%, P=.009). Website comprehensiveness was not associated with any program characteristics assessed. This study highlights deficits in online resources available for adult reconstruction fellowship program information and the gap that exists between the current means of information sharing. Efforts to encourage an online presence and optimize a program’s online content may be of benefit to prospective applicants and the program itself. [Orthopedics. 2020;43(5):e447–e453.]

Abstract

This investigation assessed the availability and comprehensiveness of adult reconstruction fellowship websites. A list of adult reconstruction fellowship programs was compiled using 4 online directories: American Association of Hip and Knee Surgeons, San Francisco Match, Fellowship and Residency Electronic Interactive Database Access, and American Academy of Orthopaedic Surgeons. These directories and Google (Alphabet Inc, Menlo Park, California) searches were used to assess for the presence and functionality of websites for each program. Each website was reviewed for the presence of 21 variables related to education and recruitment. The relationship between several program characteristics and presence of a website or website comprehensiveness was evaluated. In total, 81 programs were identified, of which 80.2% (65 of 81) had a functional website and 19.8% (16 of 81) did not. The mean±SD number of variables present on a website was 11.3±3.8 of 21 total variables, equating to mean comprehensiveness of 54.0%. Fellowships affiliated with an orthopedic residency were more likely to have a website than those that were not (91.8% vs 62.5%, P=.001). Fellowships that were accredited by the Accreditation Council for Graduate Medical Education were more likely to have a website than those that were not (100.0% vs 73.8%, P=.009). Website comprehensiveness was not associated with any program characteristics assessed. This study highlights deficits in online resources available for adult reconstruction fellowship program information and the gap that exists between the current means of information sharing. Efforts to encourage an online presence and optimize a program’s online content may be of benefit to prospective applicants and the program itself. [Orthopedics. 2020;43(5):e447–e453.]

It is well recognized that the overwhelming majority of orthopedic surgery residents go on to complete an additional year of subspecialized fellowship training. Despite the socioeconomic and financial implications of the additional time spent training, the literature shows that greater than 90% of orthopedic residents complete at least 1 fellowship, with some electing to complete more than 1 fellowship.1–3 Since the initiation of fellowship accreditation by the Accreditation Council for Graduate Medical Education (ACGME) in the 1980s, the focus of the orthopedic job market has shifted dramatically toward the recruitment of fellowship-trained orthopedic surgeons.1 This trend highlights the perceived professional advantages compelling surgical trainees to complete a fellowship as part of a comprehensive training experience.

Advances in web-based platforms and internet accessibility have resulted in substantial emphasis on the internet as a source of information while applying to residency and fellowship programs. The literature has shown that nearly all applicants applying to residency and fellowship programs use a program’s website for information4–7 and that the majority identify a program’s website as the single most important source of information.5 Despite this increasing reliance on websites for information gathering, numerous studies across multiple subspecialties have demonstrated substantial deficiency in the availability of information online and the underutilization of the internet as a recruitment tool.8–14 In fact, the information provided on a program website and the quality of a program website have been shown to influence applicants’ decision to apply to that program.7

In a recent study in the Journal of Arthroplasty, it was found that 13% of adult hip and knee reconstruction fellowship programs did not maintain a functional website and that 10% had no online presence at all when accessed through the American Academy of Hip and Knee Surgeons (AAHKS) online directory or simple Google (Alphabet Inc, Menlo Park, California) search.15

However, no study to date has evaluated the accessibility and content of adult reconstruction fellowship websites (ARFWs) when accessed through the other major relevant online fellowship directories. The aim of the current study was to evaluate the accessibility and content of ARFWs across a variety of reputable online sources in an attempt to highlight the importance of standardization of fellowship program information. The authors hypothesized that despite extending the analysis to include the AAHKS, San Francisco Match (SFM), the Fellowship and Residency Electronic Interactive Database Access (FREIDA), the American Academy of Orthopaedic Surgeons (AAOS), and Google, the accessibility and content of ARFWs would remain unsatisfactory, and they support the need for the standardization of ARFWs or similar online resources.

Materials and Methods

Fellowship Directories

Four directories containing lists of orthopedic adult reconstruction fellowships were accessed in December 2017: SFM, the AAHKS website, American Medical Association’s FREIDA, and the AAOS website. These lists are potentially used by prospective applicants and warrant investigation. The names of all fellowship programs, website links (if provided), and number of fellowship positions per year (if provided) were extracted into a table for each of these databases. A master list of all unique programs across the 4 databases was then generated.

Fellowship Directory Website Hyperlink Evaluation

For each of the 4 directories, the website hyperlink for each fellowship program was evaluated as follows: not present, present but nonfunctioning, present but providing no information about the fellowship program, indirect (functional but requiring multiple further steps to get to the program website), or direct (functional and directly linking to the program website).

Internet Search

Using the master list generated from all 4 databases, each unique program was searched in Google. Two separate searches were performed: (1) program name followed by “adult reconstruction surgery fellowship” and (2) program name followed by “orthopedic adult reconstruction surgery fellowship,” and the first 10 search results were assessed. The second, more specific search term was only conducted if the first search term failed to yield a direct link to the program’s fellowship website. Program websites were assessed as follows: present (fellowship website found within first 10 search results), indirectly present (not found within first 10 results but accessed through program’s home website or through fellowship databases), or not present/nonfunctioning (could not be found through Google, program’s home website, or fellowship database, or website was not working). As each program was searched, the direct link to the ARFW (when available) was noted.

Website Evaluation

For each program on the master list, the ARFW was evaluated for comprehensiveness in 2 categories: education and recruitment. The presence of 10 education variables and 11 recruitment variables was assessed without regard for the quality or degree of detail provided (Table 1). These variables were selected based on similar prior studies of orthopedic fellowship websites.8–14 Using the methodology of previous studies, a website comprehensiveness score was then computed for each website in the education and recruitment categories based on the percentage of education or recruitment variables present for that website, respectively.8–14 For each variable, the percentage of websites that contained that variable was similarly computed.

Website Evaluation: Education and Recruitment Variables

Table 1:

Website Evaluation: Education and Recruitment Variables

Comparison of Fellowship Websites by Program Characteristics

Each fellowship program was further categorized based on several variables as follows. Geographic region was assessed as northeast, midwest, south, or west based on the US Census Bureau map. Presence of an orthopedic residency program affiliated with the fellowship institution was assessed using the American Association of Medical Colleges (AAMC) Electronic Residency Application Service (ERAS) list of MD orthopedic surgery residency programs. Affiliation of the fellowship program with a “top 20 orthopedic hospital” based on the U.S. News and World Report ranking of hospitals was assessed. The ACGME accreditation status of the fellowship program was assessed using FREIDA. The number of annual fellowship positions for each program was also noted and used as a proxy for program size. For most programs, this was obtained from the fellowship databases. For the 4 programs that did not have number of positions listed in any of the databases, it was obtained from the program’s website or by emailing the program contact person. All of these sources were accessed in December 2017.

After characterizing programs as described, analysis was conducted to determine whether the presence of a program website or website comprehensiveness was significantly associated with geographic region, affiliation with an orthopedic residency program, affiliation with a top 20 orthopedic hospital, ACGME accreditation status, or program size. Because the mean and median number of fellows per year was 2, programs with fewer than 3 fellows per year were considered “small” and programs with 3 or more fellows per year were considered “large” for program size.

Chi-square (or Fisher’s exact test where appropriate) and Student’s t test/one-way analysis of variance were used for categorical and continuous variables, respectively. All tests were 2-tailed, significance was set to P<.05, and Stata 14.0 (StataCorp, College Station, Texas) was used for statistical analysis. Because all of the information used for this study was publicly available, it was exempt from review by the institutional review board.

Results

Fellowship Program Directories and Hyperlink Evaluation

The SFM directory listed 79 programs, the AAHKS directory listed 81 programs, FREIDA listed 22 programs, and the AAOS directory listed 54 programs. The number of program website links provided in each directory and assessment of provided links is found in Figures 14. The presence of hyperlinks for programs was variable. For example, FREIDA provided a website link for 21 of 22 programs and SFM provided a link for 69 of 79 programs, whereas AAHKS provided links for only 14 of 81 programs listed. A comparison of these 4 directories is provided in Table 2.

San Francisco Match (SFM) directory.

Figure 1:

San Francisco Match (SFM) directory.

American Association of Hip and Knee Surgeons (AAHKS) directory.

Figure 2:

American Association of Hip and Knee Surgeons (AAHKS) directory.

American Medical Association Fellowship and Residency Electronic Interactive Database Access (FREIDA) directory.

Figure 3:

American Medical Association Fellowship and Residency Electronic Interactive Database Access (FREIDA) directory.

American Academy of Orthopaedic Surgeons (AAOS) directory.

Figure 4:

American Academy of Orthopaedic Surgeons (AAOS) directory.

Comparison of Fellowship Directories

Table 2:

Comparison of Fellowship Directories

From the 4 directories, a list of 86 unique programs was generated. Four Canadian programs and 1 American program reserved for an international fellow were excluded, yielding a final master list of 81 unique programs.

Internet Search

Of the 81 programs that were queried on Google, 76.5% (62 of 81) of programs had websites present within the first 10 search results, 3.7% (3 of 81) programs had websites that were indirectly present (not found within the first 10 results but accessed through program’s home website or through fellowship databases), and 19.8% (16 of 81) programs did not have websites or had nonfunctioning websites. Use of the second more specific search term did not change the result for any programs that were not found within the first 10 search results. Therefore, in total 80.2% (65 of 81) of programs had functioning websites and 19.8% (16 of 81) of programs did not.

Website Evaluation: Education and Recruitment Content

Sixty-five ARFWs were evaluated for the 10 education variables and 11 recruitment variables (Figures 56). The presence of a given variable ranged from less than 10% of websites (eg, interview dates) to greater than 90% of websites (eg, operative experience, program description).

Fellowship website evaluation: education variables.

Figure 5:

Fellowship website evaluation: education variables.

Fellowship website evaluation: recruitment variables. Abbreviations: AAHKS, American Association of Hip and Knee Surgeons; SFM, San Francisco Match.

Figure 6:

Fellowship website evaluation: recruitment variables. Abbreviations: AAHKS, American Association of Hip and Knee Surgeons; SFM, San Francisco Match.

Given a total of 21 education and recruitment variables, the mean±SD number of variables present on a fellowship website was 11.3±3.8, equating to a mean comprehensiveness score of 54.0% across all 65 websites. Fellowship websites contained anywhere from 1 to 17 of the 21 variables examined.

Comparison Based on Program Characteristics

Presence of Fellowship Website. Fellowship programs affiliated with an orthopedic residency program were significantly more likely to have a fellowship website than those not affiliated with an orthopedic residency program (91.8% vs 62.5%, P=.001) (Table 3). Similarly, fellowship programs that were ACGME accredited were more likely to have a fellowship website than those not ACGME accredited (100.0% vs 73.8%, P=.009). Geographic region, affiliation with a top 20 orthopedic hospital, or program size were not associated with presence or absence of a fellowship website.

Association of Program Characteristics With Presence of Fellowship Website for 81 Fellowship Programs

Table 3:

Association of Program Characteristics With Presence of Fellowship Website for 81 Fellowship Programs

Fellowship Website Comprehensiveness. There was no significant relationship between fellowship website comprehensiveness and affiliation with an orthopedic residency program, geographic region, top 20 orthopedic hospital, ACGME accreditation status, or program size (Table 4).

Association of Program Characteristics With Website Comprehensiveness for 65 Fellowship Websites

Table 4:

Association of Program Characteristics With Website Comprehensiveness for 65 Fellowship Websites

Discussion

With the tremendous emphasis on orthopedic subspecialization and health care’s growing reliance on technology, the accessibility and maintenance of fellowship websites for education and recruitment is essential. The current study was a cross-sectional analysis of the accessibility and content of ARFWs when accessed through 4 major online fellowship directories as well as Google. Of the 81 adult reconstruction fellowship programs identified, nearly 20% did not maintain a functional website.

In addition to the significant percentage of programs without a functional website, the content varied considerably among those that did have a website. Of the 21 education and recruitment variables assessed in this study, the average comprehensiveness score was 54% across the 65 fellowship websites. These findings highlight the significant room for improvement with respect to information on education and recruitment provided on program websites. Moreover, these findings are consistent with previously reported data demonstrating lack of comprehensiveness of fellowship websites across a variety of orthopedic subspecialties.8–14

The current study also found that fellowships affiliated with an orthopedic residency program and those that are ACGME accredited are significantly more likely to have an ARFW than those that are not. A possible explanation for this finding is that fellowships affiliated with a residency or ACGME accredited are more likely to be academic institutions that may be provided a website from the parent institution or may be provided the resources to develop one. This is in contrast to a program that only has an adult reconstruction fellowship without major academic affiliations, and may not have the resources to invest in development of a website. However, of the 65 ARFWs identified, the content comprehensiveness was not associated with orthopedic residency affiliation, ACGME accreditation status, geographic region, top 20 orthopedic hospital affiliation, or program size. These findings are consistent with data previously published by Hinds et al8 showing no difference in the content scores of orthopedic trauma fellowship websites with regard to those same variables other than program size.

Furthermore, there was substantial variability in the number of programs listed and in the presence of hyperlinks to ARFWs across the 4 fellowship directories. Of note, the AAHKS database was the most comprehensive, listing more programs than SFM, FREIDA, and AAOS. This is likely explained by the fact that AAHKS is the sponsor of the adult reconstruction match and, as a result, may be the preferred database for prospective applicants. However, AAHKS only had functional website links for 16% of programs. Therefore, given the program overlap and inconsistency of information across the 4 directories, the authors propose the use of a single fellowship directory administered by AAHKS. This directory should standardize the information it provides for each program, including listing of up-to-date ARFW hyperlinks, and encourage programs to develop and use a website that is updated regularly. This would facilitate the ability of prospective fellows to access information on programs more easily and from a single all-encompassing directory run by AAHKS.

There were several potential limitations to the current study. First, this analysis represented a cross-sectional analysis of the available online resources and did not account for the evolving nature of web-based information. Second, evaluation of fellowship website content was based merely on the presence of information and not the quality or accuracy of the information provided. Moreover, the perceived value of the variables chosen for assessment of educational and recruitment content may not truly represent the variables important to applicants. Future studies should include a survey of adult reconstruction applicants to determine what they deem essential information on fellowship websites when applying.

Conclusion

Overall, this study highlights the deficits in online resources available for adult reconstruction fellowship program information and the gap that exists between the current means of information sharing. The authors propose that steps be taken to standardize and centralize the delivery of information through a web-based platform to improve the accessibility and content of orthopedic fellowship websites. They also propose a single fellowship website directory with frequently updated content. This will improve program information and accessibility for potential fellowship applicants.

References

  1. Morrell NT, Mercer DM, Moneim MS. Trends in the orthopedic job market and the importance of fellowship subspecialty training. Orthopedics. 2012;35(4):e555–e560. doi:10.3928/01477447-20120327-13 [CrossRef] PMID:22495859
  2. Daniels AH, DiGiovanni CW. Is subspecialty fellowship training emerging as a necessary component of contemporary orthopaedic surgery education?J Grad Med Educ. 2014;6(2):218–221. doi:10.4300/JGME-D-14-00120.1 [CrossRef] PMID:24949124
  3. Hariri S, York SC, O’Connor MI, Parsley BS, McCarthy JC. Career plans of current orthopaedic residents with a focus on sex-based and generational differences. J Bone Joint Surg Am. 2011;93(5):e16. doi:10.2106/JBJS.J.00489 [CrossRef] PMID:21368070
  4. Chu LF, Young CA, Zamora AK, et al. Self-reported information needs of anesthesia residency applicants and analysis of applicant-related web sites resources at 131 United States training programs. Anesth Analg. 2011;112(2):430–439. doi:10.1213/ANE.0b013e3182027a94 [CrossRef] PMID:21081766
  5. Lagwinski N, Hunt JL. Fellowship trends of pathology residents. Arch Pathol Lab Med. 2009;133(9):1431–1436. doi:10.1043/1543-2165-133.9.1431 [CrossRef] PMID:19722750
  6. Mahler SA, Wagner M-J, Church A, Sokolosky M, Cline DM. Importance of residency program web sites to emergency medicine applicants. J Emerg Med. 2009;36(1):83–88. doi:10.1016/j.jemermed.2007.10.055 [CrossRef] PMID:18439790
  7. Gaeta TJ, Birkhahn RH, Lamont D, Banga N, Bove JJ. Aspects of residency programs’ web sites important to student applicants. Acad Emerg Med Off J Soc Acad Emerg Med. 12(1):89–92. doi:10.1197/j.aem.2004.08.047 [CrossRef]
  8. Hinds RM, Capo JT, Egol KA. Information on orthopedic trauma fellowships: online accessibility and content. Am J Orthop (Belle Mead NJ). 2017;46(5):E320–E325. PMID:29099889
  9. Mulcahey MK, Gosselin MM, Fadale PD. Evaluation of the content and accessibility of web sites for accredited orthopaedic sports medicine fellowships. J Bone Joint Surg Am. 2013;95(12):e85. doi:10.2106/JBJS.L.00785 [CrossRef] PMID:23783216
  10. Davidson AR, Murphy RF, Spence DD, Kelly DM, Warner WC Jr, Sawyer JR. Accessibility and quality of online information for pediatric orthopaedic surgery fellowships. J Pediatr Orthop. 2014;34(8):831–834. doi:10.1097/BPO.0000000000000217 [CrossRef] PMID:24840658
  11. Silvestre J, Guzman JZ, Skovrlj B, et al. The Internet as a communication tool for orthopedic spine fellowships in the United States. Spine J. 2015;15(4):655–661. doi:10.1016/j.spinee.2014.11.024 [CrossRef] PMID:25457471
  12. Hinds RM, Danna NR, Capo JT, Mroczek KJ. Foot and ankle fellowship websites: an assessment of accessibility and quality. Foot Ankle Spec. 2017;10(4):302–307. doi:10.1177/1938640016677811 [CrossRef] PMID:27837038
  13. Young BL, Oladeji LO, Cichos K, Ponce B. Content and accessibility of shoulder and elbow fellowship web sites in the United States. Iowa Orthop J. 2016;36:36–40. PMID:27528833
  14. Yayac M, Javandal M, Mulcahey MK. Accredited orthopaedic sports medicine fellowship websites: an updated assessment of accessibility and content. Orthop J Sports Med. 2017;5(1):2325967116683942. doi:10.1177/2325967116683942 [CrossRef] PMID:28203601
  15. Gu A, Lehman JD, Sardana A, Cohen JS, Richardson SS, Sculco PK. Adult reconstruction hip and knee fellowship program content and accessibility. J Arthroplasty. 2018;33(6):1630–1633. doi:10.1016/j.arth.2018.01.075 [CrossRef] PMID:29523444

Website Evaluation: Education and Recruitment Variables

Education VariableRecruitment Variable
Didactic instructionProgram description
Journal clubApplication or selection process
Meetings, courses, or academic conferences routinely attended by fellowsInterview dates SFM link
Research requirements or opportunitiesAAHKS link
Call responsibilitiesCurrent fellows
Rotation scheduleFaculty
Operative experience (eg, case volume, types of cases performed)Salary information Affiliated hospitals
Clinic experienceLocation description
Fellow evaluation criteria
Teaching responsibilities
Contact information (program director or coordinator)

Comparison of Fellowship Directories

Fellowship DirectoryNo.

Programs ListedPrograms With Website Link ProvidedPrograms With Website Link Functional
SFM796961
AAHKS811413
AMA FREIDA222119
AAOS542016

Association of Program Characteristics With Presence of Fellowship Website for 81 Fellowship Programs

CharacteristicNo.

Programs Without WebsitePrograms With WebsiteP
Orthopedic residency.001
  Affiliated4 (8.2%)45 (91.8%)
  Not affiliated12 (37.5%)20 (62.5%)
Geographic region.323
  Midwest2 (16.7%)10 (83.3%)
  Northeast7 (35.0%)13 (65.0%)
  South4 (13.8%)25 (86.2%)
  West3 (15.0%)17 (85.0%)
Top 20 orthopedic hospital.105
  Affiliated1 (5.6%)17 (94.4%)
  Not affiliated15 (23.8%)48 (76.2%)
Accreditation Council for Graduate Medical Education accreditation.009
  Accredited0 (0.0%)20 (100.0%)
  Not accredited16 (26.2%)45 (73.8%)
Program size.129
  Small (<3 fellows/y)14 (25.0%)42 (75.0%)
  Large (≥3 fellows/y)2 (8.0%)23 (92.0%)

Association of Program Characteristics With Website Comprehensiveness for 65 Fellowship Websites

CharacteristicNo.Mean±SD ComprehensivenessP
Orthopedic residency.131
  Affiliated4556.5%±16.5%
  Not affiliated2048.3%±20.8%
Geographic region.130
  Midwest1060.5%±12.1%
  Northeast1359.0%±14.7%
  South2554.3%±19.1%
  West1745.9%±20.1%
Top 20 orthopedic hospital.122
  Affiliated1759.1%±14.0%
  Not affiliated4852.2%±19.2%
Accreditation Council for Graduate Medical Education accreditation.336
  Accredited2057.4%±19.1%
  Not accredited4552.5%±17.7%
Program size.845
  Small (<3 fellows/y)4254.3%±19.1%
  Large (≥3 fellows/y)2353.4%±16.6%
Authors

The authors are from the Department of Orthopaedic Surgery (SAA, MM, AHD, EMC), Warren Alpert Medical School of Brown University (SAA, WE, AEME, MM, AHD, EMC), Providence, Rhode Island.

Mr Ahmed, Mr ElNemer, Dr Eltorai, Dr Mariorenzi, and Dr Cohen have no relevant financial relationships to disclose. Dr Daniels has received a grant from Orthofix and is a paid consultant for Orthofix, Stryker, Spineart, EOS, and Southern Spine.

Correspondence should be addressed to: Shaan A. Ahmed, BA, Department of Orthopaedic Surgery, Warren Alpert Medical School of Brown University, 222 Richmond St, Providence, RI 02903 ( shaan_ahmed@alumni.brown.edu).

Received: March 29, 2019
Accepted: July 24, 2019
Posted Online: August 06, 2020

10.3928/01477447-20200721-10

Sign up to receive

Journal E-contents