The Orthopaedic In-Training Examination (OITE) has been administered by the American Academy of Orthopaedic Surgeons (AAOS) since 1963.1 The 4 goals of the OITE, as described by Mankin,2 are to: (1) help graduating residents evaluate the effectiveness of their orthopedic education, (2) allow residents to evaluate their orthopedic education in relationship to their peers, (3) provide orthopedic residency programs with a tool by which they can measure the education they provide to residents, and (4) evaluate the effectiveness of new educational techniques in orthopedics. The OITE is now administered to more than 4000 residents per year,3 and many residency programs and residents are placing increased importance on residents’ OITE scores.
Although the exact correlation between residents’ performance on the OITE and their pass rate on the American Board of Orthopaedic Surgery (ABOS) Part I examination is debatable,4 increasing evidence shows that residents who score well on the OITE have a higher first-time pass rate on the ABOS Part I examination than residents who score below the 30th percentile.5–7 The emergence of this correlation comes during a time when restricted resident work hours are causing many surgical residencies to decrease the amount of time spent on formal education during duty hours.8 Therefore, it may be no surprise that the amount of literature published on the OITE and the implications it may have on resident education have increased dramatically in recent years.9
Despite the large influx of recent literature and the increasing attention placed on the OITE, many aspects of the test have not been explored. One of the most important questions that remains unanswered is how residents can most efficiently study for the examination. Considering the restricted work hours and the apparently endless barrage of educational material available to residents today,10 it seems pivotal for residents to find and study sources that provide them with the most material that seems likely to be tested. Although publications have examined the sources that are most commonly cited within the OITE,9 the material that residents most commonly use to prepare for the OITE,3,9 and residents’ study habits for the OITE,5 no literature helps residents determine which review sources most commonly contain the answers to questions that have been tested on the OITE.
In an effort to address this question, this study was designed to determine which of 3 widely used orthopedic review sources (Miller’s Review of Orthopaedics, 5th edition5,11 [MRO], American Academy of Orthopaedic Surgeons Comprehensive Orthopaedic Review12 [COR], and www.orthobullets.com13 [OB]) contains the most answers to previously asked OITE questions. In addition, the time needed to find the answers in each source and the number of years between the publication date of the recommended readings for each question and the OITE administration date were examined.
Materials and Methods
The 3 review sources (MRO, COR, and OB) were selected based on previous studies listing them as common review sources.5,14,15 In addition, all 3 sources state explicitly that they are “designed to prepare orthopaedic surgeons for standardized exams, including the OITE and ABOS Part I,”13 “designed to facilitate studying for board examinations and the Orthopaedic In-Training Examination,”12 or “indispensable for exam preparation at all levels of orthopaedic training.”16
Standardizing the Search
All of the questions from the 2006 to 2010 OITEs were searched by 1 author (I.S.) to reduce bias. Each question was placed into 1 of 13 categories based on the category designations provided in the program directors’ report for each year. Of note, in 2009, the category titled Orthopaedic Science was renamed Basic Science and Tumors. To provide continuity with previous years’ categories and to better understand which questions were directly related to tumors and which were most related to basic science, questions in this study were categorized as either basic science or tumor.
The 2 books (MRO and COR) were used in a searchable electronic format to help standardize the search for correct answers between sources. The designation of yes or no to whether a source contained an answer can be subjective. To mitigate this subjectivity, a systematic approach was used. Key history, radiographic, and clinical and physical examination findings, along with critical words or phrases, for the question stems and answers were used for the electronic searches. To start, the question stem topic was found in each source and searched for an answer that matched the correct multiple-choice answer. If that approach did not provide the correct answer, the topic of each answer choice was searched in an effort to find the answer for the question. If that method failed, a next step was to search the sources for the correct answer directly and then determine whether the information would adequately answer the question was used. The index provided at the end of the text for each of the hard copy sources was used to assist with the answer searches. This stepwise approach was thought to best mimic the process used by most residents to find OITE answers within a review source. An example question can be found in Table 1.
Table 1: Example Question
A 12-minute time limit for finding the answer to each question was used for each source. This time limit was chosen for 2 reasons. First, if a resident spends 12 minutes reviewing each OITE question, it would equate to spending approximately 1 hour each week reviewing each year’s OITE. Second, 12 minutes was estimated to be the shortest time needed to find and briefly skim the recommended readings assigned to each OITE question for an answer.
Whether the answer for each question was found in each source was recorded. If the answer was found, the page number, section, or URL address was noted. The amount of time needed to find answers to each year’s OITE in each source was calculated. Finally, the publication date for the recommended readings associated with each OITE question was documented.
The proportion of questions in each OITE that were deemed to be represented in the given review sources were compared annually and by subject matter. Chi-square test was used to compare the different congruency outcomes for the sources across the years and question topics in each source. Statistical significance was set at a P value less than .05.
A total of 1358 questions were evaluated (questions from 2006–2008 thrown out during scoring were excluded). The 13 categories into which questions were placed included: basic science, foot and ankle, hand, hip and knee, medically related issues, pediatrics, rehabilitation, orthopedic disease, shoulder and elbow, spine, sports medicine, trauma, and tumor. The number and percentage of questions for each of the 13 topics are listed in Table 2. In total, OB had significantly more answers to questions (1350 [99%]) than did MRO (803 [60%]) and COR (830 [62%]) (P<.0001). Almost half (655 [49%]) of the questions had answers that were found in all 3 sources. The number of answers found in each source is shown Figure 1.
Table 2: Total Number of Questions Asked by Topic on the OITE From 2006–2010
Figure 1: Graph showing the number of questions that were answerable by the 3 sources. Abbreviations: COR, American Academy of Orthopaedic Surgeons Comprehensive Orthopaedic Review; MRO, Miller’s Review of Orthopaedics, 5th edition; OB, www.orthobullets.com.
By year, OB had significantly more answers to each OITE than did MRO and COR (P<.0001) (Table 3) and had significantly more answers to questions in all topics than did MRO or COR (P<.0001). More than 60% of the answers were found in all 3 sources for questions in orthopedic disease, pediatrics, and tumors, and less than 40% of the answers were found in all sources for questions in basic science, trauma, and medically related issues. The percent of answers for questions in each topic that test takers would not have been exposed to if they had only used certain sources is shown in Figure 2.
Table 3: Total Number of Answers Found in Each Source by Year
Figure 2: Percentage of answers for each topic that were found in the 3 sources. Abbreviations: COR; American Academy of Orthopaedic Surgeons Comprehensive Orthopaedic Review; MRO, Miller’s Review of Orthopaedics, 5th edition; OB, www.orthobullets.com.
Finding answers on OB was approximately 10 hours faster per year than in MRO and approximately 20 hours per year faster than in COR (Table 4).
Table 4: Amount of Time Needed to Find Answers to Each OITE Question by Year and Source
Approximately 11% of all recommended journal readings were published within 1 year of the OITE being administered. More recommended journal and book readings were published within 5 years of the OITE being administered than at any other time point (Table 5).
Table 5: Recommended Journal and Book Readings for the OITE Published Within Different Time Spans of the Referenced Question
There appears to be an increased focus on the OITE in recent years,15 in part because of the association between OITE scores and passing the ABOS Part I.1,5,6,17Still, with residents spending little time preparing for the examination3 and the need to optimize the time used for formal education because of work hours limits,18 it is important for residents and program directors to optimize the time used for OITE preparation. This study was designed to determine how many answers to previously asked OITE questions were found in 3 commonly used orthopedic review sources and the amount of time needed to find the answers within each source.
This study shows that OB contains significantly more answers to previously asked OITE questions than does MRO or COR (P<.0001). This finding is important because one of the few factors that correlates with increased resident scores on the OITE is reviewing previous OITE questions.5 In addition, it shows that finding answers on OB is faster than in MRO and COR. Miyamoto et al5 reported that review books were the most common resource used for OITE preparation. The current study shows that all review sources appear to be good sources of OITE information. Papp et al19 were able to find 65% of the answers to the pediatric questions for the 2002 to 2006 OITE in Orthopaedic Knowledge Update: Pediatrics 3.20 This percentage is similar to the percentage of answers that were found in the current study in MRO (60%) and COR (62%) but much less than the percentage of answers found on OB (99%).
The ability to be continually updated may be one of the reasons why OB had significantly more answers than did MRO and COR (P<.0001). As opposed to online sources, books typically take years to compose and publish, remaining static until a new edition is released. Another possible reason for OB having more answers is that it contains a question bank that is specifically focused on the content of previous OITEs, similar to the Self-Assessment Examinations published by the AAOS. The Self-Assessment Examinations are typically published on a yearly basis, allowing them to incorporate information from recent orthopedic literature that may be used on upcoming OITEs in an analogous manner to a frequently updated Web site.
The current authors found no previous literature examining the time needed to find OITE answers within different sources. This may be because time could vary significantly among individuals depending on their knowledge and methodology. The methods used in this study sought to provide a standardized approach that would not favor any source, providing general guidance on the amount of time needed to find answers in each source. These methods showed that finding answers in OB was much faster than finding answers in MRO and COR, suggesting it may be a more efficient source to find OITE-related information.
Previous studies have recommended that residents read general orthopedic journals to improve their OITE performance,5,10,15 while also showing that a major source of question material for the OITE examinations comes from sub-specialty journals.10,15,19,21–24 Reading all general and subspecialty orthopedic journals is unrealistic and is likely one of the reasons that review sources are so popular among residents.5 Similarly, although recommended readings are available for each OITE question, reviewing all of the recommended readings would take an exceptional amount of time. In addition, such readings do not necessarily correspond with the question’s answer, leading residents to have difficulty finding the answers within the recommended readings for some questions.
Recent literature has shown that between 55% and 75% of the recommended readings for the OITE questions are based on journal articles10,19,22–24 and that 10% of the recommended readings for some OITE topics are published within 1 year of the OITE being administered,21 with more than 50% being published within 5 years.23 These percentages agree with the the results of the current study because 72% of the recommend readings were for journal articles and approximately 50% of the recommended journal articles were published within 5 years of the OITE examinations; of these, more than 10% were published within 1 year of the OITE examination. The approximately 10% of OITE questions with recommended readings published within 1 year of OITE administration translates to more than 25 questions on an examination. Considering that the difference between scoring in the top 30% and the bottom 30% is approximately 50 questions, these questions from the most recent literature may have an effect on a test-taker’s percentile. Therefore, to maximize performance on these questions, residents may require a review source that can be updated at least every year.
One limitation of the current study is that it compared 2 books with a Web site. However, although their platforms and layouts were different, all 3 sources stated that they were designed for standardized test preparation,12,13,16 and it was believed that the 3 sources could be compared fairly. Another limitation of the study is that more answers to questions may have existed in each source and were not found despite standardizing the search methods and searching all materials in a standard fashion. It is also recognized that none of the 3 sources can be used to accurately predict which questions will be included in future OITEs. However, previous studies have shown that some diseases are repeatedly tested on the OITE,1,6,19,21,25 and it seems reasonable to assume that sources that increase exposure to these topics may assist in test preparation. With a new edition of MRO just published and a new edition of COR likely to be published soon, it is possible that these more current books may have contained more answers. Finally, the current study is specific to OITE preparation, and although some elements may be able to be applied to ABOS Part I, it is not intended to pertain to more general orthopedic education that requires the reading of journal manuscripts and textbooks.
This study shows that OB provides more answers to previously asked OITE questions and requires less time to find those answers than does the MRO or COR. The OITE itself recently has been converted from a traditional print format to its current DVD-ROM–based format to allow for the use of videos, higher-resolution pictures, and improved usability by the residents.26 Although developing a broad knowledge of orthopedics will continue to require residents to read and review orthopedic literature and textbooks, it may be time for orthopedic review materials to look at ways to update their formats to allow for more frequent updating and improved search capabilities. Such features may help residents with their OITE preparation.
- Frassica FJ, Papp D, McCarthy E, Weber K. Analysis of the pathology section of the OITE will aid in trainee preparation. Clin Orthop Relat Res. 2008; 466(6):1323–1328. doi:10.1007/s11999-008-0235-2 [CrossRef]
- Mankin HJ. The Orthopaedic In-Training Examination (OITE). Clin Orthop Relat Res. 1971; (75):108–116. doi:10.1097/00003086-197103000-00014 [CrossRef]
- LaPorte DM, Marker DR, Seyler TM, Mont MA, Frassica FJ. Educational resources for the Orthopaedic In-Training Examination. J Surg Educ. 2010; 67(3):135–138. doi:10.1016/j.jsurg.2010.04.004 [CrossRef]
- Carmichael KD, Westmoreland JB, Thomas JA, Patterson RM. Relation of residency selection factors to subsequent Orthopaedic In-Training Examination performance. South Med J. 2005; 98(5):528–532. doi:10.1097/01.SMJ.0000157560.75496.CB [CrossRef]
- Miyamoto RG Jr, Klein GR, Walsh M, Zuckerman JD. Orthopedic surgery residents’ study habits and performance on the Orthopedic In-Training Examination. Am J Orthop (Belle Mead NJ). 2007; 36(12):E185–E188.
- Crawford CH III, Nyland J, Roberts CS, Johnson JR. Relationship among United States Medical Licensing Step I, Orthopedic In-Training, Subjective Clinical Performance Evaluations, and American Board of Orthopedic Surgery examination scores: a 12-year review of an orthopedic surgery residency program. J Surg Educ. 2010; 67(2):71–78. doi:10.1016/j.jsurg.2009.12.006 [CrossRef]
- Klein GR, Austin MS, Randolph S, Sharkey PF, Hilibrand AS. Passing the Boards: can USMLE and Orthopaedic In-Training Examination scores predict passage of the ABOS Part-I examination?J Bone Joint Surg Am. 2004; 86(5):1092–1095.
- Gelfand DV, Podnos YD, Carmichael JC, Saltzman DJ, Wilson SE, Williams RA. Effect of the 80-hour workweek on resident burnout. Arch Surg. 2004; 139(9):933–940. doi:10.1001/archsurg.139.9.933 [CrossRef]
- Marker DR, LaPorte DM, Seyler TM, et al. Orthopaedic journal publications and their role in the preparation for the Orthopaedic In-Training Examination. J Bone Joint Surg Am. 2009; 91(suppl 6):59–S66. doi:10.2106/JBJS.I.00559 [CrossRef]
- Marker DR, Mont MA, McGrath MS, Frassica FJ, LaPorte DM. Current hand surgery literature as an educational tool for the Orthopaedic In-Training Examination. J Bone Joint Surg Am. 2009; 91(1):236–240. doi:10.2106/JBJS.H.00972 [CrossRef]
- Miller MD, ed. Review of Orthopaedics. 5th ed. Philadelphia, PA: Saunders; 2008.
- Lieberman J, ed. AAOS Comprehensive Orthopaedic Review. 1st ed. Rosemont, IL: American Academy of Orthopaedic Surgeons; 2009.
- Orthobullets. Content focus. http://www.orthobullets.com/content. Accessed March 1, 2012.
- Taylor BC, Fowler TT, Dimitris C. Achieving educational excellence: a strategic initiative to enhance orthopedic resident academic performance. J Surg Educ. 2011; 68(3):162–166. doi:10.1016/j.jsurg.2011.01.006 [CrossRef]
- Marker DR, Mont MA, Seyler TM, LaPorte DM, Frassica FJ. Current literature: an education tool to study osteonecrosis for the Orthopaedic In-Training Examination?Orthop Clin North Am. 2009; 40(2):299–304. doi:10.1016/j.ocl.2008.10.011 [CrossRef]
- Miller Review Course. The book. http://millerreview.org/?page_id=190. Accessed March 1, 2012.
- Benjamin R, Nyland J, Crawford CH III, Roberts CS, Johnson JR. Orthopaedic In-Training Examination performance: a nine-year review of a residency program database. South Med J. 2008; 101(8):791–796. doi:10.1097/SMJ.0b013e31817c0ba6 [CrossRef]
- Karam MD, Marsh JL. Does a trauma course improve resident performance on the trauma domain of the OITE?J Bone Joint Surg Am. 2010; 92(13):e19. doi:10.2106/JBJS.J.00368 [CrossRef]
- Papp DF, Ting BL, Sargent MC, Frassica FJ. Analysis of the pediatric orthopedic surgery questions on the Orthopaedic In-Training Examination, 2002 through 2006. J Pediatr Orthop. 2010; 30(5):503–507. doi:10.1097/BPO.0b013e3181df1560 [CrossRef]
- Abel MF, ed. Orthopaedic Knowledge Update: Pediatrics 3. Rosemont, IL: American Academy of Orthopaedic Surgeons; 2006.
- Srinivasan RC, Seybold JD, Salata MJ, Miller BS. An analysis of the Orthopaedic In-Training Examination sports section: the importance of reviewing the current orthopaedic subspecialty literature. J Bone Joint Surg Am. 2010; 92(3):778–782. doi:10.2106/JBJS.I.00466 [CrossRef]
- Lackey WG, Jeray KJ, Tanner S. Analysis of the musculoskeletal trauma section of the Orthopaedic In-Training Examination (OITE). J Orthop Trauma. 2011; 25(4):238–242. doi:10.1097/BOT.0b013e3181e59da9 [CrossRef]
- Taylor BC, Fowler TT. Analysis of the trauma section of the Orthopaedic In-Training Examination. Orthopedics. 2011; 34(7):e261–e266.
- Osbahr DC, Cross MB, Bedi A, et al. Orthopaedic In-Training Examination: an analysis of the sports medicine section. Am J Sports Med. 2011; 39(3):532–537. doi:10.1177/0363546510387492 [CrossRef]
- Farjoodi P, Khanna AJ, Marker DR, Frassica FJ. Evaluation of the Orthopaedic In-Training Examination: spine questions. J Surg Educ. 2010; 67(3):139–142. doi:10.1016/j.jsurg.2010.04.001 [CrossRef]
- Marsh JL, Hruska L, Mevis H. An electronic Orthopaedic In-Training Examination. J Am Acad Orthop Surg. 2010; 18(10):589–596.
|OITE 2010 Question #14
|Figures 14a and 14b show the radiographs of a 45-year-old farmer whose right arm was caught in a grain auger with a resultant open fracture of the proximal radius. The extensor carpi ulnaris and supinator muscles have been destroyed, and the posterior interosseous nerve has a 6-cm segmental loss distal to the bicipital tuberosity. After multiple surgical debridements, the radius is plated and the bone and soft tissue envelope go on to heal at 3 months. A complete posterior interosseous nerve palsy remains. What is the next most appropriate step in surgical reconstruction?
Neurotization of the radial nerve to the posterior interosseous nerve.
Wrist fusion with transfer of the flexor carpi radialis to the finger extensors.
Transfer of the pronator teres to the wrist extensors and the flexor carpi radialis to the finger extensors.
Transfer of the flexor carpi radialis to the wrist extensors, the flexor digitorum superficialis to the finger extensors, and the palmaris longus to the extensor pollicis longus.
Transfer of the flexor carpi radialis to the finger extensors and the palmaris longus to the extensor pollicis longus.
|Methods of searching:
|1st step: Searching for answers based on the question stem. This was a question seeking knowledge of how to treat a posterior interosseous nerve palsy with muscle loss. Namely, it appeared to be seeking information regarding the indications for tendon transfers and how those tendon transfers should be performed. Therefore, both the sections regarding posterior interosseous nerve palsy and tendon transfers in the review sources were found and reviewed to see if the answer could be determined. This would allow the reader to find the general concepts for each subject and determine if the answer could be deduced from these general concepts.
|2nd step: Each answer was specifically searched for in the electronic text. Search terms would include neurtotization of radial nerve, pronator teres transfer, and flexor carpi radialis transfe. If specific phrases could not be found, more general phrases (eg, flexor carpi radialis) would be searched with the search being directed at the area of the text that discusses the appropriate topic (in this case, tendon transfers).
|3rd step: The correct answer was identified as answer 5. This answer would then be searched more extensively, trying different combinations of words and phrases than had been searched before. In addition, this answer clearly focuses on which tendon transfers should be performed for a patient who lacks finger and thumb extension secondary to posterior interosseous nerve palsy. Using this information, a focused search would continue in the tendon transfer section.
Total Number of Questions Asked by Topic on the OITE From 2006–2010
||No. (%) of Questions
|Foot and ankle
|Hip and knee
|Medically related issues
|Shoulder and elbow
Total Number of Answers Found in Each Source by Year
||No. (%) of Answers
Amount of Time Needed to Find Answers to Each OITE Question by Year and Sourcea
Recommended Journal and Book Readings for the OITE Published Within Different Time Spans of the Referenced Question
||Published, No. (%)
||Within 1 y
||Between 1 and 5 y
||Between 6 and 10 y