Orthopedics

Feature Article 

Surgeons Cannot Predict Pilon Fracture Outcomes Based on Initial Radiographs

Mark J. Gage, MD; Daniel Mascarenhas, BA; Dimitrius Marinos, BA; Michael A. Maceroli, MD; Brent T. Wise, MD; Suneel B. Bhat, MD; G. David Potter, MD; Gerard P. Slobogean, MD, MPH; Marcus F. Sciadini, MD; Christopher T. Lebrun, MD; Jason W. Nascone, MD; Theodore T. Manson, MD; Nathan N. O'Hara, MHA; Robert V. O'Toole, MD

Abstract

This study sought to determine (1) whether surgeons can accurately predict functional outcomes of operative fixation of pilon fractures based on injury and initial postoperative radiographs, (2) whether the surgeon's level of experience is associated with the ability to successfully predict outcome, and (3) the association between patients' demographic and clinical characteristics and surgeons' prediction scores. A blinded, randomized provider survey was conducted at a level I trauma center. Seven fellowship-trained orthopedic traumatologists and 4 orthopedic trauma fellows who were blinded to outcome reviewed data regarding 95 pilon fractures in random order. Injury ankle radiographs, initial postoperative fixation radiographs, and brief patient histories were assessed. Midterm follow-up functional outcome scores obtained a mean 4.9 years after surgery were available for all patients. Main outcome measures were Pearson correlation coefficient-assessed functional outcomes and surgeon-predicted outcomes. A mixed-effect model determined the association between patients' characteristics and surgeons' prediction scores. Minimal positive correlation was observed between functional outcomes and prediction scores. No difference was noted between the attending and fellow groups in prediction ability. When surgeons' prediction confidence level was greater than 1 SD above the mean confidence level, correlation between functional outcome and prediction improved, although poor correlation was still observed. AO/OTA type 43C fractures, high-energy mechanisms, and older patient age were characteristics associated with lower prediction scores. Surgeons had poor ability to predict functional outcomes of patients with pilon fractures based on injury and initial postoperative radiographs, and level of experience was not associated with ability to predict outcome. [Orthopedics. 2020; 43(1): e43–e46.]

Abstract

This study sought to determine (1) whether surgeons can accurately predict functional outcomes of operative fixation of pilon fractures based on injury and initial postoperative radiographs, (2) whether the surgeon's level of experience is associated with the ability to successfully predict outcome, and (3) the association between patients' demographic and clinical characteristics and surgeons' prediction scores. A blinded, randomized provider survey was conducted at a level I trauma center. Seven fellowship-trained orthopedic traumatologists and 4 orthopedic trauma fellows who were blinded to outcome reviewed data regarding 95 pilon fractures in random order. Injury ankle radiographs, initial postoperative fixation radiographs, and brief patient histories were assessed. Midterm follow-up functional outcome scores obtained a mean 4.9 years after surgery were available for all patients. Main outcome measures were Pearson correlation coefficient-assessed functional outcomes and surgeon-predicted outcomes. A mixed-effect model determined the association between patients' characteristics and surgeons' prediction scores. Minimal positive correlation was observed between functional outcomes and prediction scores. No difference was noted between the attending and fellow groups in prediction ability. When surgeons' prediction confidence level was greater than 1 SD above the mean confidence level, correlation between functional outcome and prediction improved, although poor correlation was still observed. AO/OTA type 43C fractures, high-energy mechanisms, and older patient age were characteristics associated with lower prediction scores. Surgeons had poor ability to predict functional outcomes of patients with pilon fractures based on injury and initial postoperative radiographs, and level of experience was not associated with ability to predict outcome. [Orthopedics. 2020; 43(1): e43–e46.]

Pilon fractures are challenging injuries to successfully treat and are associated with some of the poorest outcomes of any extremity fracture type. Several contributors to functional outcomes after open reduction and internal fixation (ORIF) of these fractures have been previously hypothesized. Fracture type and quality of postoperative reduction have been highlighted as 2 critical contributors to functional outcome in several cohort studies.1–4

The goal of this study was to determine whether fellowship-trained orthopedic traumatologists and orthopedic trauma fellows can accurately predict midterm functional outcomes after ORIF of pilon fractures based primarily on their assessment of the injury and initial postoperative radiographs. The primary hypothesis was that a strong positive association exists between outcome scores predicted by orthopedic surgeons based on basic clinical information, radiographs, and midterm functional outcome scores of patients with pilon fractures. The secondary hypothesis was that more experienced traumatologists have a stronger positive association compared with orthopedic trauma fellows. The authors also sought to determine the association between the patients' demographic and clinical characteristics and the surgeons' prediction scores and confidence levels.

Materials and Methods

This study was approved by the internal review board at the current authors' institution. A search of the billing database for all operatively treated pilon fractures from 2007 through 2015 was conducted and yielded a total of 823 patients. The patients then were screened and included in the study if they were 18 years or older at the time of surgery, had undergone definitive treatment of a pilon fracture with ORIF, and had their definitive surgery performed at least 18 months before being contacted. Patients were excluded if they had sustained ipsilateral foot or ankle injury, were treated surgically at an outside facility, were non-English speaking, underwent eventual amputation of the affected extremity, had fractures that did not involve the plafond articular surface (AO/OTA type 43A), had their pilon fracture treated by means other than internal fixation, or had inadequate injury and postoperative radiographic imaging.

Patients who met the inclusion criteria (n=295) were contacted by phone to complete a questionnaire administered by trained research assistants who were blinded to participants' radiographs and clinical outcomes. The questionnaire used was the American Academy of Orthopaedic Surgeons (AAOS) Foot and Ankle Questionnaire, which is a widely accepted and validated outcome measure that scores pain, stiffness, and function. Ninety-five patients agreed to complete the functional score (Table 1). Patients were predominantly male (70%) with a mean age of 46.2 years (SD, 22.1). Closed injuries comprised 81% of the fractures in the study; one-fourth of the fractures were classified as AO/OTA type 43B and three-fourths were classified as 43C. The mean functional outcome score of the patients was 64.3 at an average of 4.9 years after surgery, suggesting overall poor function.

Patient and Injury Characteristics (N=95)

Table 1:

Patient and Injury Characteristics (N=95)

Eligible surgeons were sent an electronic examination that contained 95 cases including patient radiographs and clinical vignettes (Figure 1). Both injury and initial postoperative radiographs were provided. Participants were blinded to patient-identifying information, and cases were reviewed in randomized order. The clinical information included the patient's age and sex, the mechanism of injury, and whether the fracture was open or closed. Information about perioperative complications was not included. Based on this information, the surgeons were asked to predict functional outcome scores on a scale of 0 through 10 and to provide a confidence level for their prediction on a scale of 0 through 10, with 10 being the highest function and the highest confidence. Each participating surgeon was briefed on the components of the AAOS Foot and Ankle Questionnaire to prepare for prediction based on that score.

Sample screenshot from survey.

Figure 1:

Sample screenshot from survey.

Of the 13 orthopedic surgeons who were contacted to participate, 11 completed the study. Two-thirds of the respondents were attending surgeons, and the remaining respondents were completing a 1-year orthopedic traumatology fellowship. The number of years in practice of the attending surgeons ranged from 4 to 17.

To control for variation of each surgeon's prediction, the prediction scores were normalized for each respondent using overall mean and SD. This allowed for more accurate comparison between a study participant who had a large range of prediction scores throughout the survey and a study participant who had a smaller range. Pearson correlation coefficients were calculated using a pairwise model to assess the direction and strength of the linear relationship between patients' functional outcomes and surgeons' predicted outcomes. Comparisons of Pearson correlation coefficients among subgroups that included the individual surgeon's level, the surgeon's training level, and confidence levels were conducted using Fisher r to z transformation.5

A mixed-effect model was used to determine the association between the patient's characteristics presented in the survey and the surgeon's prediction score. All possible determinants, including fracture classification (AO/OTA type 43B vs 43C), mechanism of injury (high energy vs low energy), decade of age, sex, and fracture type (open vs closed), were included as main effects in the model. Each respondent was assigned a unique code that was included as a random effect in the model. Model estimates are presented as the mean decrease in the surgeon's prediction score associated with each determinant relative to the stated reference level. All statistical analyses were performed using JMP Pro Version 13 (SAS Institute, Cary, North Carolina).

Results

The mean surgeons' prediction score was 4.6 of 10 (SD, 2.1), with a mean confidence level of 6.4 of 10 (SD, 1.6). The authors' analysis determined a minimal positive correlation (r=0.11; 95% confidence interval [CI], 0.05–0.17; P<.001) between the functional outcome scores and the surgeons' normalized prediction scores. The Pearson correlation coefficient of surgeon respondents ranged from 0.05 to 0.28, with no significant difference among the surgeons (P=.11). No difference was noted between the attending group and the fellow group regarding association between prediction scores and functional outcome scores (r=0.08 vs r=0.16; P=.21). When a surgeon's confidence in a prediction was greater than 1 SD above the mean confidence, the correlation between prediction score and functional score was significantly improved compared with predictions with less confidence (r=0.26 vs r=0.09; P=.04).

When assessing the association between patients' characteristics and surgeons' prediction scores, fracture classification, type of fracture, and patient age had significant influences. AO/OTA type 43C fractures were associated with a 0.98-point decrease (95% CI, −1.26 to −0.69) in predicted score compared with AO/OTA type 43B fractures. High-energy injuries were associated with a 0.67-point decrease (95% CI, −1.00 to −0.35) in predicted scores compared with low-energy injuries. For every 10 years in increased patient age, surgeons' prediction score was reduced by 0.18 of a point (95% CI, −0.26 to −0.09) (Table 2).

Patient Characteristics Associated With Decrease in Predicted Scorea

Table 2:

Patient Characteristics Associated With Decrease in Predicted Score

Regarding confidence scores, surgeon's level of training was the only observed variable associated with level of confidence. On average, the attending surgeons had a point increase in their confidence scores (95% CI, 0.86–1.25) compared with the orthopedic trauma fellows (P<.0001).

Discussion

Pilon fractures are complex injuries with a multitude of factors that can contribute to functional outcome. Although it is possible that the quality of reduction and the severity of osseous injury play critical roles in midterm function, experienced traumatologists were unable to successfully predict outcomes when relying solely on those factors. In some ways, this should not be too surprising considering existing evidence that other clinical factors, including medical comorbidities, socioeconomic factors, and the nature of the cartilage damage, might play important roles in clinical outcomes after plafond injuries.2,6–8 Those factors were not available to the surgeons in the current study. It is possible that predicted outcomes would have been more accurate if the additional information had been provided to the participating surgeons; however, the results indicate that surgeons cannot determine outcomes based on injury and initial postoperative radiographs alone.

The results of this study should be interpreted with a knowledge of the potential limitations. As with many studies of this type, selection bias regarding which patients had follow-up and were included in the sample is possible and might have changed the results. The patients included in the study were all treated definitively with ORIF. As a result, it is possible that more severe soft-tissue injuries that were not suitable for internal fixation were excluded from this group. It also is possible the studied cohort represented less severe extremity injuries considering that these patients would have been excluded if they had ipsilateral fractures of the extremity or if they went on to fusion or amputation.

In addition, the generalizability of this study is limited. The participating surgeons were 7 experienced orthopedic trauma surgeons with an overall average of 9 years of clinical experience who practiced at a high-volume level I trauma center. Their experience and volume influence their opinions and insights differently when predicting functional outcomes compared with other surgeons who have different levels of experience. It also is possible that prediction results could differ with the incorporation of more participants in the study. Of note, computed tomography has become increasingly more important in the evaluation of pilon fractures. One could argue the surgeons would have been able to better predict outcomes if computed tomography scans had been available.

This study is the first to describe the lack of relationship between surgeon-predicted and actual functional midterm outcomes after ORIF of pilon fractures. The inability of experienced surgeons to predict outcomes further emphasizes the challenge in treating these injuries and illustrates that other factors, including the extent of soft-tissue injury, can drive functional outcomes. Even at the outlying confidence scores greater than 1 SD from the mean, correlation between predicted and actual function was still poor.

Further investigation is needed to better understand these injuries and the various clinical factors that influence their outcome. Even with modern techniques, pilon fractures continue to be devastating injuries with poor functional outcomes.4,6,9,10 This is consistent with the midterm quality of function found in the study participants. It is clear that much room for improvement exists in the care of these patients. Ultimately, as the critical factors that drive outcome are better defined, the ability to successfully predict results, educate patients, and maximize function will improve significantly. Clinicians can incorporate these findings into their practices by understanding that they should not have high confidence in their ability to predict outcomes based on injury and initial postoperative radiographs.

Conclusion

The data from this study show that surgeons have poor ability to accurately predict functional outcomes after ORIF of pilon fractures based on injury and initial postoperative radiographs. Increased level of experience does not result in more successful outcome prediction. The characteristics of fracture classification, fracture type, and patient age influence prediction scores. Although higher surgeon confidence in predictions can result in improved correlation with functional outcomes, overall predictive ability is still poor.

References

  1. Teeny SM, Wiss DA. Open reduction and internal fixation of tibial plafond fractures: variables contributing to poor results and complications. Clin Orthop Relat Res. 1993;292:108–117. https://doi.org/10.1097/00003086-199307000-00013 PMID: doi:10.1097/00003086-199307000-00013 [CrossRef]
  2. Marsh JL, Weigel DP, Dirschl DR. Tibial plafond fractures: how do these ankles function over time?J Bone Joint Surg Am. 2003;85(2):287–295. https://doi.org/10.2106/00004623-200302000-00016 PMID: doi:10.2106/00004623-200302000-00016 [CrossRef]12571307
  3. Harris AM, Patterson BM, Sontich JK, Vallier HA. Results and outcomes after operative treatment of high-energy tibial plafond fractures. Foot Ankle Int. 2006;27(4):256–265. https://doi.org/10.1177/107110070602700406 PMID: doi:10.1177/107110070602700406 [CrossRef]16624215
  4. Sommer C, Nork SE, Graves M, Blauth M, Rudin M, Stoffel K. Quality of fracture reduction assessed by radiological parameters and its influence on functional results in patients with pilon fractures: a prospective multicentre study. Injury. 2017;48(12):2853–2863. https://doi.org/10.1016/j.injury.2017.10.031 PMID: doi:10.1016/j.injury.2017.10.031 [CrossRef]29079366
  5. Diedenhofen B, Musch J. cocor: a comprehensive solution for the statistical comparison of correlations. PLoS One. 2015;10(3):e0121945. https://doi.org/10.1371/journal.pone.0121945 PMID: doi:10.1371/journal.pone.0121945 [CrossRef]25835001
  6. Sirkin M, Sanders R, DiPasquale T, Herscovici D Jr, . A staged protocol for soft tissue management in the treatment of complex pilon fractures. J Orthop Trauma. 1999;13(2):78–84. https://doi.org/10.1097/00005131-199902000-00002 PMID: doi:10.1097/00005131-199902000-00002 [CrossRef]10052780
  7. Pollak AN, McCarthy ML, Bess RS, Agel J, Swiontkowski MF. Outcomes after treatment of high-energy tibial plafond fractures. J Bone Joint Surg Am. 2003;85(10):1893–1900. https://doi.org/10.2106/00004623-200310000-00005 PMID: doi:10.2106/00004623-200310000-00005 [CrossRef]14563795
  8. Liporace FA, Yoon RS. Decisions and staging leading to definitive open management of pilon fractures: where have we come from and where are we now?J Orthop Trauma.2012;26(8):488–498. https://doi.org/10.1097/BOT.0b013e31822fbdbe PMID: doi:10.1097/BOT.0b013e31822fbdbe [CrossRef]22357091
  9. Sands A, Grujic L, Byck DC, Agel J, Benirschke S, Swiontkowski MF. Clinical and functional outcomes of internal fixation of displaced pilon fractures. Clin Orthop Relat Res. 1998;347:131–137. https://doi.org/10.1097/00003086-199802000-00016 PMID: doi:10.1097/00003086-199802000-00016 [CrossRef]
  10. Boraiah S, Kemp TJ, Erwteman A, Lucas PA, Asprinio DE. Outcome following open reduction and internal fixation of open pilon fractures. J Bone Joint Surg Am. 2010;92(2):346–352. https://doi.org/10.2106/JBJS.H.01678 PMID: doi:10.2106/JBJS.H.01678 [CrossRef]20124061

Patient and Injury Characteristics (N=95)

CharacteristicValue
Sex, No. (%)
  Male66 (70)
  Female29 (30)
Age, mean (SD), y46.2 (22.1)
Fracture type, No. (%)
  Closed77 (81)
  Open18 (19)
Fracture classification, No. (%)
  AO/OTA 43B25 (26)
  AO/OTA 43C70 (74)
Injury mechanism, No. (%)
  High energy79 (83)
  Low energy16 (17)
Functional outcome score, mean (SD)64.3 (22.1)

Patient Characteristics Associated With Decrease in Predicted Scorea

CharacteristicMean Difference in Predicted Score95% CIP
Fracture classification
  AO/OTA 43BReference (0.00)
  AO/OTA 43C−0.98−1.26 to −0.69<.0001
Fracture type
  ClosedReference (0.00)
  Open−0.67−1.00 to −0.35<.0001
Age, each additional decade−0.18−0.26 to −0.09<.0001
Sex
  FemaleReference (0.00)
  Male−0.22−0.50 to 0.06.13
Mechanism
  Low energyReference (0.00)
  High energy−0.19−0.54 to 0.17.29
Authors

The authors are from the R Adams Cowley Shock Trauma Center, Department of Orthopaedics, University of Maryland School of Medicine, Baltimore, Maryland.

Dr Gage, Mr Mascarenhas, Mr Marinos, Dr Maceroli, Dr Wise, Dr Bhat, Dr Potter, Dr Lebrun, Dr Manson, and Mr O'Hara have no relevant financial relationships to disclose. Dr Slobogean is a paid consultant for Zimmer Biomet and Smith & Nephew. Dr Sciadini is a paid consultant for Stryker Orthopaedics and Globus Medical. Dr Nascone is a paid consultant for Zimmer Biomet and Smith & Nephew, receives royalties from Coorstek and DePuy Synthes, and holds stock in Imagen. Dr O'Toole has been a paid consultant for Smith & Nephew and Imagen and receives royalties from Coorstek.

The authors thank Dori Kelley, MA, for professional manuscript editing.

Correspondence should be addressed to: Robert V. O'Toole, MD, R Adams Cowley Shock Trauma Center, Department of Orthopaedics, University of Maryland School of Medicine, 22 S Greene St, Rm T3R62, Baltimore, MD 21201 ( rotoole@umoa.umm.edu).

Received: February 19, 2018
Accepted: January 02, 2019
Posted Online: November 26, 2019

10.3928/01477447-20191122-04

Sign up to receive

Journal E-contents