Orthopedics

Feature Article 

Inter- and Intraobserver Reliability of the Radiographic Diagnosis and Treatment of Acromioclavicular Joint Separations

Matthew J. Kraeutler, BS; Gerald R. Williams, MD; Steven B. Cohen, MD; Michael G. Ciccotti, MD; Bradford S. Tucker, MD; Joshua S. Dines, MD; David W. Altchek, MD; Christopher C. Dodson, MD

Abstract

The management of acromioclavicular joint separations, in particular Rockwood types III and V, remains controversial. The purpose of this study was to investigate the observer reliability of shoulder surgeons when presented with the same cases of acromioclavicular joint separations. The authors retrospectively identified 28 patients who were diagnosed with a type III, IV, or V acromioclavicular joint separation. A PowerPoint presentation was compiled that contained an anteroposterior and axial radiograph from each patient prior to treatment. Radiographs were sent to surgeons, who diagnosed each injury according to the Rockwood classification and stated whether they recommended operative or nonoperative treatment for each patient.

Inter- and intraobserver reliability were calculated from the surgeons’ reviews. Repeat diagnoses were returned by 8 surgeons. A single-measure intraclass correlation coefficient (ICC) was used to determine interobserver reliability for the surgeons’ Rockwood classifications (ICC=0.602) and their decision to operate (ICC=0.469). Intraobserver reliability also was calculated for Rockwood classifications (ρ=0.694) and decision to operate (κ=0.366). Two (25%) of 8 surgeons stated that they would have used open and arthroscopic techniques for repairing the dislocations, whereas the remaining (75%) surgeons would have performed open techniques. Individual surgeons were consistent in their grading of acromioclavicular joint dislocations, but less observer agreement existed among the surgeons. Poor agreement among surgeons for the decision to operate indicates that this decision is heavily influenced by clinical factors and the radiographic classification.

Mr Kraeutler and Drs Williams, Cohen, Ciccotti, Tucker, and Dodson are from the Rothman Institute, Philadelphia, Pennsylvania; and Drs Dines and Altcheck are from the Hospital for Special Surgery, New York, New York.

Mr Kraeutler and Drs Williams, Cohen, Ciccotti, Tucker, Dines, Altchek, and Dodson have no relevant financial relationships to disclose.

The authors thank Charles L. Getz, MD; Matthew L. Ramsey, MD; Matthew Pepe, MD; and Luke S. Austin, MD, for serving as radiograph reviewers.

Correspondence should be addressed to: Matthew J. Kraeutler, BS, Rothman Institute, 925 Chestnut St, Philadelphia, PA 19107 (mjk8x@virginia.edu).

Abstract

The management of acromioclavicular joint separations, in particular Rockwood types III and V, remains controversial. The purpose of this study was to investigate the observer reliability of shoulder surgeons when presented with the same cases of acromioclavicular joint separations. The authors retrospectively identified 28 patients who were diagnosed with a type III, IV, or V acromioclavicular joint separation. A PowerPoint presentation was compiled that contained an anteroposterior and axial radiograph from each patient prior to treatment. Radiographs were sent to surgeons, who diagnosed each injury according to the Rockwood classification and stated whether they recommended operative or nonoperative treatment for each patient.

Inter- and intraobserver reliability were calculated from the surgeons’ reviews. Repeat diagnoses were returned by 8 surgeons. A single-measure intraclass correlation coefficient (ICC) was used to determine interobserver reliability for the surgeons’ Rockwood classifications (ICC=0.602) and their decision to operate (ICC=0.469). Intraobserver reliability also was calculated for Rockwood classifications (ρ=0.694) and decision to operate (κ=0.366). Two (25%) of 8 surgeons stated that they would have used open and arthroscopic techniques for repairing the dislocations, whereas the remaining (75%) surgeons would have performed open techniques. Individual surgeons were consistent in their grading of acromioclavicular joint dislocations, but less observer agreement existed among the surgeons. Poor agreement among surgeons for the decision to operate indicates that this decision is heavily influenced by clinical factors and the radiographic classification.

Mr Kraeutler and Drs Williams, Cohen, Ciccotti, Tucker, and Dodson are from the Rothman Institute, Philadelphia, Pennsylvania; and Drs Dines and Altcheck are from the Hospital for Special Surgery, New York, New York.

Mr Kraeutler and Drs Williams, Cohen, Ciccotti, Tucker, Dines, Altchek, and Dodson have no relevant financial relationships to disclose.

The authors thank Charles L. Getz, MD; Matthew L. Ramsey, MD; Matthew Pepe, MD; and Luke S. Austin, MD, for serving as radiograph reviewers.

Correspondence should be addressed to: Matthew J. Kraeutler, BS, Rothman Institute, 925 Chestnut St, Philadelphia, PA 19107 (mjk8x@virginia.edu).

Acromioclavicular joint separation is a common shoulder injury. Injuries to the acromioclavicular joint account for approximately 9% of all shoulder injuries.1 Since the Rockwood classification of acromioclavicular joint separations (types I–VI)2 was adopted, much discussion has occurred regarding the best treatment options for each separation type. Thus, it is important for surgeons to accurately diagnose these separations to guide treatment decisions.

Type I and II separations are essentially nondisplaced separations and are almost universally treated with nonoperative management. Some disagreement exists regarding the appropriate management of type III separations, which involve a rupture of the acromioclavicular and coracoclavicular ligaments,2 with the shoulder complex displaced inferiorly and the clavicle displaced superiorly (Figure). Heers and Hedtmann3 hypothesized that the variability of soft tissue lesions in type III lesions is the main reason why no standard treatment method exists for these separations. Although a trend exists toward nonoperative care for type III separations,4,5 this decision usually depends on the patient’s sport, throwing demands, time of year with respect to the athletic season of interest, arm dominance, and occupation.6 In addition, patients with persistent pain and other symptoms a few months after beginning nonoperative treatment are also candidates for surgical reconstruction.2 Most studies that have compared operative with nonoperative treatment for type III separations have found no significant difference in outcome between treatments.7–10

Anteroposterior radiographs showing Rockwood type III (A), IV (B), and V (C) acromioclavicular joint separations. Axial radiographs showing Rockwood type III (D), IV, (E), and V (F) acromioclavicular joint separations.

Figure: Anteroposterior radiographs showing Rockwood type III (A), IV (B), and V (C) acromioclavicular joint separations. Axial radiographs showing Rockwood type III (D), IV, (E), and V (F) acromioclavicular joint separations.

Type V separations are characterized by severe superior displacement of the distal clavicle (Figure). Operative treatment is typically recommended for these patients and includes various techniques.11–14 Distinguishing between type III and V separations can be difficult. As described by Rockwood,2 type III separations result in an approximately 100% displacement of the clavicle, whereas type V separations result in a displacement exceeding 100%, usually at approximately 300%. In addition, type III separations may be reduced in the clinic, whereas type V separations are irreducible due to penetration of the clavicle into or through the trapezius fascia.15

Type IV separations are usually treated operatively and are characterized as a severe form of type III separation with posterior displacement of the distal clavicle into or through the trapezius (Figure). Type IV separations are sometimes difficult to diagnose and require an adequate axillary view to determine posterior translation.

Despite these recommendations, the treatment rendered often varies significantly among surgeons. No reports in the literature illustrate the variability of surgical decision making among surgeons with regard to acromioclavicular joint separations. The purpose of the current study was to investigate the observer reliability in a group of fellowship-trained shoulder surgeons at 2 different academic centers. The authors hypothesized that inter- and intraobserver agreement for surgical management would be poor.

Materials and Methods

Institutional review board approval was obtained for this study. The authors retrospectively identified 28 patients who were diagnosed with a type III, IV, or V acromioclavicular joint separation between September 2007 and March 2011 by 1 of 7 surgeons based on standard radiographs, physical examination, and patient history. The Rockwood classification for each patient was mentioned in each surgeon’s chart notes following initial evaluation. Some of these patients were scheduled for acromioclavicular joint reconstructive surgery; the remaining patients continued with nonoperative treatment.

In April 2011, at least 1 month following initial diagnosis of all 28 patients, a PowerPoint (Microsoft, Redmond, Washington) presentation was compiled that contained an anteroposterior and axial radiograph of each patient before or during initial evaluation. Patient names and dates were removed from each radiograph. This presentation was prepared by a research fellow (M.J.K.) who was not associated with the original patient diagnoses. The order of patients was randomized so that physicians would be less likely to recognize any given radiograph. For each patient, the anteroposterior radiograph was shown first, followed by the axial radiograph. The presentation was sent to a group of fellowship-trained shoulder surgeons from 2 academic centers who (1) diagnosed each acromioclavicular joint dislocation according to the Rockwood classification, (2) stated whether they recommended operative or nonoperative treatment for each patient, and (3) stated their recommended operative method (arthroscopic vs open). Surgeons were blinded to the initial diagnosis.

Statistical analysis was performed using SPSS version 16.0 software (SPSS, Inc, Chicago, Illinois). Inter- and intraobserver agreement were calculated from the surgeons’ reviews. Interobserver reliability was calculated to understand the agreement between the different surgeons’ reviews of the radiographs. One numeric representation of the interobserver reliability (all reviewers) for each review was calculated using the intraclass correlation coefficient (ICC).

Intraobserver agreement was calculated based on the agreement of individual surgeons between their first and second reviews of each patient. Spearman’s rank correlation coefficient was used to determine the intraobserver agreement for the reviewing group as a whole based on the Rockwood classifications. A kappa statistic was measured to determine intraobserver agreement based on the decision to operate.

Results

Average patient age at surgery or initial evaluation was 43 years (range, 19–74 years). The difference in age for operative and nonoperative patients approached statistical significance (P=.07), with operative patients being younger. Twenty-two (79%) patients were men. Table 1 shows the original diagnoses and treatment decisions for each surgeon. Repeat diagnoses were returned by 8 surgeons, including 1 surgeon (J.S.D.) from a different institution. Mean time from initial to repeat diagnosis was 29 months (range, 1–43 months). A single-measure ICC was used to determine interobserver reliability for the surgeons’ Rockwood classifications (ICC=0.602) and their decisions to operate (ICC=0.469).

Initial Patient Diagnoses

Table 1: Initial Patient Diagnoses

Of the 8 surgeons who performed a review based on radiographs alone, 5 (S.B.C., C.C.D., C.L.G. [acknowledged reviewer], B.S.T., G.R.W.) were also involved in the initial diagnoses based on radiographs, physical examination, and patient history. These 5 surgeons accounted for 18 (64%) of the 28 radiographic sets. Spearman’s rank correlation coefficient (ρ) was used to determine intraobserver reliability for these 5 surgeons based on Rockwood classifications (ρ=0.694).

To determine intraobserver reliability for the decision to operate, a 2×2 contingency table compared initial (clinical) decisions and later (radiographic) decisions (Table 2). Initially, the surgeons decided to perform surgery on 14 (78%) of 18 patients. However, when reviewing these patients’ radiographs at least 1 month later, the same surgeons would have performed surgery on 11 (61%) of 18 patients (ρ=0.089). Overall, the surgeons agreed on 13 (72%) of 18 patients. Four (80%) of 5 disagreements involved an operative clinical decision changed to a nonoperative radiographic decision. Of these, 2 Rockwood classifications were changed from type V to type III, and 2 were changed from type III to type II (Table 3). One case occurred in which a surgeon did not perform surgery on a patient based on clinical findings but stated he would have performed surgery based on radiographs alone. This patient was diagnosed with a type V separation both times. The κ statistic for intraobserver reliability was 0.366, indicating a slight agreement between these 2 decision-making processes.

Treatment Distribution

Table 2: Treatment Distribution

Comparison of Clinical and Radiographic Decisionsa

Table 3: Comparison of Clinical and Radiographic Decisions

Two (25%) of 8 surgeons stated they would have used open and arthroscopic techniques for repairing the dislocations, whereas the remaining (75%) surgeons indicated they would have used only open techniques. More specific data, including graft type, fixation used, and postoperative rehabilitation, were not obtained.

Discussion

This study illustrates an overall lack of agreement regarding management among a group of orthopedic surgeons who regularly treat acromioclavicular joint separations operatively and nonoperatively. However, this study shows consistency among the surgeons in their own decision making based on the Rockwood classification system and some discrepancy in their decisions to operate.

The lack of consensus among surgeons could be due to the lack of agreement regarding classification, particularly between type III and V separations. Perhaps the greatest reason for this disagreement is that although type V separations are based on a clavicular displacement, usually at approximately 300%, surgeons may still make a type V diagnosis if this displacement exceeds 100%.16 For displacements close to 100%, it may be difficult to decide between a type III or V diagnosis. This may complicate a surgeon’s clinical decision making because type V separations are typically treated operatively, whereas type III separations are handled on a case-by-case basis.

Nevertheless, inconsistency exists among surgeons because the Rockwood classification system is primarily based on plain radiographs and may have limited interobserver reliability. Even with multiple radiographic views, it is possible that radiographic evaluation may be insufficient for an effective diagnosis of acromioclavicular joint separation by under- or overestimating the level of soft tissue injury.3 Furthermore, the decision to operate depends on more than radiographic analysis. Table 3 shows that 1 patient with a type V separation was treated nonoperatively and 3 patients with a type III separation were treated operatively. Patient age, method of injury, and involvement in sports are additional contributing factors in the decision to operate.

The high Spearman’s rank correlation coefficient showed that individual surgeons are fairly consistent in their grading of acromioclavicular joint dislocations using the Rockwood classification. Although the agreement between surgeons is slightly lower, it is still somewhat high. Low reliability values for the decision to operate show that this decision depends on more than radiographic assessment. Ultrasound3,17 and magnetic resonance imaging18 may be useful in further assessing these separations.

Strengths of this study include the number of surgeons who participated in the initial and repeat diagnoses. Initial patient diagnoses were well balanced between types III, IV, and V acromioclavicular joint separations. The limitations of this study were that the surgeons did not consistently document the Rockwood classification in their clinical notes, and several patients did not have both anteroposterior and axial radiographs available for review. Therefore, the current cases do not represent a consecutive series of all of the high-grade acromioclavicular joint separations treated during the study period. All of the cases that met inclusion criteria were included. Furthermore, not every physician ordered comparison views in the database of case examples; therefore, the authors elected not to include comparison views.

Conclusion

The purpose of this study was to determine a measure of consistency between surgeons in grading acromioclavicular joint separations. Based on inter- and intraobserver reliability values, individual surgeons are consistent in their grading of acromioclavicular joint dislocations, but less agreement exists among surgeons. Low reliability values for the decision to operate show that this decision depends on more than radiographic assessment, in particular, a physical examination and history.

References

  1. Mazzocca AD, Arciero RA, Bicos J. Evaluation and treatment of acromioclavicular joint injuries. Am J Sports Med. 2007; 35(2):316–329. doi:10.1177/0363546506298022 [CrossRef]
  2. Rockwood CA Jr, Williams GR Jr, Young DC. Disorders of the acromioclavicular joint. In: Rockwood CA Jr, Matsen FA III, , eds. The Shoulder. Philadelphia, PA: WB Saunders; 1998:483–553.
  3. Heers G, Hedtmann A. Correlation of ultrasonographic findings to Tossy’s and Rockwood’s classification of acromioclavicular joint injuries. Ultrasound Med Biol. 2005; 31(6):725–732. doi:10.1016/j.ultrasmedbio.2005.03.002 [CrossRef]
  4. Bishop JY, Kaeding C. Treatment of the acute traumatic acromioclavicular separation. Sports Med Arthrosc. 2006; 14(4):237–245. doi:10.1097/01.jsa.0000212330.32969.6e [CrossRef]
  5. Nissen CW, Chatterjee A. Type III acromioclavicular separation: results of a recent survey on its management. Am J Orthop. 2007; 36(2):89–93.
  6. Trainer G, Arciero RA, Mazzocca AD. Practical management of grade III acromioclavicular joint separations. Clin J Sport Med. 2008; 18(2):162–166. doi:10.1097/JSM.0b013e318169f4c1 [CrossRef]
  7. Galpin RD, Hawkins RJ, Grainger RW. A comparative analysis of operative versus nonoperative treatment of grade III acromioclavicular separations. Clin Orthop Relat Res. 1985; (193):150–155.
  8. Larsen E, Bjerg-Nielsen A, Christensen P. Conservative or surgical treatment of acromioclavicular dislocation: a prospective, controlled, randomized study. J Bone Joint Surg Am. 1986; 68(4):552–555.
  9. Press J, Zuckerman JD, Gallagher M, Cuomo F. Treatment of grade III acromioclavicular separations: operative versus nonoperative management. Bull Hosp Jt Dis. 1997; 56(2):77–83.
  10. Smith TO, Chester R, Pearse EO, Hing CB. Operative versus non-operative management following Rockwood grade III acromioclavicular separation: a meta-analysis of the current evidence base. J Orthop Traumatol. 2011; 12(1):19–27. doi:10.1007/s10195-011-0127-1 [CrossRef]
  11. DeBerardino TM, Pensak MJ, Ferreira J, Mazzocca AD. Arthroscopic stabilization of acromioclavicular joint dislocation using the AC graftrope system. J Shoulder Elbow Surg. 2010; 19(2 suppl):47–52. doi:10.1016/j.jse.2009.12.014 [CrossRef]
  12. Dimakopoulos P, Panagopoulos A, Syggelos SA, Panagiotopoulos E, Lambiris E. Double-loop suture repair for acute acromioclavicular joint disruption. Am J Sports Med. 2006; 34(7):1112–1119. doi:10.1177/0363546505284187 [CrossRef]
  13. Morrison DS, Lemos MJ. Acromioclavicular separation. Reconstruction using synthetic loop augmentation. Am J Sports Med. 1995; 23(1):105–110. doi:10.1177/036354659502300118 [CrossRef]
  14. Salzmann GM, Walz L, Buchmann S, Glabgly P, Venjakob A, Imhoff AB. Arthroscopically assisted 2-bundle anatomical reduction of acute acromioclavicular joint separations. Am J Sports Med. 2010; 38(6):1179–1187. doi:10.1177/0363546509355645 [CrossRef]
  15. Kim AC, Matcuk G, Patel D, et al. Acromioclavicular joint injuries and reconstructions: a review of expected imaging findings and potential complications [published online ahead of print May 26, 2012]. Emerg Radiol.
  16. Cote MP, Wojcik KE, Gomlinski G, Mazzocca AD. Rehabilitation of acromioclavicular joint separations: operative and nonoperative considerations. Clin Sports Med. 2010; 29(2):213–228. doi:10.1016/j.csm.2009.12.002 [CrossRef]
  17. Kock HJ, Jürgens C, Hanke J, Schmit-Neuerburg KP. Standardized ultrasound examination for classification of instability of the acromioclavicular joint [in German]. Unfallchirurgie. 1994; 20(2):66–71. doi:10.1007/BF02588145 [CrossRef]
  18. Alyas F, Curtis M, Speed C, Saifuddin A, Connell D. MR imaging appearances of acromioclavicular joint dislocation. Radiographics. 2008; 28(2):463–479. doi:10.1148/rg.282075714 [CrossRef]

Initial Patient Diagnoses

SurgeonInitial Rockwood ClassificationOperative/NonoperativeOpen/Arthroscopic
1VNonoperative
1IIINonoperative
2VOperativeOpen
2IIIOperativeOpen
2IIINonoperative
2IIINonoperative
2VOperativeOpen
2VOperativeOpen
3IIINonoperative
3VOperativeOpen
3IV/VOperativeOpen
3IIINonoperative
3IIINonoperative
4IIIOperativeOpen
4IIINonoperative
4IIIOperativeOpen
4IIIOperativeOpen
4IIIOperativeOpen
5IV/VOperativeArthroscopic
5IIIOperativeOpen
5VOperativeOpen
5VOperativeOpen
5IIIOperativeOpen
5VOperativeOpen
5IVOperativeOpen
6VOperativeArthroscopic
7VOperativeOpen
7VOperativeOpen

Treatment Distribution

Radiographic DecisionClinical Decision
Total
OperativeNonoperative
Operative10111
Nonoperative437
Total14418

Comparison of Clinical and Radiographic Decisionsa

Treatment DecisionRockwood Classification
ClinicalRadiographicClinicalRadiographic
NonoperativeOperativeVV
NonoperativeNonoperativeIIIIII
NonoperativeNonoperativeIIIII
NonoperativeNonoperativeIIIIII
OperativeNonoperativeVIII
OperativeNonoperativeIIIII
OperativeOperativeVV
OperativeOperativeVV
OperativeOperativeIV/VV
OperativeNonoperativeIIIII
OperativeOperativeVIV
OperativeOperativeVV
OperativeOperativeIIIIV
OperativeOperativeVV
OperativeOperativeIVIV
OperativeOperativeVV
OperativeOperativeVV
OperativeNonoperativeVIII

10.3928/01477447-20120919-16

Sign up to receive

Journal E-contents