While excellent clinical results have been seen with total knee replacement (TKR), extensive documentation exists in variations in outcomes due to factors such as hospital and surgeon volume. The hypothesis of this study was that statistically significant variation exists in the processes of care delivered to patients undergoing TKR at 3 affiliated hospitals.
Retrospective chart review was used to compare the quality of care delivered to a sample of patients from an academic medical center, public county hospital, and private community hospital. Two hundred twenty-four patients undergoing primary TKR were included. Quality of care was measured by determining adherence to a set of 31 evidence-based quality indicators created using the RAND/UCLA modified Delphi expert panel methodology. The overall rate of adherence to the quality indicators was 53% (95% confidence interval [CI], 52%-55%) for the 224 patients. There was a statistically significant difference between sites, with patients treated at the high-volume academic center demonstrating a 58% rate of adherence (95% CI, 56%-61%) compared to 50% (95% CI, 48-51%; P=.008) at the lower-volume public hospital and 52% (95% CI, 51%-54%; P=.03) at the lower-volume private hospital.
Further study is warranted to determine the extent of variation in the delivery of care and its relationship to variation in outcomes of care for patients undergoing TKR.
Total knee replacement (TKR) is one of the most commonly performed elective inpatient surgical procedures.1 The Medicare program alone received claims for 430,726 TKRs from 1998 to 2000.2 Consensus exists on the excellent outcomes and cost-effectiveness of TKR.3 However, a growing body of evidence documents variations between providers in the rates of common complications, including infection, thromboembolic disease, revisions, and mortality. This evidence includes previous studies by our research group and others that use administrative databases to document an association between hospital and surgeon volume and the outcomes of orthopedic surgical procedures, including TKR.4-10 These differences in outcome persist after adjusting for patient factors such as age, race, and insurance type that also have statistically significant associations with rates of mortality and complications. 5,6,9,10 These variations may be the result of other differences in the case-mix between providers, but they raise the concern that they are due to variations in the quality of care being delivered.
This study attempts to identify variations in the quality of care associated with TKR at 3 affiliated hospitals by measuring adherence to an explicit set of evidence-based quality measures. The quality of health care can be assessed based on structure (the attributes of the hospital and provider), process (the actual details of the care provided), or the outcomes of care.11 While many evidence-based process measures exist for general medical care, development of quality measures for surgical care has been limited.11 Many performance measures, such as those used by the Surgical Care Improvement Project and other quality improvement programs, have been designed for general surgical care, as opposed to orthopedic surgery. As a result, existing quality measures do not address aspects of care specific to the indications, technique, and postoperative care of patients undergoing TKR. This study addresses this relative lack of evidence-based measures for orthopedic surgery by using a comprehensive set of quality indicators that address the perioperative care of patients undergoing TKR.12 Adherence to these measures was compared at 3 affiliated hospitals with differing characteristics.
The value of this study is that it identifies variations in the adherence to indicators of high-quality care for TKR. This variation reinforces the need for further empirical testing in a larger, population-based sample to identify which components of high-quality care are prone to variation or are not being delivered optimally, making them logical targets in attempts to improve quality and decrease complication rates associated with TKR.
Materials and Methods
The 3 hospitals included in the study are affiliated with a single academic institution, the University of California, Los Angeles (UCLA) School of Medicine. The first site, UCLA Medical Center, is a high-volume academic medical center that performed >100 TKRs annually during the study period. Cases at this teaching hospital were staffed by 3 surgeons with full-time academic appointments who were assisted by resident physicians. Two of the 3 surgeons had a primary subspecialty in total joint replacement, and 1 had a primary subspecialty in sports medicine with a long-standing total joint replacement component to practice.
The second site, Santa Monica-UCLA Medical Center is a lower-volume community hospital affiliated with and owned by UCLA. The operating physicians were a separate group that included 2 physicians with full-time academic appointments. One of the 2 physicians at this site had a subspecialty in total joint replacement, and the second had a general orthopedic practice with a long-standing component of total joint replacement. This site performed <100 TKRs annually during the study period. Resident physicians did not routinely participate in care of the patients from this site.
The third site, Harbor-UCLA Medical Center, is a lower-volume public hospital supported by county funds located in an underserved area. This site is affiliated with UCLA but owned by Los Angeles County. This site performed <100 TKRs annually during the study period. The cases at this site were performed by 2 physicians with part-time academic appointments, both with a subspecialty in total joint replacement. Residents participated in the care of all patients at this site.
The orthopedic residency training programs at UCLA and Harbor-UCLA are independent with separate groups of residents. There is rotation of residents between the UCLA and Harbor-UCLA programs, but rotating residents are not assigned to services where they participate routinely in the care of patients undergoing total joint replacement. As a result, the patients at the UCLA and Harbor-UCLA sites were cared for by a different group of orthopedic surgery residents. Residents did not participate in the care of patients at the Santa Monica-UCLA site. Patients at each site were admitted to nursing units designated for orthopedic surgery patients. This designation was not specific for the subgroup of orthopedic patients undergoing total joint replacement.
Inclusion and Exclusion Criteria
Institutional Review Board approval was obtained for a review of patients undergoing TKR at all 3 hospitals affiliated with our institution beginning with the calendar year 2004. The International Classification of Diseases, 9th Revision (ICD9) diagnosis code (81.54) for TKR was used to identify a convenience sample of 75 consecutive patients in the Healthcare Financial Services database of each hospital. The ICD9 codes were also used to define exclusion criteria. Patients were excluded if they had diagnosis codes consistent with revision arthroplasty, infection, or pathologic fracture. Inflammatory arthritis was not an exclusion criterion. The ICD9 codes used to include and exclude patients are listed in Table 1.
Development of Indicators of Quality
The first step in evaluating quality of care was developing a set of quality indicators that served as the criteria for high-quality care. To accomplish this objective, a proposed set of quality-of-care indicators has been developed through a comprehensive literature search and structured interviews with expert clinicians.12 An expert panel of orthopedic surgeons was convened to rate the validity of these quality indicators using the RAND/UCLA modified Delphi methodology. Indicators were classified as valid by the panel based on the median panel rating and the amount of dispersion of panel ratings. An overview of the process to identify and rate quality indicators is shown in the Figure.
|Figure: Overview of the RAND/UCLA Appropriateness Method used to develop and validate quality of care indicators.
This modified Delphi method is iterative with 2 rounds of independent ratings by a group of expert panelists and a face-to-face group discussion between rounds.13,14 Each panelist has an equal weight in determining the final ratings. This RAND/UCLA methodology, which also includes an explicit synthesis of findings from the literature, has been shown to produce quality indicators that have face, construct, and predictive validity.15-19 The quality indicators developed include specific pre-, intra-, and postoperative processes of care rated as indicative of higher quality using the RAND/UCLA modified Delphi expert panel methodology.
Medical Records Abstraction
A combination of both electronic and paper chart review was performed for each of the eligible patients. The chart components reviewed included physician notes, nursing notes, ancillary service notes, results, and orders. The electronic medical records included inpatient notes during the patient’s stay; follow-up visits were unavailable. The inpatient paper chart and electronic medical records review included preoperative notes including history, physical examination, laboratory and imaging reports, and preoperative anesthesia and medical consultation notes. Additional outside office charts and nonhospital documents were not available to review. The chart reviews were performed by co-investigators not directly involved with the care of the patients.
Univariate analyses were used to calculate the marginal distributions of the patient demographics and outcome measures, including the central location (mean and median), variation (standard deviation), and minimum and maximum values. The bivariate relationship of the outcome measure (adherence to care) and the site of care, as well as patient covariates, were analyzed. When comparing continuous variables (adherence to standard care) across categorical variables (site), we used analysis of variance (ANOVA) to test the means of the continuous variable across the different levels of the categorical variables.20,21
Next, multivariate analyses were conducted using analysis of covariance (ANCOVA) to evaluate the joint effect of independent variables on the outcome of adherence rate. The independent variables included the site of treatment (UCLA, Harbor-UCLA, or Santa Monica-UCLA Medical Center), as well as patient-level characteristics including age, sex, race, and insurance type.
One hundred one candidate indicators of quality were identified in the 6 domains of preoperative processes of care, intraoperative processes, postoperative processes, implant selection and the use of new technology, privileging of hospitals and surgeons, and outcomes and comorbidity assessment. A total of 68 of the 101 indicators were rated as valid with statistical agreement. For the current proposed study, we identified the subset of 31 indicators that were potentially feasible for measurement using chart review. This set of 31 indicators includes quality measures that outline specific processes of pre-, intra-, and postoperative care rated as valid measures of higher quality with statistical agreement by the expert panel. The measures were specifically designed such that the lack of documentation of adherence was considered evidence of poor quality.
A total of 224 patients were included in the study, with 75 patients undergoing treatment at Harbor-UCLA and Santa Monica-UCLA Medical Center, and 74 patients at UCLA. One patient from the UCLA site was excluded due to inadequate documentation of demographic information. Mean patient age was 67 years (range, 21-92 years). Women made up 71% and men 29% of the patient sample. The patient population consisted of 53% white patients, 13% black patients, 28% Hispanic patients, and 6% Asian patients. There was a statistically significant difference in the study sites, with the low-volume public hospital having a younger patient population, a higher percentage of Hispanic patients, and a higher rate of patients covered by Medicaid or public insurance when compared to the high-volume academic center and low-volume private hospital (Table 2).
Analysis of Variation in Care Delivered
The mean rate of overall adherence to the quality measures was 53%, with a 95% confidence interval (CI) of 52% to 55%. There was a statistically significant difference (P<.05) in the mean overall rate of adherence between patients at the 3 study sites, with the low-volume public hospital (50%; 95% CI, 48%-51%) and low-volume private hospital (52%; 95% CI, 51%-54%) having a lower adherence rate to the quality indicators compared to the high-volume academic center (58%; 95% CI, 56%-61%) (Table 3). Multivariate analysis using ANCOVA demonstrated a persistent statistically significant difference with higher quality of care at the UCLA Medical Center site compared to both Harbor-UCLA (P=.008) and Santa Monica-UCLA Medical Center (P=.03) hospitals, after correcting for the factors of age, race, sex, and insurance type. Age, race, sex, and insurance type were not statistically significant independent predictors of percent adherence to the quality indicators in multivariate analysis.
Analysis of Care Delivered
The level of adherence to the 16 preoperative quality measures is shown in Table 4 for the overall sample and by study site. Chart review demonstrated rates of adherence to high-quality processes exceeding 70% of patients for documentation of surgical indications (indicator #2), radiographic indications (indicator #6), social history (indicator #10), preoperative risk assessment with anesthesia or internal medicine consultation (indicator #11), and adequate informed consent (indicator #14). There were low levels of adherence for fully documenting all components of the physical examination (indicator #4), the review of systems (indicators #8 and #9), and the use of formal preoperative education regarding postoperative recovery (indicator #16).
Table 5 shows the trend toward a higher level of adherence for the 5 intraoperative indicators. This includes following time-out protocols (indicator #18), appropriate discussion of anesthetic options (indicator #19), documentation in the operative notes of implants used as well as stability and range of motion (indicator #20), and operative times <3 hours from incision to closure (indicator #21). There was a lower rate of adherence to antibiotic protocols as the study period preceded some of the current monitoring efforts and recommendations endorsed by the expert panel.
For the postoperative period, there were high rates of adherence to thromboprophylaxis medication selection (indicator #27), mobilization on postoperative day 1 (indicator #26), use of transfusion triggers when indicated (indicator #30), and appropriate postoperative follow-up (indicator #31). At the time of the study, clinical care pathways (indicator #22), preoperative discharge planning (indicator #24), and formalized written discharge instructions (indicator #25) were not in routine use at the 3 study sites (Table 6).
This study identifies statistically significant variation in the delivery of care for patients undergoing TKR at 3 affiliated hospitals. Prior documentation exists of variations in the outcomes of TKR based on provider characteristics such as surgical volume. The results of this study suggest that there is also underlying variation in the actual processes of care delivered to patients. These results indicate the need for further examination of variation among a broader sample that includes hospitals from diverse practice settings and geographic locations. This will allow for the determination of whether variations in outcome among providers are linked directly to variations in processes of care, which can be targeted for improvement. This has the potential to create a model for translating advances in evidence-based medicine into improved outcomes for the many patients undergoing total joint replacement.
Several prior studies have attempted to examine the overall quality of care delivered to patients for a variety of medical conditions. McGlynn et al16 reviewed the care of a representative sample of patients and noted that the overall rate of adult patients receiving recommended care for a broad survey of medical conditions is low at 54.9%. Adherence to a limited set of 9 quality indicators specifically targeted to hip fracture and patients at risk of falling demonstrated a low rate of compliance at 22.8%.16 A similar study in pediatric patients noted that 46.5% of recommended care is delivered to children in the ambulatory care setting.19 The current study expands these methodologies to orthopedic surgery by examining the quality of care using a comprehensive set of quality measures in a sample of 224 patients undergoing TKR. Our results showed an overall rate of adherence in the current study of 53%, which falls within the range seen in prior studies. In addition to the rate of adherence to evidence-based processes of care, we also were able to demonstrate statistically significant variation among 3 affiliated hospitals in the overall quality of care.
The results of the current study raise concern that variation is likely to exist across providers on a broader scale, given that these findings were noted in hospitals that have some overlap in physician staff, are close geographically, and are affiliated with the same medical school. Further study is needed to examine the contribution of differences in hospital characteristics and patient populations in exacerbating disparities in the quality of care. We plan additional investigations to determine the level of variation in the quality of care using a more representative sample, including hospitals in different practice settings and geographic regions. Ultimately, the demonstration of variations in care is important because it may be the key to identifying correctable causes for variations in the outcomes of total joint replacement.
- Jha AK, Fisher ES, Li Z, Orav EJ, Epstein AM. Racial trends in the use of major procedures among the elderly. N Engl J Med. 2005; 353(7):683-691.
- Skinner J, Weinstein JN, Sporer SM, Wennberg JE. Racial, ethnic, and geographic disparities in rates of knee arthroplasty among Medicare patients. N Engl J Med. 2003; 349(14):1350-1359.
- Archibeck MJ, White RE Jr. What’s new in adult reconstructive knee surgery. J Bone Joint Surg Am. 2006; 88(7):1677-1686.
- Hervey SL, Purves HR, Guller U, Toth AP, Vail TP, Pietrobon R. Provider volume of total knee arthroplasties and patient outcomes in the HCUP-nationwide inpatient sample. J Bone Joint Surg Am. 2003; 85(9):1775-1783.
- SooHoo NF, Lieberman JR, Ko CY, Zingmond DS. Factors predicting complication rates following total knee replacement. J Bone Joint Surg Am. 2006; 88(3):480-485.
- SooHoo NF, Zingmond DS, Lieberman JR, Ko CY. Primary total knee arthroplasty in California 1991 to 2001: does hospital volume affect outcomes? J Arthroplasty. 2006; 21(2):199-205.
- Katz JN, Barrett J, Mahomed NN, Baron JA, Wright RJ, Losina E. Association between hospital and surgeon procedure volume and the outcomes of total knee replacement. J Bone Joint Surg Am. 2004; 86(9):1909-1916.
- Katz JN, Mahomed NN, Baron JA, et al. Association of hospital and surgeon procedure volume with patient-centered outcomes of total knee replacement in a population-based cohort of patients age 65 years and older. Arthritis Rheum. 2007; 56(2):568-574.
- Mahomed NN, Barrett JA, Katz JN, et al. Rates and outcomes of primary and revision total hip replacement in the United States Medicare population. J Bone Joint Surg Am. 2003; 85(1):27-32.
- Solomon DH, Losina E, Baron JA, et al. Contribution of hospital characteristics to the volume-outcome relationship: dislocation and infection following total hip replacement surgery. Arthritis Rheum. 2002; 46(9):2436-2444.
- Birkmeyer NJ, Birkmeyer JD. Strategies for improving surgical quality—should payers reward excellence or effort? N Engl J Med. 2006; 354(8):864-870.
- SooHoo NF, Lieberman JR, Farng E, Park S, Jain S, Ko CY. Development of quality of care indicators for patients undergoing total hip or total knee replacement. BMJ Qual Saf. 2011; 20(2):153-157.
- Fitch K, Bernstein SJ, Aguilar MD, et al. The RAND/UCLA Appropriateness Method User’s Manual. Santa Monica, CA: RAND; 2001.
- Shekelle P. The appropriateness method. Med Decis Making. 2004; 24(2):228-231.
- Higashi T, Shekelle PG, Adams JL, et al. Quality of care is associated with survival in vulnerable older patients. Ann Intern Med. 2005; 143(4):274-281.
- McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003; 348(26):2635-2645.
- Shekelle PG, Chassin MR, Park RE. Assessing the predictive validity of the RAND/UCLA appropriateness method criteria for performing carotid endarterectomy. Int J Technol Assess Health Care. 1998; 14(4):707-727.
- Liu H, Weiss RE, Jennrich RI, Wenger NS. PRESS model selection in repeated measures data. Computational Statistics & Data Analysis. 1999; (30):169-184.
- Mangione-Smith R, DeCristofaro AH, Setodji CM, et al. The quality of ambulatory care delivered to children in the United States. N Engl J Med. 2007; 357(15):1515-1523.
- Cochran WG. Some methods for strengthening the common chi-square tests. Biometrics. 1954; 10(4):417-451.
- Mantel N, Haenszel W. Statistical aspects of the analysis of data from retrospective studies of disease. J Natl Cancer Inst. 1959; 22(4):719-748.
Drs SooHoo, Krenek, and Eagan are from the Department of Orthopedic Surgery, University of California, Los Angeles, Dr Tang is from Harbor-UCLA Medical Center, and Dr McGlynn is from RAND Health, Los Angeles, California.
Drs SooHoo, Tang, Krenek, Eagan, and McGlynn have no relevant financial relationships to disclose.
This study was funded by the Orthopaedic Research and Education Foundation through a research grant.
Correspondence should be addressed to: Nelson F. SooHoo, MD, Department of Orthopedic Surgery, UCLA, 10945 Le Conte Ave, PVUB #3355, Los Angeles, CA 90095 (email@example.com).