patient-reported outcome measures are a valuable tool that can
give orthopedists important information, the results of these measures can be
called into question through a number of patient and surgeon variables. In
addition, new health care legislation, and advanced surgical techniques and
devices are changing the evolution of these measures.
William R. Beach, MD, an orthopedic surgeon at Tuckahoe Orthopaedics in
Richmond, Va., said that he uses patient-reported outcome measures (PROMs) as a
method to confirm whether a surgery was successful and advocates their use in
“Patient-specific outcomes, are the newer
wave,” Beach told Orthopedics Today. “If you consider
what is being reported in most articles in arthroscopy and sports medicine,
they are objective rating scales. The subjective scales are and should be used
beside them, not in place of them.”
Compared to other specialties, Gaurav Khanna, MD, noted that orthopedics
has been slow to adapt to PROMs. However, he said that standard
patient-reported outcome tests should be simple enough for patients to complete
“Orthopedics is one of those fields where it sort of lends itself
easily to patients reporting their own outcomes,” Khanna, an orthopedic
surgeon and researcher in La Jolla, Calif., told Orthopedics
Today. “Every field has an emphasis on learning how patients are
doing after their procedures, but I think orthopedics, because it is based
solely on motion, function and pain relief, lends itself easily to asking some
basic questions and assessing how well the patient is doing
William R. Beach, MD, said that questionnaires
to obtain patient-reported outcomes should be detailed, but not onerous.
Image: David Everette
In a 2011 study, Khanna assessed the accuracy of the patient-reported
American Knee Society and Oxford Knee Society scores compared to
clinician-assessed scores. Using the American Knee Society scale, he found that
the mean patient-reported scores for knee pain were four points lower than
clinician-assessed scores and mean patient-reported function subscores were 10
points lower than clinician-assessed scores. When the Oxford score was used,
Khanna found no significant differences between the patient and clinician
Although patients self-reported worse outcomes on the take-home American
Knee Society questionnaires, Khanna said one of the most interesting results
came when patients returned for follow-up appointments.
“The most significant part of that was the patients typically
reported a higher function — a better outcome — when they came back
for their follow-up appointment as opposed to when they were filling out the
forms in the privacy of their own home,” Khanna said. “Within 2
weeks, you would not expect such a dramatic increase, but many times they were
reporting eight or 10 points higher on their pain and function scores using the
American Knee Society Score.”
He concluded that the 2-week time period was too soon to see such
dramatic change, but provided one theory as to why patients were reporting
higher outcomes: they did not want to disappoint their surgeons.
Reducing patient bias
Reducing patient bias is also an issue with PROMs. This bias can be
limited with proper administration of patient questionnaires, according to
Karen M. Briggs, MBA, MPH, director of clinical research at the Steadman
Philippon Research Institute. To limit bias, she suggests that questionnaires
be administered digitally.
“When people collect outcomes on all their patients, [it] reduces
the selection bias. These scores are being tested to make sure they work over
different populations, so that again reduces the bias,” Briggs told
Orthopedics Today. “It is important that patients fill out
these forms on their own. These forms get filled out before they get to see the
physician, so you do not have as much bias as the physician placing on how the
patient how they are doing.”
A digital collection process also “eases the pain
of data collection” and frees staff members to tend to other patients
rather than collecting data, Briggs said. At her institute, patients fill out a
form digitally and those outcomes are referenced by physicians during the first
visit. The physicians fill out a digital form themselves, and the records are
collected in a database where the patient file is kept indefinitely.
Some studies have indicated that gender, age and
ethnicity may play a role in patient-reported outcomes.
In a study by Ageberg and colleagues, researchers analyzed 10,164
patients who underwent ACL reconstruction and filled out either the Knee Injury
and Osteoarthritis Outcome Score or EuroQol questionnaires. The investigators
found that female gender was a significant predicator of lower outcome scores.
However, the authors were unable to conclude exactly why women were predisposed
to reporting worse scores than men.
Similarly, a study conducted by Kenneth A. Egol, MD, and colleagues
discovered that, of 443 patients treated for distal radius fractures,
African-American and Latino patients reported worse outcome scores than
Caucasian patients at almost all follow-up visits. However, African-American
patients improved wrist function at 1-year follow-up when compared to either
Caucasian or Latino patients at that time interval. The investigators were
unable to conclude why ethnicity played a significant role in outcomes.
“We had some theoretical reasoning as to why we thought this might
be — it is probably a multifactorial sociodemographic type of [situation],
possibly a difference in patient expectations, possibly language or cultural
differences, but we really do not know for sure,” Egol, who is professor
and vice chairman of the of the Department of Orthopaedic Surgery at the
Hospital for Joint Diseases Langone Medical Center, told Orthopedics
because it is based solely on motion, function and pain relief, lends itself
easily to asking some basic questions and assessing how well the patient is
— Gaurav Khanna,
Obtaining patient-reported outcomes from children can prove challenging,
as investigators must ensure that patients fully understand questions and
correctly interpret their responses.
“For children, they have to be old enough to understand what the
question means,” Beach said. “If someone explains the questionnaire
to them, obviously that has to be consistent. I do not think that using the
reported outcomes is particularly difficult if you have confidence in the way
the study was set up or have confidence in what the study can tell you.”
Some surgeons use the Pediatric Outcomes Data Collection Instrument
(PODCI) to get subjective data from children. In a study by Kunkel and
colleagues, researchers concluded the instrument is a valid collection
mechanism for children and adolescents, noting it was responsive to changes
over time and was able to discriminate between populations in patients with
hand and wrist injuries.
Beach noted that worker’s compensation status can also influence
patient-reported outcomes. Such patients can be slow to recover in an effort to
gain money from employers during their leave from work and may feel less
satisfied with their care.
“Secondary gain is always a contributing factor to how you
improve.” Beach said. “Everybody that knows about worker’s
compensation knows that the faster you go back to work, the less time you are
off, the better you will do. With worker’s compensation [patients], you do
as good as you can, get them back to some kind of activity as soon as you can,
if it is safe, and try to minimize the amount of time they are out of
Various comorbidities have been shown in the literature to have a
negative impact on PROMs. Tashjian and colleagues found that patients with
rotator cuff tears had a mean of 2.07 comorbidities. The investigators found
that a greater number of comorbidities negatively correlated with
patient-reported outcomes. According to the study abstract, the authors
concluded that factors such as diabetes, smoking, worker’s compensation,
previous surgeries, revision surgery and duration of symptoms impacted
patient-reported outcomes for preoperative baseline pain, function and general
“There are so
many different scores, I think the problem becomes trying to compare and
contrast results of papers when different outcome measures are used.”
— Kenneth A. Egol, MD
Gauge patient satisfaction
Patient satisfaction also plays a pivotal role in PROMs, with pain
control and close patient monitoring listed at the top of concerns surgeons try
to address with their patients. However, Briggs said that sometimes, such as in
cases with patients suffering from arthritis, patient conditions can degrade
and a case may become more about managing pain rather than remedying it
“As people lose joint space and become more degenerative, some
procedures are less likely to provide relief of symptoms and increase
satisfaction. Satisfaction is tightly linked to symptoms and function, which
always relates to what kind of activity you do,” Briggs said.
Surgeons should reasonably set patient expectations before surgery, she
said. While achieving good motion in the joint is important, establishing a
timeline for returning to a desired activity can also increase a patient’s
John Nyland, DPT, SCS, EdD, ATC, CSCS, FACSM, told Orthopedics
Today that psychological factors have a lasting impact on patients’
desire to return to sport after ACL reconstruction. Patient factors such as
level of commitment, altered motivation, fear of re-injury and kinesiophobia,
curb nearly one-third of athletes’ returning to sports. Nyland uses
positive reinforcement and group rehabilitation to help combat these
“Modeling or vicarious learning — seeing others perform
sports-related movement tasks with success — can have a strong effect on
improving one’s confidence and self-efficacy,” Nyland said.
“This can be done through performing some rehabilitation activities in
groups or having patients observe and discuss an educational movement video, so
they have a better understanding of movement challenges and how they can be
successfully performed following surgery.”
Nyland noted that no widely accepted standardized outcome measurement
exists that evaluates combined concepts such as fear, anxiety, motivation and
commitment level, making it difficult to directly compare studies in this area.
“We need to develop a concise survey that measures these factors.
Even if normal knee range of motion, proprioception, strength and power are
re-established, if the patient is afraid or has low self-efficacy, it is
unlikely that they will return to their previous level of sports participation.
When this happens, patients may be dissatisfied and outcome scores will be
reduced despite the fact they had an otherwise successful ACL reconstruction
and rehabilitation,” he said.
Use in studies
Within orthopedics, PROMs range from the general to the specific. Some,
such as the SF-36, are widely used among subspecialties. Some measures, such as
the Single Assessment Numeric Evaluation (SANE) score, rate a patient’s
outcome on a 100-point scale using one question, while others such as the DASH
score contains 30 questions to assess physical function and symptoms.
“There are so many different scores, I think the problem becomes
trying to compare and contrast results of papers when different outcome
measures are used,” Egol said. “I am not saying one [scoring system]
is better than the other, but perhaps journals and societies that sponsor these
research trials should have standardized scores that everyone should be
utilizing so people can make comparisons between different studies.”
In addition, surgeons must decide which outcome measures to use in
studies, taking into account various demographic factors and whether to give
simple or complex questionnaires to a patient group. Another issue to consider
is the ceiling effect — the higher the number of questions on a form, the
more likely it is that patients will not pick extreme high or low answers.
Beach uses a custom “bubble” form that uses SANE, American
Shoulder and Elbow scale (ASES) and EuroQol (EQ-5D) outcome measures to assess
“The problem is if you make the questionnaires too long or complex,
nobody finishes it. Patients will not take the time to do six pages of
questions,” Beach said. “You want it to be as granular as you can be
without making it onerous.”
The exception is in the military, where soldiers are a “captive
audience” and can be asked to fill out detailed questionnaires. As
soldiers constitute an active population, results from military studies are
also “almost immediately applicable to everyone in the sports medicine
community,” Beach said.
“People in private practice have less control over how much time
[patients] are going to spend on a particular task like answering a
questionnaire,” he added.
PROMs and reimbursement
A growing issue regarding PROMs stems from new health care legislation.
The current administration hopes to use new policy guidelines and PROMs to
gauge which physicians will receive Medicare reimbursement. Raj Rao, MD,
professor and director of spine surgery at the Medical College of Wisconsin,
said these new laws are making physicians uneasy, as it not only changes how
they collect data, but how they interact with patients.
“I think the
biggest immediate problem is a lot of apprehension — a lot of uncertainty
— about exactly how the model is going to evolve… ”
— Raj Rao, MD
“I think the biggest immediate problem is a lot of apprehension
— a lot of uncertainty — about exactly how the model is going to
evolve, and exactly what physicians and physician offices will need to do in
order to document and determine patient-related outcomes in addition to their
normal daily volume of work,” Rao told Orthopedics Today.
Although the details of these proposals are evolving, the new laws will
reward physicians for implementing electronic health care recorders or be
penalized for failing to do so. Although many physicians are already recording
their patient data in this manner, there are some practices that may not have
the time — or the finances — to make this change.
“These are huge undertakings, to change the way we practice
medicine and prescribe medicine, [transitioning] from a paper system to an
electronic system,” Rao said. “In reality, what has happened is these
things have come down the pipe rapidly, and physician practices have been
required to adapt or adopt these new practices in a somewhat hurried fashion
without a clear understanding of what the ramifications of these various
electronic processes might be.”
Future outcome scores
As new surgical techniques and implant types develop, older outcome
measures, some of which were developed more than 25 years ago, will become
outdated and new measurements will take their place. One example of a score
that measures factors beyond traditional PROMs is the Forgotten Joint Score, or
FJS-12, which takes its name from the ideal situation of a total hip or total
knee replacement surgery that is so successful that patients “forget”
that they have an implant.
“In our opinion, this is the ultimate goal to ensure maximum
patient satisfaction,” Behrend and colleagues wrote in their study.
“This new concept integrates a variety of variables such as pain,
stiffness, function in activities of daily living, patients’ expectations,
patients’ activity levels and psychosocial factors.”
In an initial validation study of the FSJ-12, the authors noted that it
outperformed the WOMAC osteoarthritis index in several areas, including
discriminatory power and combating the ceiling effect, despite having roughly
half the number of questions as the WOMAC index.
Another new outcome measure comes from the Knee Society, replacing the
Knee Society Scoring System created in 1989. This new system differs from the
previous iteration through the creation of new measures to analyze the younger,
more diverse population that now undergoes total knee arthroplasty (TKA).
In a study by Noble and colleagues, the survey was given to 497 patients
and validated against the Knee Injury and Osteoarthritis Score and SF-12
scores. The authors wrote that the new system is a “valid instrument based
on surgeon and patient-generated data, adapted to the diverse lifestyles of
contemporary patients with TKA.” – by Jeff Craven
- Ageberg E, Forssblad M, Herbertsson P, Roos EM. Sex differences in
patient-reported outcomes after anterior cruciate ligament reconstruction: Data
from the Swedish knee ligament register. Am J Sports Med.
- Behrend H, Giesinger K, Giesinger JM, Kuster MS. The forgotten
joint as the ultimate goal in joint arthroplasty. Validation of a new
patient-reported outcome measure. J Arthroplasty. 2011. Published online
before print Oct. 12, 2011.
- Brand E, Nyland J. Patient outcomes following anterior cruciate
ligament reconstruction: The influence of psychological factors.
- Walsh MG, Davidovitch RI, Egol KA. Ethnic disparities in functional
and mental recovery following fracture healing. Paper #34. Presented at the
24th Annual Meeting of the Orthopaedic Trauma Association. Oct. 16-18. 2008.
- Khanna G, Singh JA, Pomeroy DL, Gioe TJ. Comparison of
patient-reported and clinician-assessed outcomes following total hip
arthroplasty. J Bone Joint Surg Am. 2011:19; 93(20):e117(1)-(7).
- Kunkel S, Eismann E, Cornwall R. Utility of the pediatric outcomes
data collection instrument for assessing acute hand and wrist injuries in
children. J Pediatr Orthop. 2011:31(7):767-772.
- Noble PC, Scuderi GR, Brekke AC, et al. Development of a new knee
society scoring system. Clin Orthop Relat Res. 2012:470(1):20-32.
- Tashjian RZ, Henn RF, Kang L, Green A. The effect of comorbidity on
self-assessed function in patients with a chronic rotator cuff tear. J Bone
Joint Surg Am. 2004;86(2):355-362.
- William R. Beach, MD, can be reached at Tuckahoe Orthopedics, 1501
Maple Ave., Richmond, VA 23226; 804-285-2300; email:
- Karen Briggs, MBA, MPH, can be reached at the Steadman Clinic
Philippon Research Institute, 181 West Meadow Dr., Suite 1000, Vail, CO 81657;
970-479-9797; email: email@example.com.
- Kenneth A. Egol, MD, can be reached at the Hospital for Joint
Diseases, New York University Longhorne Medical Center, 303 2nd Ave., New York,
NY 10003; 212-598-6137; email: firstname.lastname@example.org.
- Gaurav Khanna, MD, can be reached at the Scripps Clinic, Orthopedic
Surgery, 10666 N. Torrey Pines, MS 116, La Jolla, CA 92037; 800-780-1277;
- John Nyland, DPT, SCS, EdD, ATC, CSCS, FACSM, can be reached at the
Division of Sports Medicine, Department of Orthopaedic Surgery, University of
Louisville, 2301 South 3rd St., Louisville, KY 40292; 502-852-2782; email:
- Raj Rao, MD, can be reached at the Department of Orthopaedic
Surgery, Medical College of Wisconsin, 9200 W. Wisconsin Ave., Milwaukee, WI
- Disclosure: Beach, Briggs, Egol, Khanna, Nyland and Rao have
no relevant financial disclosures.
How much weight should be given to patient-reported outcomes compared
to clinician-assessed, objective outcomes?
Depends on the treatment objectives
It is increasingly recognized that traditional clinician-based outcome
measures need to be complemented by measures that focus on the patient’s
concerns in order to evaluate interventions and identify whether one treatment
is better than another. The amount of weight given to one form or another
depends on the specific treatment objective.
Consider the most common symptom in orthopedics: pain. There is no
clinical-based measure of this symptom, and yet, a common indication for many
orthopedic procedures is “pain relief.” On the other hand, an equally
important parameter, “function,” may be best objectively measured and
reported through a clinician-based outcomes measure (e.g., 3-meter walk test).
It is clear that to obtain the best overall picture of a patient’s
condition both clinician-based and patient-reported outcome measures are
critical to assessing success. There is no generic formula to advise about
relative importance of these in every situation. Their importance depends upon
the condition in question, availability of valid measurement instruments and,
most importantly, upon patient preferences for health outcomes. In the near
future, incorporating the role of patient expectations will be the new frontier
of outcomes research.
Michael Suk, MD, JD, MPH, FACS, is the
chairman of the department of orthopedic surgery at Geisinger Health System in
Disclosure: Suk has no relevant financial
Both outcomes have pros and cons
Both patient-reported outcomes and clinical assessment outcomes have
strengths and weaknesses. Certainly, in the past, orthopedic surgeons focused
more on clinically assessed outcomes. Over the last decade or so,
patient-reported outcomes have come to the forefront and have become an
important outcome tool to assess patient satisfaction after a certain
intervention. Patient-reported outcomes have the strength in that they directly
assess the patient’s perception of the outcome of the intervention.
Certainly, this is an important outcome measure when the patient is the one
receiving the outcome.
Patient outcomes can be influenced by many factors, not all of which may
be directly related to the intervention given. For instance, if the patient
undergoes a total knee replacement and has a pain-free knee with good range of
motion but was unhappy with the hospital food or the parking situation in the
hospital, it may affect the reported outcome after the intervention when in
fact the actual intervention was successful. Hence, care should be taken when
using patient-reported outcomes also, as there are a lot of variables that may
affect their results.
Clinical-assessed outcomes have the strength of using the
clinician’s knowledge and experience to choose what he or she feels are
important. There are many validated outcome measures in all areas of orthopedic
surgery, and many of those focus on what the surgeon feels are important
parameters for the patient (in range of motion, functional improvement and pain
relief). The problem with clinical-assessed objective outcomes is that they
are, of course, influenced when given to the patient by other factors other
than the direct outcome of the surgery. If patients socially like the
clinician, they may answer the questions differently than if they feel they did
not have a good relationship with the physician despite the success or lack of
success of that intervention.
Hence, at the end of the day, both patient-reported outcomes and
clinically assessed outcomes are important as long as one recognizes the
strengths and weaknesses of both tools.
Robert T. Trousdale, MD, is an
Orthopedics Today Editorial Board member and a professor of orthopedics
at the Mayo Clinic in Rochester, Minn.
receives royalties from DePuy, MAKO Surgical Corp. and Ortho Development.