The following play a role in my medical decision-making: (1) my personal experience and preference; (2) what others have told me has worked for them; (3) articles or reports recommending a specific approach; (4) the established standard of practice (ie, what is currently “popular”); and (5) literature proving the best outcome with a specific technique. Regarding item 4, there are numerous examples in orthopedics of surgeons abandoning one procedure with an excellent reputation for another procedure more popular in the area where they practice.
Vexing questions in our daily practice include the following: How much attention do we pay to each of these items? Which one eventually carries the day? How reliable is the “literature?” Let us examine the last question. I am sure you could cite examples of prospective, randomized trials regarding the same subject with completely different results.
The truth is that the medical literature at large is unreliable. After an in-depth study, Ioannidis1 proved and explained that if the exact same protocol is used, a published series, experiment, or method will be reproduced in fewer than half of subsequent studies. This is why most published research findings are false. Furthermore, most true findings are not useful. According to Ioannidis,1 change and improvement are overdue. Subjectively, we already know that. However, we are sometimes reluctant to change our ways and would like to see more “evidence.”
The best, most reliable results are not necessarily published by the well-known “expert.” Plus, many experts change their methods completely with time. Further, we have come to rely on registries. These are not publications but rather de facto reports of collective results.
We should be wary of a single manuscript with definitive conclusions and should wait for more publications proving the same results. If conflicting results exist, we must pay attention to our own practice results and experience. We must absolutely not disregard the literature. Our quest should be to refine it and improve on its dismal track record.
Regarding the peer review process, journal editors most likely would not consider publishing a manuscript that had been rejected by a reviewer. I have performed well-planned experiments that have been rejected when submitted for publication, yet others who have performed the same experiments have been published. The entire peer review process needs to be revised and refined so that factual and truthful results are constantly produced. Sometimes, personal, biased opinions and criticisms are offered that do not necessarily speak to the validity of studies.
Finally, we must practice according to an integrated collection of reasons why we prefer a certain technique. The best procedure we can offer patients should be based on all of the items mentioned in the first paragraph of this editorial. This is a procedure that is an established standard of practice with at least some validation in the literature that we have experience with and can perform well. Also, we need to be brutally honest with ourselves; we must not shy away from reviewing our own results and objectively criticizing ourselves to ensure that we do not repeat bad results. We should be skeptical about the literature and should integrate it with our own vast training and experience.
I have just returned from a successful orthopedic conference. When presenting cases to either the audience (mostly experienced orthopedic surgeons in that field) or the expert panel, the speakers' approaches often differed widely. This clearly demonstrates that experience ultimately carries the day.
- Ioannidis JP. How to make more published research true. PLoS Med. 2014;11(10):e1001747. doi:10.1371/journal.pmed.1001747 [CrossRef]