We have significant concerns regarding the methodology used in the Khan et al.1 meta-analysis and the subsequent size of treatment effect reported. In particular, the result stating that “patients treated with dexamethasone implants for diabetic macular edema (DME) refractory to anti-VEGF [vascular endothelial growth factor] therapy gained a mean of four lines (20 logMAR letters)” appears inaccurate, as none of the included studies reported a treatment effect of the same magnitude.
There are several methodological weaknesses highlighted below.
Misinterpretation of the Standardized Mean Difference
Figure 3 of the paper in question plots the “standard difference in means,” but this effect measure is not clearly defined in the methods section. If it is the “standardized mean difference,” then this measure of effect is the size of the intervention effect in each study relative to the variability observed in that study and is reported in units of standard deviation.2 We suspect the authors have mistakenly assumed the value of 0.471 has units of logMAR (rather than units of standard deviation) and hence have incorrectly estimated a value of 23 logMAR letters as the mean visual acuity gain achieved.
Study Inclusion and Handling
We question the wisdom of combining results from such diverse study designs.3 The report would have benefited from more detailed reporting of design characteristics of the included studies, the data used to calculate the measure of effect, and justification for the judgements on the risk of bias.
Some studies also appear to be included incorrectly. Using the published references, the authors have included in their meta-analysis two studies4,5 where one is a subanalysis of the other. Another is from the GENEVA group evaluating efficacy of dexamethasone (Ozurdex; Allergan, Dublin, Ireland) treating macular edema associated with branch or central retinal vein occlusion,6 and another is a study of four children treated for diseases such as Coats' disease, macular telangiectasia, and uveitis.7
The authors reported “all patients were treated for recalcitrant diabetic macular edema with at least six prior treatments of intravitreal anti-VEGF therapy.” They reported “the total number of patients studied was 3,859.” On review of the included studies, we struggle to reach this number and precisely say how many injections patients received.
In view of the above methodological limitations, we have concerns with the conclusions drawn.
1Charles Hennings, MBBS, MEng
2Jennifer Evans, BA (Cantab), MSc, PhD
1Hemal Mehta, MA, FRCOphth
1Royal Free London NHS Foundation Trust
2International Centre for Eye Health
London School of Hygiene and Tropical Medicine
- Khan Z, Kuriakose RK, Khan M, Chin EK, Almeida DRP. Efficacy of the intravitreal sustained-release dexamethasone implant for diabetic macular edema refractory to anti-vascular endothelial growth factor therapy: Meta-analysis and clinical implications. Ophthalmic Surg Lasers Imaging Retina. 2017;48(2):160–166. doi:10.3928/23258160-20170130-10 [CrossRef]
- Cochrane Library. 220.127.116.11 The standardized mean difference. Cochrane Database of Systematic Reviews. http://handbook-5-1.cochrane.org/chapter_9/9_2_3_2_the_standardized_mean_difference.htm. Accessed March 21, 2018.
- Cochrane Library. 9.5.3 Strategies for addressing heterogeneity. Cochrane Database of Systematic Reviews. http://handbook-5-1.cochrane.org/chapter_9/9_5_3_strategies_for_addressing_heterogeneity.htm. Accessed April 23, 2018.
- Yoon YH, Boyer DS, Belfort R Jr, et al. Long-term efficacy and safety of dexamethasone intravitreal implant in phakic and pseudophakic eyes with diabetic macular edema. Investigative Ophthalmology & Visual Science. 2014;55:1779–1779.
- Boyer DS, Yoon YH, Belfort R Jr, et al. Three-year, randomized, sham-controlled trial of dexamethasone intravitreal implant in patients with diabetic macular edema. Ophthalmology. 2014;121(10):1904–1914. doi:10.1016/j.ophtha.2014.04.024 [CrossRef]
- Haller JA, Bandello F, Belfort R Jr, et al. Randomized, sham-controlled trial of dexamethasone intravitreal implant in patients with macular edema due to retinal vein occlusion. Ophthalmology. 2010;117(6):1134–1146.e3. doi:10.1016/j.ophtha.2010.03.032 [CrossRef]
- Lei S, Wai-Ching L. Efficacy and safety of dexamethasone intravitreal implant for refractory macular edema in children. Can J Ophthalmol. 2015;50(3):236–241. doi:10.1016/j.jcjo.2015.01.007 [CrossRef]
Thank-you for taking the time to bring to our attention the comments about our previously published study.1 We value the critical appraisal, and below are our responses to the queries put forth.
With respect to standardized means, we recognize that the use of units with a standardized mean difference is debatable. Nonetheless, even in the absence of logMAR units, using the “rule of thumb for effect sizes” method, the results (0.471 with a range of 0.32 to 0.64) fall well into the “moderate effect size” category.2 Our study's conclusion shows a moderate vision gain and therefore remains valid.
In noting our approach to study inclusion, the reviewers undoubtedly appreciate that designing a meta-analysis for a relatively rare clinical scenario is always challenging. The inclusion criteria were kept broad to the most possible numbers of studies to be analyzed. The consequence is high heterogeneity, and this is well-addressed in the discussion section of our paper. We mention this multiple times, and it supplants our notion to best model the meta-analysis so as to be applied to real world patients and outcomes.
In critiquing prior treatment, our paper does not refer to the total number of injections patients received. The total number presented represents the cumulative number of study participants reported when considering all studies. This is an important distinction because it provides the most robust — and thus clinically relevant — analysis to be performed. As evident in recent publications of patients with recalcitrant diabetic macular edema, there is large debate regarding what constitutes this “recalcitrant” designation; moreover, the possibility of undertreatment may confound outcomes. It is for this very reason that our meta-analysis was designed with the strictest adherence to accepted meta-analysis protocols, and we have thoroughly detailed this in our methods section.
We appreciate the opportunity to respond and are grateful for the interest our study has garnered.
Zainab Khan, MD
Montreal, Quebec, Canada