Diabetic retinopathy (DR) is a common complication of diabetes mellitus (DM) and the leading cause of blindness among working-age people. Diabetic macular edema (DME) and proliferative DR are the major causes of visual loss in DR. Early detection and prompt interventions have been proven to be efficient in previous studies.1–3
The pathologic changes caused by diabetes mellitus occur to most cell types of the retina.4 The morphological features of the inner retinal vessels are mostly used for evaluation of the disease due to its visibility. The severity of DR can be evaluated according to the International Clinical Diabetic Retinopathy Disease Severity Scale, the Airlie House classification system.5 A more detailed and accurate scoring system has been introduced by the Early Treatment Diabetic Retinopathy Study (ETDRS) and applies to clinical research; however, this remains a hierarchical evaluation. The development of computational image processing makes accurate quantitative lesion evaluation possible.6,7
Focal or grid laser photocoagulation has been applied for the treatment of DME for several decades.8 The efficacy of the intravitreal injections of anti-vascular endothelial growth factor inhibitors (VEGFs) has been confirmed by a range of randomized, control trials. Their effect on DR has also been reported, but not quantitatively.9,10 Their effect on the progression of DR has not come to conclusion yet. Several randomized, controlled trials have reported less DR worsening, even with the improvement in patients receiving intravitreal anti-VEGFs.11,12 A quantitative measurement method of lesions of DR could provide more dependable evidence for conclusion.
With the implementation of machine learning algorithms, the present study team developed software focusing on the segmentation and quantitative evaluation of DR changes including microaneurysm, intraretinal hemorrhage, and hard exudate.13 The software achieved an area under the receiver operating characteristic curve of 0.925 on a subset of a Kaggle dataset, and 0.960 on Messidor for the detection of DR images comparing with human experts. It also achieved an F1-score of 0.924 with a sensitivity 0.995 and precision 0.863 on DIARETDB1 using the connected component-level validation for DR lesions. The details of the development and validation of our software was previously published.14,15
Conbercept (KH902; Chengdu Kanghong Biotechnology Co. Ltd., Chengdu, China) is a recently developed anti-VEGF drug. Similar to aflibercept (Eylea; Regeneron, Tarrytown, NY), conbercept is an engineered protein that contains the extracellular Domain 2 of VEGF receptor 1 and extracellular Domains 3 and 4 of VEGF receptor 2 fused to the Fc portion of human immunoglobulin G1.16
In the present study, our team applied the validated machine learning algorithms to automatically detect, locate, and quantify the intraretinal hemorrhage and hard exudate lesions in color photography images of DME patients with intravitreal conbercept treatment.
Patients and Methods
Clinical Data and Imaging Protocol
A retrospective analysis was conducted to explore the intraretinal hemorrhage and hard exudate lesion change during the course of the intravitreal conbercept treatment in patients with DME.
A total of 19 patients from our clinic with clinically significant DME receiving intravitreal conbercept treatment and monthly visits for longer than 1 year were pooled in for analysis.
The inclusion criteria mainly included the following: (1) Center-involved macular edema on optical coherence tomography (OCT) and a mean central retinal thickness of 320 μm or greater using Spectralis (Heidelberg Engineering, Heidelberg, Germany); (2) visual acuity letter score in the study eye of less than 73 and greater than or equal to 24; (4) Type 1 or type 2 diabetes; and (5) age younger than 75 years and 18 years or older.
The exclusion criteria mainly included: (1) Macular edema secondary to ocular conditions other than diabetes mellitus (eg, retinal vein or arterial occlusion, retinal detachment, macular hole, choroidal neovascularization, or uveitis); (2) myopia of 8.0 diopters or greater in the study eye; (3) panretinal photocoagulation treatment for the study eye during the study course; (4) previous history of anti-VEGF treatment for DME in the study eye; (5) previous history of intraocular surgery in the study eye within 6 months before the study or intraocular surgery taken during the course of the study; or (6) poor glycemic control (HbA1c ≥ 8%).
The treatment protocol was one initial injection of conbercept followed by as-need injections according to the patients' visual and morphological outcomes in the monthly follow-up visit. The retreatment criteria followed the DRCR.net anti-VEGF retreatment algorithm.17 Briefly, monthly injections followed by continued injections at monthly intervals until vision and edema were no longer improving or can no longer improve (best-corrected visual acuity [BCVA] of 79 letters or better or central foveal thickness of 320.0 μm or less). Re-injection was at the investigator's discretion if edema recurred or worsened and was recommended if there was edema to treat.
Monthly visit was required for each enrolled patient. At each visit, refraction and visual acuity measurement, a dilated eye examination, and OCT were performed.18 The modified seven-field fundus images were obtained at baseline and at the third, sixth, and twelfth month visit, whereas the modified two-field fundus images were obtained at the other monthly visits. All color fundus images were taken from a TRC-50-DX (Topcon, Tokyo, Japan) with a Nikon D300 camera (Nikon, Tokyo, Japan). All OCT images were taken by a Heidelberg Spectralis OCT.
The study was approved by the institutional review board and ethics committee of Shanghai General Hospital. All participants provided written informed consent.
Automated Software Using Convolutional Neural Networks for Retinal Hemorrhage and Hard Exudate Lesion Detection
Our team developed algorithms based on combined neural networks for DR detection. The algorithms jointly learn the features and classifiers from data and achieve a significant improvement on detecting DR images and its typical lesions comparing with single neural network structure. The details of the development and validation of our software were previously reported and published.14,15
Color fundus photography images were analyzed by our automated software for intraretinal hemorrhage and hard exudate lesion segmentation and area measurement. Differences in the data obtained were assessed using paired sample t-test. A P value less than .05 was considered statistically significant. All statistical analyses were performed using SPSS version 21.0 software (IBM Corp., Armonk, NY).
A total of 19 subjects with DME and DR were included in the present study according to the inclusion and exclusion criteria. The mean age of the subjects was 60.6 years old (standard deviation = 8.7). Nine out of the 19 subjects were female. The participants received a mean of 6.8 injections of conbercept during the 12-month follow-up period.
Modified seven-field or three-field digital color fundus images for each monthly visit were available for the included 19 patients. There were 3,496 color fundus images pooled in for intraretinal hemorrhage and hard exudate lesion area software analysis. Figure 1 is an example of the image processing results.
Image segmentation results of the automated algorithms. (A) The original image. (B) Intraretinal hemorrhage segmentation. The hemorrhage lesion pixels were marked in purple. (C) Hard exudate segmentation. The exudate lesion pixels were marked in green.
Change in BCVA
The mean ETDRS BCVA of the enrolled patients increased from 57.1 ± 12.7 letters to 67.1 ± 13.7 letters from baseline to the 12-month visit. There was a mean of 10.0 letters gained from the baseline to the last visit (Figure 2A).
Best-corrected visual acuity (BCVA) and central foveal thickness change from baseline to 12 months. Vertical bars are ±1 standard error of the mean. (A) An average 10.0 BCVA letters gained was observed from the baseline to the 12-month visit (P < .05). (B) An average of 143.4 μm decrease of central foveal thickness was observed from the baseline to the 12-month visit (P < .05).
Change in Central Foveal Thickness From OCT
The mean central foveal thickness at baseline was 459.9 μm ± 127.5 μm. The mean central foveal thickness was 316.5 μm ± 53.0 μm at the 12-month visit, which decreased by 143.4 μm when compared with baseline (Figure 2B).
Area Change of Intra-retinal Hemorrhage Lesions
The mean total area of intraretinal hemorrhage in the study eye in the seven fields at baseline was 5.656 ± 1.176 mm2 at baseline, 2.438 ± 0.976 mm2 at the third month visit, 2.901 ± 0.521 mm2 at the sixth month visit, and 2.122 ± 0.582 mm2 at the twelfth month the end of the study visit. The area of intraretinal hemorrhage was decreased and reduced by 62.49% from baseline to the end of the study (P < .0001; Figure 3A). The monthly area change of intraretinal hemorrhage in the macular flied (Field 2) is presented in Figure 3B.
The area changes of intraretinal hemorrhage lesions over time. (A) Total area of intraretinal hemorrhage lesions in all seven fields at baseline, Month 3, Month 6, and Month 12. (Area changes comparing with baseline: P < .0001 at Month 3, P < .0001 at Month 6, and P < .0001 at Month 12. (B) The area of intraretinal hemorrhage lesions in Field 2 at each monthly visit.
Area Change of Hard Exudate Lesions
The mean total area of hard exudates in the study eye in all seven fields at baseline was 2.549 ± 0.776 mm2 at baseline, 2.233 ± 0.576 mm2 at the 3-month visit, 2.710 ± 0.621 mm2 at the 6-month visit, and 1.473 ± 0.564 mm2 at the 12-month end-of-study visit. The area of hard exudates slightly increased by 6.3% at the 6-month visit (P = .4849) and subsequently decreased by 41.1% at the 12-month (P < .0001) when compared with the first visit (Figure 4A). The monthly area change of hard exudates lesion in Field 2 is presented in Figure 4B.
The area changes of hard exudate lesions over time. (A) Total area of hard exudate lesions in all seven fields at baseline, Month 3, Month 6, and Month 12. (Area changes comparing with baseline: P = .1634 at Month 3, P = .4849 at Month 6, and P < .0001 at Month 12. (B) The area of hard exudate lesions in Field 2 at each monthly visit.
The efficacy of intravitreal injections of anti-VEGFs on DME has been well established through multiple randomized, controlled clinical trials including RESOLVE, RIDE, and RISE.18–21 The clinically significant benefits on prohibiting DR severity from progressing has also been reported previously in the RIDE and RISE open-label extension period data using ETDRS scoring.22,23 The results of our study are consistent with previous studies but presented by a novel quantitative method.
Intraretinal hemorrhage in DR is considered the eruption of the enlarged microaneurysms in outer retinal layers including the inner nuclear layer and outer molecular layer. These lesions appeared in various shapes, such as punctate, blot, or linear, with the color of venous blood or paler.5,22 Hence, it is difficult to quantify these lesions manually.
In the present study, the area of intraretinal hemorrhage decreased significantly during the course of the follow-up. The area of intraretinal hemorrhage decreased by 56.9% at the 3-month visit after one or three initial monthly injections and decreased by 48.7% at 6 months and 62.5% at the end of the year compared with the baseline visit. Similar results have been presented in several randomized, controlled trials in a hierarchical manner including VIVID, VISTA, RISE, and RIDE etc.11,12 In the present study, data were delivered in a more precise and quantitative approach.
The area change over time of hard exudate during the course of course of the conbercept treatment was more interesting. When the blood-retinal barrier was damaged, serous liquids leaked from the vascular structure. They were absorbed during the course of time, and the process may be accelerated with prompt treatment. However, lipoproteins remained to form hard exudate mostly located in the outer layers of the retina. Different descriptions of the change of HE during the resolution of diabetic macular edema have been reported. In our study, the area of hard exudate grew during the loading phase of treatment and increased by 8.9% at the sixth month. But by the end of the study, the study eye resulted in significantly greater reduction of HE area comparing with baseline. This supports the conclusion that macular edema is the first to resolve, followed by a later resolution of hard exudate lesions.23,24
Certain limitations existed in the present study. The wrong judgment of the automated software occurred in some of the images. Localized depigmentated areas such as choroid atrophy and laser scars were mistaken for hard exudate when they appeared to be morphologically similar. A few vascular segments were misjudged as intraretinal hemorrhage. The related classifiers are under development to exclude those false-positive lesions.
The follow-up time could be extended to better observe the lesion change over time. In addition, as a single-center study, the number of subjects included in the present study was limited. Large-scale, randomized, multicenter clinical trials with central reading center are proposed for further clinical evidence. Furthermore, novel modules of our software system to locate, segment, and quantify other typical lesion types of DR are under development.25–27
The automated software is a promising to be a precise and objective method for evaluating the progression of DR and the outcomes of interventions.
- Ting DS, Cheung GC, Wong TY. Diabetic retinopathy: global prevalence, major risk factors, screening practices and public health challenges: a review. Clin Exp Ophthalmol. 2016;44(4):260–277. doi:10.1111/ceo.12696 [CrossRef] PMID:26716602
- Klein BE. Overview of epidemiologic studies of diabetic retinopathy. Ophthalmic Epidemiol. 2007;14(4):179–183. doi:10.1080/09286580701396720 [CrossRef] PMID:17896294
- Cheung N, Mitchell P, Wong TY. Diabetic retinopathy. Lancet. 2010;376(9735):124–136. doi:10.1016/S0140-6736(09)62124-3 [CrossRef] PMID:20580421
- Stratton IM, Kohner EM, Aldington SJ, et al. UKPDS 50: risk factors for incidence and progression of retinopathy in Type II diabetes over 6 years from diagnosis. Diabetologia. 2001;44(2):156–163. doi:10.1007/s001250051594 [CrossRef] PMID:11270671
- Early Treatment Diabetic Retinopathy Study Research Group. Grading diabetic retinopathy from stereoscopic color fundus photographs—an extension of the modified Airlie House classification. ETDRS report number 10. Ophthalmology. 1991;98(5)(suppl):786–806. doi:10.1016/S0161-6420(13)38012-9 [CrossRef] PMID:2062513
- Zaki WMDW, Zulkifley MA, Hussain A, Halim WHWA, Mustafa NBA, Ting LS. Diabetic retinopathy assessment: towards an automated system. Biomed Signal Process Control. 2016;24:72–82. doi:10.1016/j.bspc.2015.09.011 [CrossRef]
- Abràmoff MD, Reinhardt JM, Russell SR, et al. Automated early detection of diabetic retinopathy. Ophthalmology. 2010;117(6):1147–1154. doi:10.1016/j.ophtha.2010.03.046 [CrossRef] PMID:20399502
- Chew EY, Ambrosius WT, Davis MD, et al. ACCORD Eye Study Group. Effects of medical therapies on retinopathy progression in type 2 diabetes. N Engl J Med. 2010;363(3):233–244. doi:10.1056/NEJMoa1001288 [CrossRef] PMID:20587587
- Wells JA, Glassman AR, Ayala AR, et al. Diabetic Retinopathy Clinical Research Network. Aflibercept, bevacizumab, or ranibizumab for diabetic macular edema. N Engl J Med. 2015;372(13):1193–1203. doi:10.1056/NEJMoa1414264 [CrossRef] PMID:25692915
- Brown DM, Schmidt-Erfurth U, Do DV, et al. Intravitreal aflibercept for diabetic macular edema: 100-week results from the VISTA and VIVID studies. Ophthalmology. 2015;122(10):2044–2052. doi:10.1016/j.ophtha.2015.06.017 [CrossRef] PMID:26198808
- Domalpally A, Ip MS, Ehrlich JS. Effects of intravitreal ranibizumab on retinal hard exudate in diabetic macular edema: findings from the RIDE and RISE phase III clinical trials. Ophthalmology. 2015;122(4):779–786. doi:10.1016/j.ophtha.2014.10.028 [CrossRef] PMID:25601535
- Ip MS, Domalpally A, Hopkins JJ, Wong P, Ehrlich JS. Long-term effects of ranibizumab on diabetic retinopathy severity and progression. Arch Ophthalmol. 2012;130(9):1145–1152. doi:10.1001/archophthalmol.2012.1043 [CrossRef] PMID:22965590
- Liu M, Yu Q, Yang J, et al. Local Connectedness Constraint and Contrast Normalization Based Microaneurysm Detection. In: Tan T, Li X, Chen X, Zhou J, Yang J, Cheng H, eds. 7th Chinese Conference, CCPR 2016, Chengdu, China, November 5–7, 2016, Proceedings, Part II. Springer: Chengdu, China; 2016:392–403.
- Zhou L, Yu Q, Xu X, Gu Y, Yang J. Improving dense conditional random field for retinal vessel segmentation by discriminative feature learning and thin-vessel enhancement. Comput Methods Programs Biomed. 2017;148:13–25. doi:10.1016/j.cmpb.2017.06.016 [CrossRef] PMID:28774435
- Zhou L, Zhao Y, Yang J, Yu Q, Xu X. Deep multiple instance learning for automatic detection of diabetic retinopathy in retinal images. IET Image Process. 2018;12(4):563–571. doi:10.1049/iet-ipr.2017.0636 [CrossRef]
- Li X, Xu G, Wang Y, et al. AURORA Study Group. Safety and efficacy of conbercept in neovascular age-related macular degeneration: results from a 12-month randomized phase 2 study: AURORA study. Ophthalmology. 2014;121(9):1740–1747. doi:10.1016/j.ophtha.2014.03.026 [CrossRef] PMID:24793528
- Aiello LP, Beck RW, Bressler NM, et al. Diabetic Retinopathy Clinical Research Network; Writing Committee. Rationale for the diabetic retinopathy clinical research network treatment protocol for center-involved diabetic macular edema. Ophthalmology. 2011;118(12):e5–e14. doi:10.1016/j.ophtha.2011.09.058 [CrossRef] PMID:22136692
- Massin P, Bandello F, Garweg JG, et al. Safety and efficacy of ranibizumab in diabetic macular edema (RESOLVE Study): a 12-month, randomized, controlled, double-masked, multicenter phase II study. Diabetes Care. 2010;33(11):2399–2405. doi:10.2337/dc10-0493 [CrossRef] PMID:20980427
- Nguyen QD, Shah SM, Heier JS, et al. READ-2 Study Group. Primary end point (six months) results of the Ranibizumab for Edema of the Macula in Diabetes (READ-2) study. Ophthalmology2009;116(11):2175–2181.e1. doi:10.1016/j.ophtha.2009.04.023 [CrossRef] PMID: 19700194
- Nguyen QD, Shah SM, Khwaja AA, et al. READ-2 Study Group. Two-year outcomes of the ranibizumab for edema of the mAcula in diabetes (READ-2) study. Ophthalmology. 2010;117(11):2146–2151. doi:10.1016/j.ophtha.2010.08.016 [CrossRef] PMID:20855114
- Bressler NM, Varma R, Suñer IJ, et al. RIDE and RISE Research Groups. Vision-related function after ranibizumab treatment for diabetic macular edema: results from RIDE and RISE. Ophthalmology. 2014;121(12):2461–2472. doi:10.1016/j.ophtha.2014.07.008 [CrossRef] PMID:25148789
- Stitt AW, Curtis TM, Chen M, et al. The progress in understanding and treatment of diabetic retinopathy. Prog Retin Eye Res. 2016;51:156–186. doi:10.1016/j.preteyeres.2015.08.001 [CrossRef] PMID:26297071
- Brown DM, Nguyen QD, Marcus DM, et al. RIDE and RISE Research Group. Long-term outcomes of ranibizumab therapy for diabetic macular edema: the 36-month results from two phase III trials: RISE and RIDE. Ophthalmology. 2013;120(10):2013–2022. doi:10.1016/j.ophtha.2013.02.034 [CrossRef] PMID:23706949
- Singh A, Stewart JM. Pathophysiology of diabetic macular edema. Int Ophthalmol Clin. 2009;49(2):1–11. doi:10.1097/IIO.0b013e31819fd164 [CrossRef] PMID:19349783
- Trucco E, Ruggeri A, Karnowski T, et al. Validating retinal fundus image analysis algorithms: issues and a proposal. Invest Ophthalmol Vis Sci. 2013;54(5):3546–3559. doi:10.1167/iovs.12-10347 [CrossRef] PMID:23794433
- Wong TY, Bressler NM. Artificial Intelligence With Deep Learning Technology Looks Into Diabetic Retinopathy Screening. JAMA. 2016;316(22):2366–2367. doi:10.1001/jama.2016.17563 [CrossRef] PMID:27898977
- Sun JK, Cavallerano JD, Silva PS. Future Promise of and Potential Pitfalls for Automated Detection of Diabetic Retinopathy. JAMA Ophthalmol. 2016;134(2):210–211. doi:10.1001/jamaophthalmol.2015.5099 [CrossRef] PMID:26720257