Adaptive optics scanning laser ophthalmoscopy (AO-SLO) is an imaging modality that can visualize the photoreceptor mosaic.1 The ability to quantitatively measure cone photoreceptors has attracted the attention of researchers studying a variety of retinal diseases and their response to treatments.2–8 However, use of this technology in ophthalmology is relatively new and there is still much to learn about optimizing image acquisition and analysis to make the images reliable and clinically useful.
The size of the area chosen is one important consideration for optimal imaging analysis. Various measurement areas have been used in previous AO investigations, ranging generally from 50 μm × 50 μm to 85 μm × 85 μm.9–14 One group sampled three window sizes (55 μm × 55 μm, 40 μm × 40 μm, and 25 μm × 25 μm), and they observed an increase in measurement error with smaller measurement windows. Measurement windows larger than 85 μm × 85 μm and their effect on measurement accuracy have not been previously explored.
Although other groups have stated that additional work is needed to determine how AO imaging results vary with differences in image quality,8,12 to the best of our knowledge, no study has systematically examined this. In addition, no studies have reported reliability parameters from a real-life simulated clinical situation — one in which clinic photographers acquire the images and images of varying quality levels are interpreted. If AO-SLO is going to be employed in the clinic, the image acquisition will be done by photographers who are not dedicated only to AO-SLO studies, and variable image quality will be a reality that needs to be understood and accounted for. For the first time, we evaluate the effect of image quality and larger measurement window areas in normal retina cone topography analysis. In addition, we determine intergrader, intragrader, and intersession consistencies that have not been studied with this device.
Patients and Methods
This prospective study was approved by the institutional review board at the Massachusetts Eye and Ear Infirmary and conformed to the tenets of the Declaration of Helsinki. Written informed consent was obtained from all participants. Five healthy eyes of five healthy volunteers were included in this study. All patients had visual acuities (VAs) of 20/20 in both eyes and did not have any systemic or ocular diseases except for refractive errors.
AO-SLO Image Acquisition
The images were obtained with an AO-SLO RD102 prototype (Canon, Tokyo, Japan). The technical details of the AO-SLO system used for this study have been published previously.13 All imaging was obtained through undilated pupils because pupil dilation increases third- and fourth-degree aberrations. Cones were imaged in two areas in each eye: centered on the fovea and nasal to the first image. These two locations were chosen because they were employed by a previous AO reliability study, and we wished to reproduce that protocol to permit comparison of reliability parameters across platforms.9 This focusing of the camera onto the same region in each eye was repeated five separate times by each photographer to determine intersession reliability for each photographer. The entire imaging protocol was performed separately by two photographers to be able to determine interphotographer reliability. The photographers who took the AO-SLO images are the same ones who perform a range of imaging techniques for our practice. They were trained to take the images by personnel from the Canon Corporation using the protocol developed by the investigators. There was no quality check done at the time of photography apart from the photographer themselves trying to acquire the clearest image possible. This allowed us to simulate the real-life acquisition of AO-SLO images in retina clinics where it will be performed by imaging technicians during various other imaging duties.
AO-SLO Imaging Analysis
Two masked, independent ophthalmologist readers analyzed the images. The readers were trained by training experts from Canon. Cone density, the number of cones divided by the area of retina sampled, and distance to fovea were estimated by Photoreceptor Analysis Software version 2.1 (Canon, Tokyo, Japan), which automatically counts individual cones within a selected area. There was no manual adjustment of cone counts to minimize the sources of variability in cone measurement. We avoided vessels in the measurement window area when possible to prevent error from white blood cell movement. With this AO-SLO camera, each image consists of 16 frames of a video taken during a 1-second time period. When a measurement window is selected in an image for analysis, the custom software calculates the cone density based on an average of cone densities in these 16 frames.
Effect of Measurement Size on Cone Density Measurement
To determine the optimal measurement window size for cone density measurement, cone densities were measured in 150 μm × 150 μm, 100 μm × 100 μm, 85 μm × 85 μm, and 50 μm × 50 μm window measurements in 20 images of the same anatomical place in one participant (Figure 1). The consistent localization of the measurement windows in the same anatomic location across the 20 images was determined by the grader using a reference fundus photograph with the measurement window coordinates marked on it. This camera lacks tracking capabilities.
Measurement windows with different sizes (50 μm × 50 μm, 85 μm × 85 μm, 100 μm ×100 μm, 150 × 150 μm) within one adaptive optics scanning laser ophthalmoscopy image.
Image Quality Determination
Based on clarity of the cone mosaic pattern in each image, the images were divided into three groups: high quality, medium quality, and poor quality (Figure 2). If the cone mosaic pattern was distinct in all parts of the image, the frame was placed in the high-quality group. An example of a distinct mosaic cone pattern, where the cones and their borders with one another are sharp and clear throughout the image, is shown in Figure 2A. If the mosaic pattern was visible but not perfectly clear in all parts of the image, it was classified as a medium-quality image. Figure 2B shows an example of a medium-quality image where there are some areas where the cones and their borders with other cones are not distinguishable. If the mosaic pattern was not distinguishable at all, the image was classified in the poor-quality group (Figure 2C). We excluded all poor-quality images from the subsequent analyses. The impact of image quality on inter-grader reliability was examined using 10 high-quality versus 10 medium-quality images in one patient.
Reference images for different quality categories. (A) Representative photograph of a high-quality image. In a high-quality image, the cone mosaic pattern is distinct in all parts of the image. The cones and their borders with one another are sharp and clear throughout the image. (B) Representative photograph of a medium-quality image. In a medium-quality image, the mosaic pattern is visible but not perfectly clear in all parts of the image. There are some areas where the cones and their borders with other cones are not distinguishable. The white arrows indicate the regions where the mosaic pattern is vague. (C) Representative photograph of a poor-quality image. In a poor-quality image, the mosaic pattern is not distinguishable at all.
Intergrader, Intragrader, Intersession, and Interphotographer Reliability
To determine the intergrader reliability of the cone density measurements, each grader independently used the software to measure the prespecified area in all medium- and high-quality images for each of the five eyes. Intragrader reliability was evaluated by comparing identical areas measured on two separate occasions by the same grader within a 2-month interval in the same images in the five patients in 90 medium- and high-quality images. Intersession reliability was assessed by five separate instances of the photographer focusing the camera onto the same region in each of the five eyes. To determine interphotographer reliability, each patient was imaged by two photographers from the pool of five photographers available to take images for the study.
For the comparison of the cone density measures with differing image parameters (various measurement window sizes and high versus medium image quality), multilevel linear regression was used. Bartlett's test was used to compare standard deviations among repeated cone density measurements.
Intergrader, intragrader, intersession, and interphotographer reliabilities were assessed by using the interclass correlation coefficient (ICC). The Bland-Altman method was also used to present intergrader, intragrader, and interphotographer agreement.15–17 In a Bland-Altman analysis, the agreement between two measurements is assessed by taking the difference between the two measurements and assessing the summary statistics and graphics of the differences. All analyses were performed using Stata IC 12.1 (College Station, TX).
Five healthy eyes in three women and two men (mean age ± 1 standard deviation = 30.8 years ± 5.8 years) were included. The best-corrected VAs were 20/20 in all eyes; mean spherical equivalent was −2.8 diopters (D) ± 1.4 D with a range from −0.25 D to −4.0 D.
First, we evaluated the influence of the four different measurement windows areas on the reliability of cone density counts. In each of the 20 images obtained from one eye, when the cone density counts were done within larger measurement window areas, the counts were more consistent, as evidenced by the smaller standard deviations (Bartlett's test; P = .0001). However, the mean cone densities were not significantly different across the four different measurement window areas using the mixed model linear regression test (P = .37) (Figure 3). Since cone cell density measurements were most consistent with the largest measurement window size (150 μm × 150 μm), we selected this size for all subsequent analyses.
Cone density counts as a function of measurement window size in one patient.
To investigate the impact of quality on our measurements, we compared 10 medium-quality images versus 10 high-quality images with regard to cone density in the same anatomic location in one participant. Bartlett's test showed a statistically significant difference in standard deviation between the two groups (954 cones/mm2 in medium versus 68 cones/mm2 in high-quality images, P = .01) (Figure 4). Mean cone densities were not significantly different between the two groups (16,655 cones/mm2 in medium- versus 17,051 cones/mm2 in high-quality images, P = .40 by mixed model linear regression).
Cone density counts as a function of image quality.
Table 2 shows intragrader, intergrader, intersession, and interphotographer reliabilities. Intragrader and intergrader ICCs were 0.99 (95% confidence interval [CI], 0.997–0.999) and 0.98 (95% CI, 0.975–0.989), respectively. Intersession reliability had an ICC of 0.72 (95% CI, 0.423–0.909). Interphotographer reliability had an ICC of 0.72 (95% CI, 0.461–0.866).
Means and Standard Deviations of Cone Density Measurements With Different Measurement Window Sizes
Intragrader, Intergrader, Intersession, and Interphotographer Reliabilities
Bland-Altman agreements are shown in Figure 5. The mean difference in cone density measurement for each grader was −40.7 cones/mm2 and the 95% CI was −90.5 to 8.9 (P = .29). The mean difference in cone density between graders was −92.8 cones/mm2 and the 95% CI was 215.0–29.4 (P = .80). The mean difference in cone density between images taken by different photographers was 298.7 with a 95% CI of −1,400–758 (P = .76). P values were not significant in any of Bland-Altman comparisons, which is interpreted to mean that the estimated means of the differences and the standard deviations of the differences were not significantly different.
Bland-Altman agreements plots. The mean difference in cone density for the intragrader measures is 40.7 cones/mm2 (95% confidence interval [CI], −90.5–8.9). The mean difference in cone density for the intergrader measures is −92.8.7 cones/mm2 (95% CI, −215.0–29.4). The mean difference in cone density for the interphotographer measure is −298.7 cones/mm2 (95% CI, −1.4 × 10-3-758.1).
In a simulated real-life clinical setting, variation in AO-SLO image quality strongly impacted cone analysis reliability. High-quality images had the most consistent measurements. In addition, measurements were more consistent when density was calculated from a larger measurement window area. AO-SLO cone density measurements had excellent intergrader and intragrader reliabilities. Intersession and interphotographer reliabilities were not as good but still strong.
Image quality has not been studied systematically by previous studies. This is the first study to evaluate the effect of quality on cone density measurements in a simulated real-life clinical setting. Intragrader and intergrader reliabilities were excellent because measurements were performed on identical images by graders, and differences in quality thus could not impact these measures significantly. However, interphotographer and intersession reliability, which required comparison across different images of differing quality, demonstrated less consistency.
Our results showed higher consistency with larger measurement windows. This is consistent with the results from a previous study using a different AO-SLO device where the investigators found that decreasing measurement window area was associated with increased measurement errors.10 Compared with the current study, their analyses were done using smaller measurement areas. We examined larger measurement areas and show that additional precision can be gained by using these larger measurement areas. In the previous study with the 55 μm × 55 μm measurement window, the standard deviation was 1,573.4 cones/mm2.10 We found a similar standard deviation for the 50 μm × 50 μm measurement window in our study at 1,799.5 cones/mm2 with the standard deviation decreasing significantly to 356.4 cones/mm2 with the 150 μm × 150 μm measurement window. Previous AO-SLO studies have largely limited the measurement areas to sizes ranging from 50 μm × 50 μm to 85 μm × 85 μm.9,10,12–14 Our results show that cone density measurements may be more consistent with larger measurement areas than those typically used by AO studies thus far. The reason for increased precision is that larger measurement window area minimizes the influence of sources of error, such as vessel presence and movement artifact. If a vessel is present, the impact of the area taken up by the vessel, and thus its influence on the cone density, will be proportionally greater in a smaller measurement field. Future studies of cone density with AO-SLO should consider focusing on larger measurement areas to increase the accuracy of measurements.
The ICCs found in this study are comparable to those of prior studies with different AO instruments. Liu et al. reported intergrader ICCs of 0.957 for cone density measurements using a semi-automated counting method with a different AO-SLO camera system. Our intergrader ICC was quite similar at 0.983. Garnier et al. also demonstrated high reliability of cone counts in studying 10 healthy subjects using the RT-x 1 AO retinal camera (Imagin Eyes, Orsay, France).9 In that study, intergrader and interphotographer ICCs were 0.98 and 0.96, respectively. Their interphotographer agreement was higher than of the current study (0.72). This is likely because they had two highly trained photographers for their study. In the current study, five photographers took the images. Although they were trained on how to acquire high-quality AO-SLO images, they were not dedicated solely to the AO-SLO study, and some photographers had more experience with fundus photography than others. This variability in experience was a likely the major contributor to the lower inter-photographer reliability in our study. This is an important issue that needs to be addressed in the attempt to bring AO-SLO into clinical use. Obtaining AO images requires higher level of skill than other fundus imaging modalities and the importance of proper photographer training will greatly impact clinical interpretation.
This study has some limitations. Dividing images to three groups based on quality was done by subjective assessment. Although it is subjective, it is based on clearly explained parameters regarding cone mosaic pattern and reference photographs, mimicking the methods employed for grading fundus photographs.18 Software that calculates a quantitative quality factor, similar to that available for optical coherence tomography, would provide a more objective assessment. Another limitation was that we selected five normal eyes to minimize the effect of ocular pathologies that could affect image quality, and therefore could not assess the effects of media opacity on reliability. The sample size is also modest.
There are several limitations of the AO-SLO technology that remain to be addressed before it can be employed consistently and usefully in the clinic. Correction for eye movement with an active eye-tracking system is important and has already been employed by another AO device.19 Software to ensure images are taken in the same location from visit to visit is necessary, particularly because of the drop off in cone density from the foveal center.20 Although we anticipate future improvements in this technology, analysis and interpretation of AO-SLO images at the present time need to be done carefully. Based on the results of this study and with the AO instruments and software currently available, we propose the best methods to maximize reliability in AO-SLO cone measurement include using the largest possible window measurement that will avoid vessels and carefully choosing high-quality images. It will be important to perform future reliability studies in eyes with pathology as those are the ones we hope to eventually be able to use this technology for.
- Roorda A, Romero-Borja F, Donnelly W Iii, Queener H, Hebert T, Campbell M. Adaptive optics scanning laser ophthalmoscopy. Opt Express. 2002;10(9):405–412. doi:10.1364/OE.10.000405 [CrossRef]
- Duncan JL, Zhang Y, Gandhi J, et al. High-resolution imaging with adaptive optics in patients with inherited retinal degeneration. Invest Ophthalmol Vis Sci. 2007;48(7):3283–3291. doi:10.1167/iovs.06-1422 [CrossRef]
- Morgan JI, Han G, Klinman E, et al. High-resolution adaptive optics retinal imaging of cellular structure in choroideremia. Invest Ophthalmol Vis Sci. 2014;55(10):6381–6397. doi:10.1167/iovs.13-13454 [CrossRef]
- Nakao S, Kaizu Y, Yoshida S, Iida T, Ishibashi T. Spontaneous remission of acute zonal occult outer retinopathy: Follow-up using adaptive optics scanning laser ophthalmoscopy. Graefes Arch Clin Exp Ophthalmol. 2015;253(6):839–843. doi:10.1007/s00417-014-2760-x [CrossRef]
- Ooto S, Hangai M, Sakamoto A, et al. High-resolution imaging of resolved central serous chorioretinopathy using adaptive optics scanning laser ophthalmoscopy. Ophthalmology. 2010;117(9):1800–1809, 1809.e1–2. doi:10.1016/j.ophtha.2010.01.042 [CrossRef]
- Ooto S, Hangai M, Takayama K, et al. High-resolution imaging of the photoreceptor layer in epiretinal membrane using adaptive optics scanning laser ophthalmoscopy. Ophthalmology. 2011;118(5):873–881. doi:10.1016/j.ophtha.2010.08.032 [CrossRef]
- Roorda A, Duncan JL. Adaptive optics ophthalmoscopy. Annu Rev Vis Sci. 2015;1:19–50. doi:10.1146/annurev-vision-082114-035357 [CrossRef]
- Zayit-Soudry S, Sippl-Swezey N, Porco TC, et al. Repeatability of cone spacing measures in eyes with inherited retinal degenerations. Invest Ophthalmol Vis Sci. 2015;56(10):6179–6189. doi:10.1167/iovs.15-17010 [CrossRef]
- Bidaut Garnier M, Flores M, Debellemaniere G, et al. Reliability of cone counts using an adaptive optics retinal camera. Clin Experiment Ophthalmol. 2014;42(9):833–840. doi:10.1111/ceo.12356 [CrossRef]
- Garrioch R, Langlo C, Dubis AM, Cooper RF, Dubra A, Carroll J. Repeatability of in vivo parafoveal cone density and spacing measurements. Optom Vis Sci. 2012;89(5):632–643. doi:10.1097/OPX.0b013e3182540562 [CrossRef]
- Li KY, Tiruveedhula P, Roorda A. Intersubject variability of foveal cone photoreceptor density in relation to eye length. Invest Ophthalmol Vis Sci. 2010;51(12):6858–6867. doi:10.1167/iovs.10-5499 [CrossRef]
- Liu BS, Tarima S, Visotcky A, et al. The reliability of parafoveal cone density measurements. Br J Ophthalmol. 2014;98(8):1126–1131. doi:10.1136/bjophthalmol-2013-304823 [CrossRef]
- Park SP, Chung JK, Greenstein V, Tsang SH, Chang S. A study of factors affecting the human cone photoreceptor density measured by adaptive optics scanning laser ophthalmoscope. Exp Eye Res. 2013;108:1–9. doi:10.1016/j.exer.2012.12.011 [CrossRef]
- Song H, Chui TY, Zhong Z, Elsner AE, Burns SA. Variation of cone photoreceptor packing density with retinal eccentricity and age. Invest Ophthalmol Vis Sci. 2011;52(10):7376–7384. doi:10.1167/iovs.11-7199 [CrossRef]
- Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1(8476):307–310. doi:10.1016/S0140-6736(86)90837-8 [CrossRef]
- Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res. 1999;8(2):135–160. doi:10.1177/096228029900800204 [CrossRef]
- Bland JM, Altman DG. Applying the right statistics: Analyses of measurement studies. Ultrasound Obstet Gynecol. 2003;22(1):85–93. doi:10.1002/uog.122 [CrossRef]
- No authors listed. Grading diabetic retinopathy from stereoscopic color fundus photographs – an extension of the modified Airlie House classification. ETDRS report number 10. Early Treatment Diabetic Retinopathy Study Research Group. Ophthalmology. 1991;98(5 Suppl):786–806. doi:10.1016/S0161-6420(13)38012-9 [CrossRef]
- Sheehy CK, Tiruveedhula P, Sabesan R, Roorda A. Active eye-tracking for an adaptive optics scanning laser ophthalmoscope. Biomed Opt Express. 2015;6(7):2412–2423. doi:10.1364/BOE.6.002412 [CrossRef]
- Curcio CA, Sloan KR, Kalina RE, Hendrickson AE. Human photoreceptor topography. J Comp Neurol. 1990;292(4):497–523. doi:10.1002/cne.902920402 [CrossRef]
Intragrader, Intergrader, Intersession, and Interphotographer Reliabilities
Means and Standard Deviations of Cone Density Measurements With Different Measurement Window Sizes
Measurement Window Size
Standard Deviation (cones/mm2)
|50 μm × 50 μm
|85 μm × 85 μm
|100 μm × 100 μm
|150 μm × 150 μm