Ophthalmic Surgery, Lasers and Imaging Retina

Clinical Science 

Reproducibility of Macular Thickness Measurements in Eyes Affected by Dry Age-Related Macular Degeneration From Two Different SD-OCT Instruments

Tudor C. Tepelus, PhD; Amir H. Hariri, MD; Siva Balasubramanian, MD, PhD; Srinivas R. Sadda, MD

Abstract

BACKGROUND AND OBJECTIVE:

To compare macular thickness measurement algorithms of two different spectral-domain optical coherence tomography (SD-OCT) devices in eyes affected by dry age-related macular degeneration (AMD).

PATIENTS AND METHODS:

Patients with dry AMD and healthy volunteers from the retina clinic of the Doheny Eye Center – UCLA were imaged using two different SD-OCT devices: the RS-3000 Advance (Nidek, Padova, Italy) and the Cirrus HD-OCT (Carl Zeiss Meditec, Dublin, CA). All patients had been previously diagnosed with drusen or geographic atrophy due to AMD. The commercial instrument software was used to generate the macular retinal thickness measurements, and measurements were compared between devices.

RESULTS:

Eighty-five diseased eyes from 49 patients and 16 healthy control eyes from eight normal volunteers were included in this study. The macular thickness measurements generated by the two instruments in eyes with AMD differed significantly in mean retinal thickness in the foveal center subfield (257.34 μm ± 51.72 μm using the Nidek OCT vs. 238.20 μm ± 51.89 μm using the Cirrus OCT; P < .001). The mean difference in macular thickness between the two devices was 19.14 μm ± 5.84 μm for diseased eyes and 17.06 μm ± 5.28 μm in normal control eyes, and this was not statistically different between the two groups (P > .05). The macular thickness measurements in diseased eyes, as evaluated by the two different instruments, however, showed excellent correlation (r = 0.99; P < .001), with an intraclass correlation coefficient of 0.99 (95% confidence interval, 0.98–0.99). Post hoc evaluation of cases with larger differences also showed differences in foveal center selection and variabilities in boundary selection with specific pathology.

CONCLUSION:

Macular thickness measurements provided by the Nidek and Cirrus OCT instruments in eyes with dry AMD are highly correlated but show a consistent difference, which may allow the use of a standard correction factor to be applied to better interrelate measurements between the devices.

[Ophthalmic Surg Lasers Imaging Retina. 2018;49:410–415.]

Abstract

BACKGROUND AND OBJECTIVE:

To compare macular thickness measurement algorithms of two different spectral-domain optical coherence tomography (SD-OCT) devices in eyes affected by dry age-related macular degeneration (AMD).

PATIENTS AND METHODS:

Patients with dry AMD and healthy volunteers from the retina clinic of the Doheny Eye Center – UCLA were imaged using two different SD-OCT devices: the RS-3000 Advance (Nidek, Padova, Italy) and the Cirrus HD-OCT (Carl Zeiss Meditec, Dublin, CA). All patients had been previously diagnosed with drusen or geographic atrophy due to AMD. The commercial instrument software was used to generate the macular retinal thickness measurements, and measurements were compared between devices.

RESULTS:

Eighty-five diseased eyes from 49 patients and 16 healthy control eyes from eight normal volunteers were included in this study. The macular thickness measurements generated by the two instruments in eyes with AMD differed significantly in mean retinal thickness in the foveal center subfield (257.34 μm ± 51.72 μm using the Nidek OCT vs. 238.20 μm ± 51.89 μm using the Cirrus OCT; P < .001). The mean difference in macular thickness between the two devices was 19.14 μm ± 5.84 μm for diseased eyes and 17.06 μm ± 5.28 μm in normal control eyes, and this was not statistically different between the two groups (P > .05). The macular thickness measurements in diseased eyes, as evaluated by the two different instruments, however, showed excellent correlation (r = 0.99; P < .001), with an intraclass correlation coefficient of 0.99 (95% confidence interval, 0.98–0.99). Post hoc evaluation of cases with larger differences also showed differences in foveal center selection and variabilities in boundary selection with specific pathology.

CONCLUSION:

Macular thickness measurements provided by the Nidek and Cirrus OCT instruments in eyes with dry AMD are highly correlated but show a consistent difference, which may allow the use of a standard correction factor to be applied to better interrelate measurements between the devices.

[Ophthalmic Surg Lasers Imaging Retina. 2018;49:410–415.]

Introduction

Optical coherence tomography (OCT) is a noninvasive technique for capturing cross-sectional, high-resolution images of the retina in normal and pathologic eyes.1–3 In clinical practice, it has been used mainly for retinal qualitative evaluation, such as visualization of subretinal fluid,4 macular edema,5 epiretinal membranes,6 photoreceptor layer disruption,7 macular holes,8 and choroidal neovascularization.9 OCT provides an evaluation of retinal thickness over time with a high level of reliability and reproducibility.10,11

Advances in OCT technology have facilitated the introduction of new commercially available spectral-domain OCT (SD-OCT) devices, allowing faster image acquisition speeds and higher axial resolution, as well as hardware and software improvements.12,13 Presently, there are a great many commercial OCT instruments from different manufacturers using a variety of proprietary algorithms for analysis, thereby yielding a potential for significant variability in retinal measurements and interpretation of results. These differences have posed challenges for interrelating data collected in clinical trials and clinical practice. OCT image reading centers have used various strategies to manage this challenge including restricting trials to the use of a single OCT instrument study-wide or implementing a standard correction factor. The most appropriate solution, however, needs to be established for each new device.

In this study, we compared retinal thickness measurements in a cohort of eyes with dry age-related macular degeneration (AMD) and normal control eyes obtained with the Nidek SD-OCT (Nidek Technologies, Padova, Italy) against those from the Cirrus SD-OCT (Carl Zeiss Meditec, Dublin, CA), which has been used in multiple previous clinical trials.

Patients and Methods

Subjects with dry AMD (of various stages: early, intermediate, and advanced) and normal controls were prospectively recruited from the medical retina clinic (single physician, SRS) of the Doheny – UCLA Eye Center in Pasadena, CA. The clinical diagnosis was determined by the examining physician. The research study was approved by the institutional review board of the University of California – Los Angeles and adhered to the tenets set forth in the Declaration of Helsinki. All patients and healthy volunteers signed written, informed consent after a detailed explanation of the nature of the study. Patients with other posterior segment disease (eg, glaucoma, diabetic retinopathy, macular hole), or significant media opacity (“significant” defined as having an impact on vision in the opinion of the treating physician) were excluded from the study in order to maintain the homogeneity of our cohort, as well as the image quality.

SD-OCT volume scans of both eyes were performed using the Nidek RS-3000 Advance 512 × 128 macular cube protocol with the standard available wide area scan of 9 mm × 9 mm centered on the fovea, and Cirrus HD-OCT with a 512 × 128 macular cube protocol (6 mm × 6 mm area scan, centered on the fovea). All tests were conducted by two experienced OCT examiners (TCT, AHH). The OCT images were considered acceptable when the retina was visible in every B-scan and there were no eye movements and blinking artifacts during the examination. The acquired OCT images were evaluated using NAVIS-EX (version 1.5.0.2) and Cirrus review software (version 7.5.0.56). For both instruments, the standard analysis protocols for macular thickness evaluation were used. The automatic segmentation function was used to identify the internal limiting membrane (ILM) and retinal pigment epithelium/Bruch's membrane (RPE/BM) and to compute the macular thickness. The foveal center was identified and manually corrected when it was not automatically correctly detected. For the analysis, only the foveal central subfield (central circular area of 1,000 μm) was considered since the overall scanned area differed between the two devices.

The power of the study was calculated by using the online platform provided by Rollin Brant (available at https://www.stat.ubc.ca/~rollin/stats/ssize/n2.html).

We compared macular thickness measurement algorithms of two different SD-OCT devices in eyes affected by dry AMD. Furthermore, we compared the mean differences in macular thickness generated by the two instruments between the diseased and control eyes in order to determine whether they may be generated mainly by different segmentation algorithms and/or the anatomical features specific for eyes with dry AMD.

Data were analyzed using the Kolmogorov-Smirnov test to determine the normality of data distribution. The macular thickness in the ETDRS central subfield was calculated for every eye using the two distinct algorithms, and the averages were compared performing a Wilcoxon test. The mean differences between the two instruments for diseased and control eyes were also compared using the Wilcoxon test.

The correlation of macular thickness measurements in AMD eyes evaluated by the two different instruments was assessed by Pearson correlation coefficient. Intraclass correlation coefficient and Bland-Altman plot were used to assess the reproducibility and agreement of measurements. Regression analysis was performed to generate the conversion equation between the instruments.

Statistical analyses of the data were performed using commercial software (Statistical Package for Social Science, version 19.0; SPSS, Armonk, NY; and MedCalc version 12; MedCalc software bvba; Mariakerke, Belgium). The significance level was set at a P value less than .05 for all tests.

Results

A total of 85 eyes from 49 patients with dry AMD (15 male, 34 female) and 16 eyes from eight healthy volunteers (five male, three female) were included in this study and were imaged with both the Nidek OCT and Cirrus OCT in a randomly chosen order. The mean age of patients was 76 years ± 15 years, whereas the mean age of healthy subjects was 37 years ± 5 years. The power of the study was 0.91.

The mean macular thickness measurements in the EDTRS central subfield obtained by the two OCT instruments differed significantly in AMD eyes (257.34 μm ± 51.72 μm using the Nidek OCT vs. 238.20 μm ± 51.89 μm using the Cirrus OCT; P < .001), with an average difference of 19.14 μm ± 5.84 μm.

The average difference between the two measurements for control eyes was 17.06 μm ± 5.28 μm (270.69 μm ± 12.15 μm using the Nidek OCT vs. 253.63 μm ± 16.11 μm using the Cirrus OCT; P < .01).

The discrepancy in the average difference between diseased (19.14 μm ± 5.84 μm) and control eyes (17.06 μm ± 5.28 μm) was not statistically significant (P > .05).

Figure 1 illustrates the default positions of the automated inner and outer retinal boundaries provided by each device, which led to this relatively consistent measurement difference between devices in eyes with dry AMD. Both OCT instruments identified the inner retinal boundary at the level of the internal limiting membrane, but the position of the outer retinal boundary differed between the two instruments. The Nidek OCT identifies the outer boundary at the external aspect of the presumed RPE band, whereas the Zeiss Cirrus OCT identifies the same boundary at the inner aspect of the presumed RPE band. For our study cohort, we have determined the maximum difference in macular thickness between Nidek OCT and Cirrus OCT to be 41 μm in a case of geographic atrophy involving the foveal center, whereas the minimum difference of 6 μm was noticed in a case of early stage dry AMD (drusen).

Positioning of retinal boundaries in the Nidek (A, C, E) and Zeiss Cirrus (B, D, F) optical coherence tomography (OCT) devices, (A, B) to generate the mean of the difference between measurements of the two OCT devices in an eye affected by early age-related macular degeneration (AMD) (drusen). The distance used by the two algorithms to calculate the macular thickness is indicated by the red arrow. Both instruments set the inner retinal boundary at the level of the internal limiting membrane (ILM). The outer retinal boundary is set by Nidek OCT at the external limit of retinal pigment epithelium (RPE), whereas the Zeiss Cirrus OCT identifies this boundary in the middle/within the RPE layer. Below the red arrows, we reported the corresponding mean macular thickness from the central circular 1,000 μm area; (C, D) the minimum difference in macular thickness evaluated by Nidek and Zeiss Cirrus OCT in a case of early AMD (drusen). Note the slightly different positioning of the retinal boundaries in both OCT devices, leading to a small difference in macular thickness measurement. (E,F) The maximum difference in macular thickness evaluated by Nidek and Zeiss Cirrus OCT in a case of geographic atrophy involving the foveal center. Note the different positioning of both retinal boundaries in both OCT devices, leading to a big difference in macular thickness measurement.

Figure 1.

Positioning of retinal boundaries in the Nidek (A, C, E) and Zeiss Cirrus (B, D, F) optical coherence tomography (OCT) devices, (A, B) to generate the mean of the difference between measurements of the two OCT devices in an eye affected by early age-related macular degeneration (AMD) (drusen). The distance used by the two algorithms to calculate the macular thickness is indicated by the red arrow. Both instruments set the inner retinal boundary at the level of the internal limiting membrane (ILM). The outer retinal boundary is set by Nidek OCT at the external limit of retinal pigment epithelium (RPE), whereas the Zeiss Cirrus OCT identifies this boundary in the middle/within the RPE layer. Below the red arrows, we reported the corresponding mean macular thickness from the central circular 1,000 μm area; (C, D) the minimum difference in macular thickness evaluated by Nidek and Zeiss Cirrus OCT in a case of early AMD (drusen). Note the slightly different positioning of the retinal boundaries in both OCT devices, leading to a small difference in macular thickness measurement. (E,F) The maximum difference in macular thickness evaluated by Nidek and Zeiss Cirrus OCT in a case of geographic atrophy involving the foveal center. Note the different positioning of both retinal boundaries in both OCT devices, leading to a big difference in macular thickness measurement.

The level of agreement between the two instruments for eyes with dry AMD is illustrated by the Bland-Altman plot in Figure 2. We have observed an excellent correlation between the two sets of data (r = 0.99, P < .001) (Figure 3), with an intraclass correlation coefficient of 0.99 (95% confidence interval, 0.98–0.99).

Bland-Altman analysis for testing the agreement between the Nidek device and the Zeiss Cirus device with respect to macular thickness measurement in eyes with dry age-related macular degeneration. OCT = optical coherence tomography; SD = standard deviation

Figure 2.

Bland-Altman analysis for testing the agreement between the Nidek device and the Zeiss Cirus device with respect to macular thickness measurement in eyes with dry age-related macular degeneration. OCT = optical coherence tomography; SD = standard deviation

Regression plot of macular thickness measured with the Nidek optical coherence (OCT) device and the Zeiss Cirrus OCT device in eyes with dry age-related macular degeneration. Differences in Y intercepts reflect differences in the measurement algorithms.

Figure 3.

Regression plot of macular thickness measured with the Nidek optical coherence (OCT) device and the Zeiss Cirrus OCT device in eyes with dry age-related macular degeneration. Differences in Y intercepts reflect differences in the measurement algorithms.

Discussion

In this study, we compared the macular thickness measurements generated by two different OCT instruments to test the agreement and reproducibility of the OCT devices in eyes with dry AMD.

We observed that the macular thickness measurements generated by two instruments are statistically different. However, the Bland-Altman plot shows that most of the values lie close to the mean difference of 19.14 μm and within the limits of agreement, including 95% of the differences between measurements generated by the two instruments. Specifically, the Nidek OCT measured about 19.14 μm thicker than the Cirrus OCT, corresponding to essentially the thickness of the RPE band. The very high positive correlation between the measurements from the two devices suggests that the application of conversion factor (ie, subtract 19.14 μm from a Nidek-based measurement to produce a Cirrus-equivalent) may be a reasonable strategy in studies that utilize both instruments.

Forte et al. also determined that macular retinal thickness measurements with spectral-domain scanning laser ophthalmoscope OCT (SLO/OCT; Ophthalmic Technologies, Toronto, Canada) were higher than those generated by time-domain Stratus OCT (version 4.0.1; Carl Zeiss Meditec, Dublin, CA) in healthy and diseased eyes, showing significant correlation between the two instruments.14

Another study comparing time-domain and SD-OCT instruments concluded that their measurements of macular thickness in normal eyes and eyes with diabetic macular edema, were different but could be reliably converted between devices by using a conversion equation.15 Ultimately, Alan et al. found that macular thickness measurements were highly reproducible across four SD-OCT and time-domain OCT (TD-OCT) instruments in a cohort of 40 healthy eyes.16

Other studies comparing the retinal metrics between various TD-OCT and SD-OCT devices in healthy and diseased eyes concluded that different devices provided different measurements, suggesting that the comparison between OCT instruments is nearly impossible.17–22

The advantage of the study is that by using eyes with dry AMD, we found a highly significant correlation between measurements generated by the two instruments, with the Nidek OCT macular thickness evaluation being constantly higher than Cirrus OCT. These results indicate that these two OCT instruments could be used interchangeably. To reliably apply a correction factor, the user needs to make sure that significant segmentation errors are not present.

The delineation of the retinal outer boundary was clearly different for the two instruments. In addition to this systematic difference in the segmentation protocol employed by the two devices, it is important to note that possible artifacts in retinal segmentation specific to individual diseased cases may be a source in the variability of measurements. Other important sources of variability in measurements include variable or poor image quality and pathologic alterations which specifically affect the inner and/or outer boundaries. Giani et al, for example, showed that mean retinal thickness measurements are also different between devices that set the outer boundary at the same level.17 Our study showed that differences in macular thickness generated by the two instruments are not different between eyes with dry AMD and healthy controls, suggesting that mostly segmentation algorithms rather than specific anatomical features of dry AMD may be responsible for the offset.

The limitation of the study may be that our results apply to a specific retinal pathology: dry AMD. Also, in order to yield more accurate conversion factors by reducing the standard error in regression analysis, larger sample sizes could be considered.

In conclusion, macular thickness measurements generated by the Nidek and Cirrus OCT instruments in dry AMD eyes are highly correlated, showing a consistent difference. This suggests that a correction factor may be applied to inter-relate measurements between devices in future dry AMD clinical studies.

References

  1. Huang D, Swanson EA, Lin CP, et al. Optical coherence tomography. Science. 1991;254(5035):1178–1181. doi:10.1126/science.1957169 [CrossRef]
  2. Puliafito CA, Hee MR, Lin CP, et al. Imaging of macular diseases with optical coherence tomography. Ophthalmology. 1995;102(2):217–229. doi:10.1016/S0161-6420(95)31032-9 [CrossRef]
  3. Drexler W, Fujimoto JG. State-of-the-art retinal optical coherence tomography. Prog Retin Eye Res. 2008;27(1):45–88. doi:10.1016/j.preteyeres.2007.07.005 [CrossRef]
  4. Fujimoto H, Gomi F, Wakabayashi T, Sawa M, Tsujikawa M, Tano Y. Morphologic changes in acute central serous chorioretinopathy evaluated by fourier-domain optical coherence tomography. Ophthalmology. 2008;115(9):1494–1500, 1500.e1–2 doi:10.1016/j.ophtha.2008.01.021 [CrossRef]
  5. Alkuraya H, Kangave D, Abu El-Asrar AM. The correlation between optical coherence tomographic features and severity of retinopathy, macular thickness and visual acuity in diabetic macular edema. Int Ophthalmol. 2005;26(3):93–99. doi:10.1007/s10792-006-9007-8 [CrossRef]
  6. Wilkins JR, Puliafito CA, Hee MR, et al. Characterization of epiretinal membranes using optical coherence tomography. Ophthalmology. 1996;103(12):2142–2151. doi:10.1016/S0161-6420(96)30377-1 [CrossRef]
  7. Karam EZ, Ramirez E, Arreaza PL, Morales-Stopello J. Optical coherence tomographic artefacts in diseases of the retinal pigment epithelium. Br J Ophthalmol. 2007;91(9):1139–1142. doi:10.1136/bjo.2007.114074 [CrossRef]
  8. Ko TH, Fujimoto JG, Duker JS, et al. Comparison of ultrahigh- and standard-resolution optical coherence tomography for imaging macular hole pathology and repair. Ophthalmology. 2004;111(11):2033–2043. doi:10.1016/j.ophtha.2004.05.021 [CrossRef]
  9. Liakopoulos S, Ongchin S, Bansal A, et al. Quantitative optical coherence tomography findings in various subtypes of neovascular age-related macular degeneration. Invest Ophthalmol Vis Sci. 2008;49(11):5048–5054. doi:10.1167/iovs.08-1877 [CrossRef]
  10. Muscat S, Parks S, Kemp E, Keating D. Repeatability and reproducibility of macular thickness measurements with the Humphrey OCT system. Invest Ophthalmol Vis Sci. 2002;43(2):490–495.
  11. Massin P, Vicaut E, Haouchine B, Erginay A, Paques M, Gaudric A. Reproducibility of retinal mapping using optical coherence tomography. Arch Ophthalmol. 2001;119(8):1135–1142. doi:10.1001/archopht.119.8.1135 [CrossRef]
  12. Alam S, Zawadzki RJ, Choi S, et al. Clinical application of rapid serial fourier-domain optical coherence tomography for macular imaging. Ophthalmology. 2006;113(8):1425–1431. doi:10.1016/j.ophtha.2006.03.020 [CrossRef]
  13. Chen TC, Cense B, Pierce MC, et al. Spectral domain optical coherence tomography: Ultra-high speed, ultra-high resolution ophthalmic imaging. Arch Ophthalmol. 2005;123(12):1715–1720. doi:10.1001/archopht.123.12.1715 [CrossRef]
  14. Forte R, Cennamo GL, Finelli ML, de Crecchio G. Comparison of time domain Stratus OCT and spectral domain SLO/OCT for assessment of macular thickness and volume. Eye (Lond). 2009;23(11):2071–2078. doi:10.1038/eye.2008.363 [CrossRef]
  15. Ibrahim MA, Sepah YJ, Symons RC, et al. Spectral- and time-domain optical coherence tomography measurements of macular thickness in normal eyes and in eyes with diabetic macular edema. Eye (Lond). 2012;26(3):454–462. doi:10.1038/eye.2011.293 [CrossRef]
  16. Sull AC, Vuong LN, Price LL, et al. Comparison of spectral/Fourier domain optical coherence tomography instruments for assessment of normal macular thickness. Retina. 2010;30(2):235–245. doi:10.1097/IAE.0b013e3181bd2c3b [CrossRef]
  17. Giani A, Cigada M, Choudhry N, et al. Reproducibility of retinal thickness measurements on normal and pathologic eyes by different optical coherence tomography instruments. Am J Ophthalmol. 2010;150(6):815–824. doi:10.1016/j.ajo.2010.06.025 [CrossRef]
  18. Huang J, Liu X, Wu Z, Xiao H, Dustin L, Sadda S. Macular thickness measurements in normal eyes with time-domain and Fourier-domain optical coherence tomography. Retina. 2009;29(7):980–987. doi:10.1097/IAE.0b013e3181a2c1a7 [CrossRef]
  19. Ho J, Sull AC, Vuong LN, et al. Assessment of artifacts and reproducibility across spectral- and time-domain optical coherence tomography devices. Ophthalmology. 2009;116(10):1960–1970. doi:10.1016/j.ophtha.2009.03.034 [CrossRef]
  20. Wolf-Schnurrbusch UE, Ceklic L, Brinkmann CK, et al. Macular thickness measurements in healthy eyes using six different optical coherence tomography instruments. Invest Ophthalmol Vis Sci. 2009;50(7):3432–3437. doi:10.1167/iovs.08-2970 [CrossRef]
  21. Leung CK, Cheung CY, Weinreb RN, et al. Comparison of macular thickness measurements between time domain and spectral domain optical coherence tomography. Invest Ophthalmol Vis Sci. 2008;49(11):4893–4897. doi:10.1167/iovs.07-1326 [CrossRef]
  22. Giammaria D, Ioni A, Bartoli B, Cofini V, Pellegrini G, Giannotti B. Comparison of macular thickness measurements between time-domain and spectral-domain optical coherence tomographies in eyes with and without macular abnormalities. Retina. 2011;31(4):707–716. doi:10.1097/IAE.0b013e3181eef401 [CrossRef]
Authors

From Doheny Image Reading Center, Doheny Eye Institute, Los Angeles (TCT, AHH, SB, SVRS); and the Department of Ophthalmology, David Geffen School of Medicine at UCLA, Los Angeles (TCT, AHH, SB, SVRS).

Dr. Sadda is a consultant for and receives research support from Optos and Carl Zeiss Meditec; serves as a consultant for Centervue; and has access to research instruments provided by Heidelberg Engineering, Topcon Medical Systems, Optos, Carl Zeiss Meditec, Nidek, and Centervue. The remaining authors report no relevant financial disclosures.

Address correspondence to SriniVas R. Sadda, MD, Doheny Eye Institute, 1355 San Pablo Street, DVRC 211, Los Angeles, CA 90033; email: SSadda@doheny.org.

Received: July 06, 2017
Accepted: November 02, 2017

10.3928/23258160-20180601-05

Sign up to receive

Journal E-contents