From the Fundus Photograph Reading Center, Department of Ophthalmology and Visual Sciences, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin.
Supported by Topcon Medical Systems, USA, Inc. through a research grant to the University of Wisconsin (Principle Investigator: RPD) and funded in part by an unrestricted grant from Research to Prevent Blindness, Inc.
The authors have no financial or proprietary interest in the materials presented herein.
Address correspondence to Ronald P. Danis, MD, Fundus Photograph Reading Center, 8010 Excelsior Drive, Madison, WI, 53713. E-mail: firstname.lastname@example.org
Images and standardized reports from time-domain optical coherence tomography (TD-OCT) have extensively guided clinical decision making in retinal disease since the release of the Stratus OCT (Carl Zeiss Meditec, Inc., Dublin, CA) “third generation” machine in 1995. The software-generated retinal thickness measurements have been widely used in clinical research. Multicenter clinical trials have included large numbers of clinical sites all using the same make and model TD-OCT instrument to ensure standardization of data. This era is ending, given the commercial availability of spectral-domain OCT (SD-OCT) from different manufacturers and the increasing market share of these new instruments, displacing the Stratus in many offices.
The standardized report used in common between TD-OCT and SD-OCT employs a macular grid centered on the fovea, based on the ETDRS grid used for photographic grading. Retinal thickness values are often used for study eligibility (eg, a specified minimum center subfield thickness) and to define endpoints (eg, proportion of eyes in each group with a decrease in center subfield thickness of a specified number of microns over time).1–3 The repeatability of measurements becomes a critical factor in predicting the variability to be factored in sample size calculation and, perhaps more importantly, determining what extent of change between visits is likely to be significant.4,5 The reproducibility of the macular thickness measurements of the Stratus is well established in both normal and diseased maculae.4,6,7 SD-OCT systems are commercially available from multiple manufacturers and each instrument has slightly different hardware and software, requiring separate evaluation of each instrument type. The purpose of this study was to compare the intra-session repeatability and the retinal thickness measurements of the Topcon 3D OCT 1000 (Topcon Medical Systems, Paramus, NJ), an SD-OCT machine, and the Stratus, a TD-OCT machine.
Patients and Methods
Patients requiring TD-OCT scans as part of their standard care for known retinal disease were eligible for the study. All subjects were recruited from a single academic clinic at the University of Wisconsin–Madison. Both eyes were scanned, even if the fellow eye was not suspected of having retinal disease. The study was performed with patient informed consent and under approval of the institutional review board in accordance with the ethical standards stated in the Declaration of Helsinki.
Pupils were dilated pharmacologically using one drop of tropicamide 1% and one drop of phenylephrine 2.5%, per normal clinic routine. All subjects underwent four scans in each eye, two scans with the TD-OCT and two scans with the SD-OCT. The scans were obtained by one of three experienced operators, each of whom has been certified for submission of TD-OCT scans in clinical trials. All scans of a single subject were obtained by the same operator consecutively at a single clinic visit.
The Stratus scan protocol included the Fast Macular Thickness Map pattern, which is composed of six high speed, 6-mm radial scans with a resolution of 128 A-scans per B-scan acquired in 1.9 seconds. The SD-OCT cube scanning protocol was composed of 128 B-scans sampling a 6 × 6 mm square area with resolution of 512 A-scans per B-scan, taken in 2.4 seconds.
All images were analyzed independently in their respective review software as supplied by the manufacturers (Stratus Review Software version 5.0 and Topcon 3D OCT 1000 Review Software) by two of the authors (AD and SG). Scans were categorized as poor quality if there was severe decentration (misalignment of the macular grid > 500 μm from the center of the macula), if there were substantial boundary line errors (inaccurate machine segmentation of tissue layers) considered sufficient to change any subfield thickness by greater than 10%, or both.8 A boundary line error was considered substantial on examination of TD-OCT scans if the measurement error in a single scan or cumulative error across multiple scans was estimated to be greater than 10% of the true retinal thickness in any subfield. In SD-OCT scans, boundary line errors were considered substantial if they involved greater than 10% of the retinal thickness of the subfield in at least five consecutive scans. Software provisions for retrieving data from poor quality scans, such as grid repositioning in decentered scans and manual correction of boundary line errors, were not employed.
The individual subfield thickness measurements were recorded from the software-generated retinal thickness map reports. With the SD-OCT software, two such map reports were generated: one using the retinal pigment epithelial layer as the outer boundary of the retina and the other using the photoreceptor inner segment–outer segment junction as the outer boundary (Fig. 1). The report measuring to the retinal pigment epithelial boundary was used for evaluating repeatability. The mean of the thickness measurements of replicate scans taken on each machine was used for comparison analysis.
Figure 1. Section of B-Scan Showing Two Different Outer Boundary Limit Options in Topcon 3D OCT 1000 (Topcon Medical Systems, Paramus, NJ): Upper Border of Retinal Pigment Epithelium (center Subfield Thickness = 264 μm, Top) and Inner Segment–Outer Segment Junction (center Subfield Thickness = 239 μm, Middle). Corresponding Stratus (Carl Zeiss Meditec, Inc., Dublin, CA) Image (center Subfield Thickness = 226 μm, Bottom) Is Shown for Comparison.
Assessment of repeatability addresses the difference that might be obtained between two measurements made on the same subject by the same operator using the same apparatus within a short interval of time.9 Our previous publication found the intra-session mean difference in centerpoint retinal thickness using Stratus is 2 μm (standard deviation = 25).10 On the basis of this finding, the sample size was calculated to be 128 eyes to detect an expected intra-session difference in mean centerpoint retinal thickness of 5 μm, assuming a standard deviation of 20 or less in SD-OCT.
The coefficient of repeatability and the coefficient of variation were calculated. The coefficient of repeatability is a clinically important measure and provides an estimate of the 95% probability of the magnitude (in microns) beyond which a difference in thickness between two scans of the same eye is likely to represent true change rather than measurement variability (lower coefficient values indicate greater repeatability). The coefficient of variation gives an estimate of the dispersion of the standard deviation around the mean and is expressed as a percentage. Previous studies have found that coefficient of repeatability (μm) varied with the magnitude of retinal thickness, but remained relatively constant when expressed as the coefficient of variation (%).4
The repeatability of the two machines was compared with a paired t test on log-transformed data of within-subject variance. The 95% limits of agreement between the retinal thickness measurements of the two machines were calculated.9,11 All statistical analysis was performed using SAS software (version 9.1; SAS Institute, Inc., Cary, NC).
Paired scans of 127 eyes (68 subjects) were available for analysis. Eyes excluded had one of two principal artifact types: decentration (malpositioning of the scan grid with the macula) and boundary line error (inaccurate machine segmentation of tissue layers).8 Decentration was the primary artifact among SD-OCT scans, whereas boundary line errors predominated in TD-OCT scans (Table 1). Sixty-three eyes had no artifact in both pairs of scans from both instruments in all of the nine grid subfields (ie, all four scans available for an eye were of high quality).
Table 1: Quality Evaluation of 127 Pairs of Scans Obtained with Topcon 3D OCT 1000 (SD-OCT) and Stratus (TD-OCT) in 127 Eyes with Retinal Disease
The coefficient of repeatability and the coefficient of variation for retinal thickness in the central, inner, and outer subfields and for total macular volume are listed in Table 2. The coefficient of repeatability for the central subfield was 20.1 μm (2.6%) in the SD-OCT subset and 27.4 μm (4%) in the TD-OCT subset. Repeatability was greater with SD-OCT in the inner and outer subfields. There was no significant difference in repeatability of the central subfield thickness between the two machines (P = .38). Five of the eight outer subfields showed better repeatability with SD-OCT (P < .01). No important differences in repeatability by disease subtype were seen, although the numbers were small in each group (Table 3).
Table 2: Intrasession Repeatability Analysis of Individual Subfield Retinal Thickness Measurements with Topcon 3D OCT 1000 (SD-OCT) and Stratus (TD-OCT)
Table 3: Disease-Specific Comparison of Central Subfield Thickness and Intrasession Repeatability Between Topcon 3D OCT 1000 (SD-OCT) and Stratus (TD-OCT)
The retinal thickness measurements were compared between SD-OCT and TD-OCT for the central subfield, the mean of the four inner subfields, and the mean of the four outer subfields (Table 4). Using the retinal pigment epithelial layer as the outer boundary line in SD-OCT scans, the mean difference in the central subfield was 24.5 μm (standard deviation = 20.9) and decreased in the peripheral subfields. The measurements were larger in the SD-OCT than in the TD-OCT. The agreement of the two machines for central subfield thickness is shown in Bland–Altman plots (Fig. 2) with 95% limits between −16.5 and 65.5 μm. When the outer boundary line in SD-OCT was selected as the inner segment–outer segment junction, the mean difference in thickness measurements was smaller. A disease-specific trend could not be interpreted in the mean differences between the thickness measurements by the two machines due to small numbers (Table 3).
Table 4: Comparison of Retinal Thickness Measurements Between Topcon 3D OCT 1000 (SD-OCT) and Stratus (TD-OCT)
Figure 2. Bland–Altman Plot for Agreement Between Central Subfield Thickness Measurement Between Topcon 3D OCT 1000 (Topcon Medical Systems, Paramus, NJ) and Stratus (Carl Zeiss Meditec, Inc., Dublin, CA). The Mean Difference (solid Line) Is 24.5 μm and the 95% Limits of Agreement (dashed Lines) Are Between −16.5 and 65.5 μm.
Retinal thickness measurements on diseased maculae performed using the Topcon 3D OCT 1000 had intra-session repeatability comparable to that of the Stratus in eyes with diverse retinal pathology. We observed a coefficient of repeatability for the central subfield thickness of 20.1 μm (2.6%) with SD-OCT and 27.4 μm (4%) with TD-OCT. These results are consistent with a previous study of normal retinas that reported a coefficient of repeatability of 14.5 μm (2.4%) for Topcon 3D OCT 1000 and 17.4 μm (3.2%) for Stratus.12 Another study reported a coefficient of repeatability of 19 μm (2.4%) for the central subfield by the Cirrus (Carl Zeiss Meditec, Inc.) and 18 μm (2.6%) by the Stratus in eyes with diabetic macular edema.13 The repeatability of the Stratus has been tested previously by several other groups and our results are similar to theirs.4,6,10
The repeatability of OCT measurements depends on many factors, including image artifacts confounding the measurements, improper centration of the scan, and different sampling densities due to the scanning patterns. We excluded all eyes where one or both pairs of scans had substantial artifacts that could invalidate thickness measurements, such as boundary line errors and decentration. Eyes with less extensive artifacts were not excluded due to their presumed minimal effect on the repeatability and thickness comparison between the two instruments.
The number of eyes excluded was somewhat higher in the SD-OCT subset than the TD-OCT subset: 36 (28%) versus 26 (20%). However, most of the excluded eyes in the SD-OCT set were due to or included decentration, which is typically an operator error rather than an instrument error.8 This could be related to the relative inexperience of the operators with the less-familiar SD-OCT machine. Eyes with age-related macular degeneration had a higher proportion of artifacts, consistent with prior reports.8,14
In the SD-OCT review software and recent versions of Stratus software (version 5.0 and above), poor quality scans can be corrected post hoc with manual boundary line redrawing, allowing an operator to move erroneous segmentation lines to the correct position. However, the procedure for correcting the boundary lines of each B-scan and re-analyzing the thickness map is labor intensive and not suitable for the analysis of large numbers of scans. In the Topcon 3D OCT 1000 software, grid repositioning can be accomplished by manually shifting the grid placement to center it on the fovea, after which the retinal thickness map is automatically recalculated. This feature, which is not available in the Stratus, could have salvaged thickness reports from scans where decentration was the only artifact.
TD-OCT and SD-OCT machines differ with respect to scan density. The grid values of the Stratus are extrapolated from 768 data points. Because of the radial scan algorithm, sampling density is greater near the center and relatively sparse at the periphery of the grid. In contrast, the scan protocol we used for the Topcon 3D 1000 has approximately 65,000 data points, with raster line scans distributed equally across the grid. The greater scan density would be expected to produce better repeatability of thickness measurements. Our data show that, although repeatability was similar for the central subfield, the difference appeared to emerge in the outer subfields, with better repeatability of the SD-OCT compared to the TD-OCT. This difference in scan density probably also plays an important role in the repeatability of total macular volume (calculated as the sum of the volumes of each grid subfield [mean thickness multiplied by the area of each subfield]).
The boundary line segmentation algorithm developed for the Stratus identifies the outer boundary line as the first prominent hyperreflective band external to the internal limiting membrane. Originally thought to be the top of the retinal pigment epithelial layer, this layer was later demonstrated to be the junction between the inner and outer segments of the photoreceptors.15 In contrast, the segmentation algorithm can be set to identify the top of the retinal pigment epithelium as the outer layer in the Topcon 3D OCT 1000. Other SD-OCT machines appear to set the outer layer in various locations.16 This results in a systematic difference in thickness measurements between the various SD-OCT machines and the Stratus.13,16–18
We found a mean absolute difference of 28.1 μm, with considerable variability, between the two machines for the central subfield in a sample with diverse retinal disease. In healthy retinas, the mean difference between the machines has been reported as 20 μm.12 When the outer boundary line detection algorithm in the Topcon 3D OCT 1000 was set to simulate the Stratus’ software segmentation algorithm, the mean absolute measurement difference was halved to 14.5 μm for central subfield (Table 4). Among normal eyes, the mean absolute difference was 0 μm (data not shown). This suggests that the difference between the measurements of the two machines is primarily due to the segmentation algorithm.
Other studies comparing the Cirrus and the Stratus in normal and diseased retinas have reported a difference of approximately 50 μm.13,19,20 No published studies have directly compared the measurements between the Topcon 3D OCT 1000 and the Cirrus in the same eyes. However, comparison of various other SD-OCT systems has shown that measurements between SD-OCT machines vary.18
The repeatability of retinal thickness measurements with the Topcon 3D OCT 1000 was comparable to the Stratus. Thus, a change in thickness of more than 20 μm or 2.6% in the central subfield is likely to be real (rather than measurement artifact) when measured with Topcon 3D OCT 1000 based on our repeatability assessment. Future studies are required to investigate whether repeatability varies by macular disease. If so, then a different threshold may be required to identify clinically meaningful differences in thickness measurement for specific macular disease categories. With the Stratus, our analysis showed that a significant change would have a threshold of 27 μm or 4% of the measurement. Despite the robust repeatability of each instrument, the limits of agreement can be up to 65 μm between the two machines in excellent quality scans. The retinal thickness measurements between the Topcon 3D OCT 1000 and the Stratus can differ significantly and may not be used interchangeably.
- Arevalo JF, Fromow-Guerra J, Quiroz-Mercado H, et al. Primary intravitreal bevacizumab (Avastin) for diabetic macular edema: results from the Pan-American Collaborative Retina Study Group at 6-month follow-up. Ophthalmology. 2007;114:743–750. doi:10.1016/j.ophtha.2006.12.028 [CrossRef]
- Cunningham ET Jr, Adamis AP, Altaweel M, et al. A phase II randomized double-masked trial of pegaptanib, an anti-vascular endothelial growth factor aptamer, for diabetic macular edema. Ophthalmology. 2005;112:1747–1757. doi:10.1016/j.ophtha.2005.06.007 [CrossRef]
- Fung AE, Lalwani GA, Rosenfeld PJ, et al. An optical coherence tomography-guided, variable dosing regimen with intravitreal ranibizumab (Lucentis) for neovascular age-related macular degeneration. Am J Ophthalmol. 2007;143:566–583. doi:10.1016/j.ajo.2007.01.028 [CrossRef]
- Diabetic Retinopathy Clinical Research Network. Reproducibility of macular thickness and volume using Zeiss optical coherence tomography in patients with diabetic macular edema. Ophthalmology. 2007;114:1520–1525.
- Patel PJ, Chen FK, Ikeji F, et al. Repeatability of Stratus optical coherence tomography measures in neovascular age-related macular degeneration. Invest Ophthalmol Vis Sci. 2008;49:1084–1088. doi:10.1167/iovs.07-1203 [CrossRef]
- Polito A, Del Borrello M, Isola M, Zemella N, Bandello F. Repeatability and reproducibility of fast macular thickness mapping with Stratus optical coherence tomography. Arch Ophthalmol. 2005;123:1330–1337. doi:10.1001/archopht.123.10.1330 [CrossRef]
- Massin P, Vicaut E, Haouchine B, Erginay A, Paques M, Gaudric A. Reproducibility of retinal mapping using optical coherence tomography. Arch Ophthalmol. 2001;119:1135–1142.
- Domalpally A, Danis RP, Zhang B, Meyers D, Kruse CN. Quality issues in interpretation of optical coherence tomograms in macular diseases. Retina. 2009;29:775–781. doi:10.1097/IAE.0b013e3181a0848b [CrossRef]
- Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1:307–310.
- Danis RP, Fisher MR, Lambert E, Goulding A, Wu D, Lee LY. Results and repeatability of retinal thickness measurements from certification submissions. Arch Ophthalmol. 2008;126:45–50. doi:10.1001/archopht.126.1.45 [CrossRef]
- Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res. 1999;8:135–160. doi:10.1191/096228099673819272 [CrossRef]
- Leung CK, Cheung CY, Lin D, Pang CP, Lam DS, Weinreb RN. Comparison of macular thickness measurements between time domain and spectral domain optical coherence tomography. Invest Ophthalmol Vis Sci. 2008;49:4893–4897. doi:10.1167/iovs.07-1326 [CrossRef]
- Forooghian F, Cukras C, Meyerle CB, Chew EY, Wong WT. Evaluation of time domain and spectral domain optical coherence tomography in the measurement of diabetic macular edema. Invest Ophthalmol Vis Sci. 2008;49:4290–4296. doi:10.1167/iovs.08-2113 [CrossRef]
- Sadda SR, Wu Z, Walsh AC, et al. Errors in retinal thickness measurements obtained by optical coherence tomography. Ophthalmology. 2006;113:285–293. doi:10.1016/j.ophtha.2005.10.005 [CrossRef]
- Costa RA, Calucci D, Skaf M, et al. Optical coherence tomography 3: Automatic delineation of the outer neural retinal boundary and its influence on retinal thickness measurements. Invest Ophthalmol Vis Sci. 2004;45:2399–2406. doi:10.1167/iovs.04-0155 [CrossRef]
- Wolf-Schnurrbusch UE, Ceklic L, Brinkmann CK, et al. Macular thickness measurements in healthy eyes using six different optical coherence tomography instruments. Invest Ophthalmol Vis Sci. 2009;50:3432–3437. doi:10.1167/iovs.08-2970 [CrossRef]
- Leung CK, Chan WM, Chong KK, et al. Alignment artifacts in optical coherence tomography analyzed images. Ophthalmology. 2007;114:263–270. doi:10.1016/j.ophtha.2006.06.059 [CrossRef]
- Han IC, Jaffe GJ. Comparison of spectral- and time-domain optical coherence tomography for retinal thickness measurements in healthy and diseased eyes. Am J Ophthalmol. 2009;147:847–858. doi:10.1016/j.ajo.2008.11.019 [CrossRef]
- Legarreta JE, Gregori G, Punjabi OS, Knighton RW, Lalwani GA, Puliafito CA. Macular thickness measurements in normal eyes using spectral domain optical coherence tomography. Ophthalmic Surg Lasers Imaging. 2008;39(4 suppl):S43–S49.
- Kiernan DF, Hariprasad SM, Chin EK, Kiernan CL, Rago J, Mieler WF. Prospective comparison of Cirrus and Stratus optical coherence tomography for quantifying retinal thickness. Am J Ophthalmol. 2009;147:267–275. doi:10.1016/j.ajo.2008.08.018 [CrossRef]
Quality Evaluation of 127 Pairs of Scans Obtained with Topcon 3D OCT 1000 (SD-OCT) and Stratus (TD-OCT) in 127 Eyes with Retinal Disease
|Eyes with artifacta||36 (28%)||26 (20%)|
| Boundary line errors||15 (12%)||19 (15%)|
| Decentration||17 (13%)||2 (2%)|
| Boundary line errors and decentration||3 (2%)||0 (0%)|
| Other artifacts||4 (3%)||5 (4%)|
|Eyes with artifact by disease|
| AMD (n = 51)||18 (35%)||16 (31%)|
| DME (n = 28)||7 (25%)||6 (21%)|
| RVO (n = 11)||1 (9%)||0 (0%)|
| Others (n = 37)||10 (27%)||4 (11%)|
Intrasession Repeatability Analysis of Individual Subfield Retinal Thickness Measurements with Topcon 3D OCT 1000 (SD-OCT) and Stratus (TD-OCT)a
|Subfield||Coefficient of Repeatability (μm)||Coefficient of Variation (%)|
Disease-Specific Comparison of Central Subfield Thickness and Intrasession Repeatability Between Topcon 3D OCT 1000 (SD-OCT) and Stratus (TD-OCT)a
|Disease Type||No.||Mean Central Subfield Thickness (μm) (SD)||Mean Absolute Difference (μm) (SD)||Coefficient of Repeatability (μm)||Coefficient of Variation (%)|
|AMD||20||246.1 (44.3)||224.3 (44.3)||23.2 (13.8)||16.9||21.2||2.5||3.4|
|DME||15||313.4 (68.0)||296.4 (68.1)||30.0 (23.0)||23.4||46.3||2.7||5.6|
|RVO||7||321.8 (137.7)||282.9 (138.5)||38.9 (9.6)||18.2||14.9||2.0||1.9|
|Others||21||257.5 (49.2)||229.7 (47.8)||27.8 (11.0)||20.9||14.9||2.9||2.3|
Comparison of Retinal Thickness Measurements Between Topcon 3D OCT 1000 (SD-OCT) and Stratus (TD-OCT)a
|Subfield||Mean Retinal Thickness (μm) (SD)||Mean Absolute Difference (μm) (SD)||Mean Difference (μm) (SD)||Bland–Altman 95% Limits of Agreement||Pb|
|Using RPE as outer boundary in SD-OCT|
| Central||274.3 (72.4)||249.8 (72.4)||28.1 (15.7)||24.5 (20.9)||−16.5, 65.5||< .001|
| Pooled inners||297.6 (38.1)||280.5 (37.6)||17.2 (6.6)||17.1 (6.9)||3.6, 30.6||< .001|
| Pooled outers||256.2 (30.8)||242.9 (32.0)||13.3 (4.8)||13.3 (5.0)||3.5, 23.1||< .001|
| Total volume||7.52 (0.89)||7.13 (0.91)||0.39 (0.13)||0.39 (0.13)||0.14, 0.64||< .001|
|Using IS-OS junction as outer boundary in SD-OCT|
| Central||256.4 (74.9)||249.8 (72.4)||14.5 (15.9)||6.6 (20.6)||−33.8, 47.0||< .001|
| Pooled inners||281.3 (37.6)||280.5 (37.6)||5.1 (6.7)||0.7 (8.4)||−15.8, 17.2||< .001|
| Pooled outers||240.5 (32.2)||242.9 (32.0)||4.3 (3.5)||−2.5 (5.0)||−12.3, 7.3||< .001|
| Total volume||7.08 (0.93)||7.13 (0.91)||0.11 (0.09)||−0.05 (0.14)||−0.32, 0.22||< .001|