From Doheny Eye Institute (RKK, MGN, SRS) and the Department of Ophthalmology (SRS), Keck School of Medicine of the University of Southern California, Los Angeles, California; and the Casey Eye Institute (OT, DH), Oregon Health & Science University, Portland, Oregon.
Supported by National Institutes of Health Grants R01 EY013516 and P30 EY03040 and by a grant from Research to Prevent Blindness.
Dr. Tan receives grant support from Optovue, Inc., Fremont, California. Dr. Huang receives royalties from an optical coherence tomography patent licensed to Carl Zeiss Meditec, Inc., Dublin, California, and stock options, travel support, research grants, speaker honorarium, and patent royalties from Optovue, Inc. Dr. Sadda has previously shared in royalties for intellectual property licensed to Topcon Medical Systems by the Doheny Eye Institute, has served as a consultant to Heidelberg Engineering, and receives research support from Optovue, Inc., Carl Zeiss Meditec, and Optos. The remaining authors have no financial or proprietary interest in the materials presented herein.
Address correspondence to Srinivas R. Sadda, MD, Doheny Eye Institute, 1450 San Pablo Street DEI 3623, Los Angeles, CA 90033. E-mail: firstname.lastname@example.org
Optical coherence tomography (OCT)1 is commonly used to evaluate various retinal diseases,2 such as vitreomacular interface disorders,3 retinal vascular diseases,4,5 age-related macular degeneration, choroidal neovascularization,6 and glaucoma.7,8 These diseases constitute the major causes of blindness in the United States and in other developed nations.9
Despite its many attributes and advantages, OCT has still not been considered a useful tool among ophthalmic imaging techniques, such as fluorescein angiography, for assessing retinal blood flow. However, retinal blood flow information can potentially be extracted from Doppler signals present within the OCT data. This technique of Doppler OCT imaging has been used to calculate blood flow in retinal vascular diseases, such as diabetic retinopathy, in which retinal blood flow was shown to be reduced in patients with diabetes mellitus compared to normal subjects.10 Although blood flow variations are well recognized in retinal venous occlusive diseases, glaucoma, and other vascular and perfusion disorders of the optic nerve head, they have not been quantitatively well demonstrated.11–14 Previous research suggests that quantitative volumetric measurement of total retinal blood flow (TRBF) in the eye may provide new insights into the pathogenesis of retinal disease.10,12,14,15
Different approaches to studying retinal blood flow have been suggested. One potential approach is to collect the Doppler flow signal from Fourier-domain OCT (FD-OCT) images, as demonstrated by Wehbe et al.16 and Wang et al.17 Accurate quantification of blood flow for use in research investigations has been previously demonstrated by successful calculation of TRBF with a semi-automated process, in which candidate vessels are identified by computer algorithms and the results are manually refined by human experts.10,17–19 The reproducibility of this approach is of critical importance for future research in this area, particularly for assessing the significance of changes in blood flow in longitudinal studies; however, such reproducibility has yet to be established.
We report the intergrader reproducibility of a variety of Doppler blood flow measurements, including TRBF, superior hemisphere flow, inferior hemisphere flow, venous area, and venous velocity.
Patients and Methods
Patients who presented to the Doheny Eye Institute for ophthalmic examination were recruited for a prospective study of Doppler OCT retinal blood flow analysis in normal and glaucomatous eyes. To be included in the study, patients had to be willing to have at least one eye scanned and had to have ocular media sufficiently clear to permit OCT imaging. For this reproducibility study, 20 eyes (10 eyes with a diagnosis of glaucoma and 10 normal eyes) from 18 consecutive subjects were randomly selected from the University of Southern California Advanced Imaging for Glaucoma Study (AIGS) dataset [ www.aigstudy.net]. The diagnostic criteria for normal and glaucomatous eyes have been described in a previous publication.8 The research protocols were approved by the Institutional Review Board of the University of Southern California and adhered to the tenets set forth in the Declaration of Helsinki.
Doppler Scan Acquisitions
Doppler scans were obtained with the RTVue FD-OCT (Optovue Inc., Fremont, CA) using the previously published dual circular scan protocol.18 The scan pattern consisted of two concentric circles (inner and outer ring diameters are 3.4 and 3.75 mm, respectively) around the optic nerve head, transecting all large retinal vessels (arteries and veins) near the optic nerve head in two locations. During a single scan acquisition sequence, a total of six dual circular scan frames were obtained and averaged for each ring. To ensure that high quality scans were available for grading, the acquisition sequence was repeated five to six times, thus yielding 30 to 36 frames for each ring for subsequent analysis. The Doppler shift in the vessel and the Doppler angle of the vessel were used to compute flow. The Doppler angle was estimated by vessel center depth difference between the two rings of a scan. The TRBF was calculated by adding flows from all branch veins at one time point.18
Preparing the Data and Manual Grading
Doppler scans and three-dimensional disk scan from the RTVue SD-OCT instrument were exported as raw data using the RTVue Doppler transfer output software. The data were then converted to the Doppler OCT of Retinal Circulation (DOCTORC) Grading Software compatible data format using ReVue software (Optovue Inc.), and then loaded into the DOCTORC software for initial automated assessment and identification of candidate vessel locations and vessel boundaries.
After the initial automatic processing and candidate vessel identification, Doppler scans were registered with the OCT projection image, infrared scanning laser ophthalmoscopy image, and a color disc fundus photograph. This process allows the grader to correlate candidate vessels seen on the Doppler OCT B-scans with vessels visible on the en face images and facilitates classification of vessels as arteries and veins (Fig. 1). Scans were manually reviewed by certified Doppler OCT reading center graders to verify and refine the blood vessel position and to check the accuracy of vessel classification type (artery vs vein). The grading procedure included adjusting vessel locations and vessel size, matching corresponding vessels on the two rings, adding any missing vessels, deleting extraneous vessels, and verifying the vessel type classification. Graders used a variety of clues to identify vessel positions on B-scans, including characteristic hyperreflectivity and posterior shadowing on OCT, and the position of vessels on ancillary images, such as the registered OCT projection images, color images, and infrared image. Figure 1A shows the DOCTORC software version 2 display of an OCT projection image of the fundus used to facilitate identification of blood vessel location. Figure 2 shows a cross-sectional image of the vessel on the OCT B-scan.
Figure 1. (A) Reconstructed fundus image of eye from spectral-domain optical coherence tomography (SD-OCT). Numbers identify the vessel location and color indicates type of vessel detected: red = artery; blue = vein. (B) Disk fundus photograph used for comparison of reconstructed fundus image from SD-OCT of the same patient to help identify the type of vessel as an artery or a vein.
Figure 2. Cross-sectional image of the vessels for grading. The grader marks the vessel circumference using a dotted sizing circle, and the diameter using a horizontal caliper line, used for vessel area calculation and angle estimation between two rings of the scan.
Two independent trained graders (RK & OT) completed manual grading by verifying the software-detected blood vessel position and its area, changing or adjusting its borders if necessary, and adding or deleting the vessel as needed as described above. The DOCTORC software graphical user interface allows the user to adjust the position of the vessel center and the width or largest horizontal diameter of the vessel. After the vessel position and its area were adjusted in both the outer and inner rings corresponding to that particular vessel, a confidence score was assigned by the grader to each vessel on every scan to reflect the grader’s certainty regarding the accuracy of the vessel grading. Confidence scores ranged from 0 to 5 and were influenced by several factors, including Doppler signal strength of the vessel, clarity of the vessel boundary, and vessel size agreement between the two concentric scan rings.
This subjective confidence score, together with the calculated Doppler angle and the variance of Doppler angle, served as the key criteria in an automated quality check that transformed each parameter into a probability and then combined all parameters into a single probability metric using Bayesian rules. This quality check was used to determine whether the flow result of a particular vessel could be considered valid and could be included in the calculation of the total blood flow. The average speed of flow in all veins was computed by summing the flow in all valid veins and dividing by the sum of the cross-sectional area of all valid veins. In cases where a retinal vessel did not pass the validity check, the flow in this vessel was estimated from its vessel area using the mean flow speed of all valid veins (ie, flow was effectively interpolated across these invalid vessels). Total retinal blood flow was computed by summing the calculated flow of valid veins and the estimated flow of invalid veins. For a particular eye, if the percentage of area of valid veins was less than a threshold (currently 50% of total vein area), then the result from that eye was considered unreliable. Although this interpolation method has the potential to underestimate or overestimate flow in certain cases, it is not relevant to the current study. However, because the focus of this study was to assess intergrader reproducibility and not the validity of the interpolation method, all eyes were used for reproducibility statistic calculations and for comparison of intergrader differences.
To determine whether vessel identification and grading reproducibility were affected by the size of the vessels, all graded vessels were divided into two groups (smaller vs larger) based on the vessel cross-sectional area. The level of intergrader agreement in vessel identification (artery versus vein) for larger (≥ 5,000 μm2) and smaller (< 5,000 μm2) vessels was evaluated.
The TRBF, venous area, superior hemispheric blood flow, inferior hemispheric blood flow, venous velocity, Doppler shift, and Doppler angle measurements for the two graders were analyzed. Intraclass correlation coefficients (ICCs) were used to look for the intergrader reproducibility (an ICC approaching 1 is considered good and indicates better reproducibility). Bland–Altman plots were generated to illustrate the limits of agreement. We also used kw (weighted kappa) statistics, a measure of intergrader concordance in categorical scales that adjusts for chance agreement, to measure the correlation between graders. The kw value is interpreted using the following ranges: 0 to 0.2 = slight agreement; 0.21 to 0.40 = mild; 0.41 to 0.60 = moderate; 0.61 to 0.80 = substantial; and 0.81 to 1.0 = almost perfect agreement. SPSS statistical software (IBM Corporation, Somers, NY) and MedCalc software (MedCalc Software, version 11.3.8, Mariakerke, Belgium) were used for these analyses. A P value of .05 was considered statistically significant.
Randomly selected Doppler OCT scan acquisitions from 20 eyes of 18 participants were included in this analysis. The mean age of the study group was 60.7 years (range: 45 to 79 years). Fourteen of the 20 eyes were left eyes, and 15 eyes were from female participants. Ten eyes were clinically diagnosed as glaucomatous; the remaining 10 eyes were classified as normal.
The mean TRBF for all participants was 38.49 μL/min for grader 1 and 39.04 μL/min for grader 2. The mean absolute difference in TRBF for the participants in this series was 3.84 ± 3.70 μL/min (range: 0 to 13.6 μL/min), and there was no statistically significant difference between the two graders for TRBF (P = .872 and ICC of 0.933 [95% CI: 0.831 to 0.974]). Figure 3A shows the Bland–Altman plot for TRBF with limits of agreement. Venous area measurements showed similar levels of agreement with a mean venous area for all participants of 47.27 and 50.18 mm2, for graders 1 and 2, respectively. The mean absolute difference in mean venous area between graders was 3.59 ± 3.70 mm2 (range: 0 to 10.9 mm2). There was no statistically significant difference between the two graders for venous area (P = .288 and ICC of 0.933 [95% CI: 0.83 to 0.973]). Figure 3B shows the Bland–Altman plot for venous area with limits of agreement. There was an almost perfect agreement on identification and classification of vessels between the two graders with a kw of 0.86 (kw for large and small vessels was 0.910 and 0.783, respectively). ICC (95% CI) values for all parameters are summarized in the table.
Figure 3. Bland–Altman plot of agreement for (A) total retinal blood flow and (B) venous area measurements between graders 1 and 2. Central solid line indicates the mean absolute difference; dotted line indicates the 95% confidence interval (upper and lower) limits.
Table: Values of Study Parameters
In this study, we observed that manual correction of automated retinal blood vessel detections using the DOCTORC software could yield reproducible calculation of retinal blood flow in normal and glaucomatous eyes. Although the TRBF and venous area measurements showed excellent reproducibility, some differences in blood vessel parameters were still observed, particularly for small vessels. However, these differences did not translate into a significant difference in TRBF. This is likely because, although only substantial agreement was observed for identification and grading of small vessels, agreement for large vessels was almost perfect. Because large vessels contribute to the bulk of the blood flow, errors in grading of small vessels are clinically less significant.18 In addition, our strategy to use estimated blood flow for vessels with invalid results also likely contributed to the excellent reproducibility between graders.
In addition to vessel size, another important factor in grading reproducibility appears to be Doppler angle. The calculated Doppler angle is based on identification of the axial position of each vessel within each of the two circular B-scans. Previous studies have confirmed the critical importance of the Doppler angle measurement for calculation of total retinal blood flow using this approach.19 We have observed that reproducibility of Doppler angle determination between graders in our study was good (ICC of 0.958). An earlier study20 had also shown that high-quality scans optimize the Doppler signal and facilitate easier identification of the vessel boundaries. Hence, we evaluated intergrader correlation of Doppler signal in our study and found it to be highly reproducible (ICC of 0.977), thus allowing more reproducible vessel identification. Eccentric positioning of the scanning beam within the pupil appears to be a useful method for maximizing the Doppler angle, and hence the Doppler signal, among the vessels.20
Although several techniques in addition to those described in this study are being used to evaluate retinal blood flow, all of these techniques have limitations. For instance, ultrasound color Doppler imaging only has sufficient resolution to measure the larger retrobulbar vessels.21 It can measure blood velocity but cannot measure vessel diameter, so volumetric blood flow cannot be determined. Several types of laser Doppler techniques can be used to measure flow in individual retinal vessels22,23 or capillary beds.24 Although TRBF can be determined by adding measurements from individual vessels, doing so requires acquiring many measurements over a long session.25 Such specialized instruments are expensive and are generally available only in major research centers. Fluorescein and indo-cyanine green angiographies are widely used to visualize retinal and choroidal circulations, but these do not provide quantitative measurements of blood flow and they require the intravenous injection of dyes that have potential side effects.26
A semi-automated grading strategy using the DOCTORC software was able to achieve reasonably reproducible measurements of total retinal blood flow from Doppler SD-OCT data. Refinement of acquisition protocols to maximize the Doppler signal with an optimal Doppler angle and development of fully automated OCT vessel detection algorithms will be critical for widespread application of Doppler OCT techniques. Retinal blood flow measurements, however, may provide new insights into disease mechanisms and response to therapies.
- Huang D, Swanson EA, Lin CP, et al. Optical coherence tomography. Science. 1991;254:1178–1181. doi:10.1126/science.1957169 [CrossRef]
- Hee MR, Izatt JA, Swanson EA, et al. Optical coherence tomography of the human retina. Arch Ophthalmol. 1995;113:325–332. doi:10.1001/archopht.1995.01100030081025 [CrossRef]
- Hee MR, Puliafito CA, Wong C, et al. Optical coherence tomography of macular holes. Ophthalmology. 1995;102:748–756.
- Jonas J, Paques M, Monés J, Glacet-Bernard A. Retinal vein occlusions. Dev Ophthalmol. 2010;47:111–135. doi:10.1159/000320076 [CrossRef]
- Horii T, Murakami T, Nishijjma K, Sakamoto A, Ota M, Yoshimura N. Optical coherence tomographic characteristics of microaneurysms in diabetic retinopathy. Am J Ophthalmol. 2010;150:840–848. doi:10.1016/j.ajo.2010.06.015 [CrossRef]
- Hee MR, Baumal CR, Puliafito CA, et al. Optical coherence tomography of age-related macular degeneration and choroidal neovascularization. Ophthalmology. 1996;103:1260–1270.
- Schuman JS, Pedut-Kloizman T, Hertzmark E, et al. Reproducibility of nerve fiber layer thickness measurements using optical coherence tomography. Ophthalmology. 1996;103:1889–1898.
- Tan O, Chopra V, Lu ATH, et al. Detection of macular ganglion cell loss in glaucoma by Fourier-domain optical coherence tomography. Ophthalmology. 2009;116:2305–2314. doi:10.1016/j.ophtha.2009.05.025 [CrossRef]
- Klaver CC, Wolfs RC, Vingerling JR, Hofman A, de Jong PT. Age-specific prevalence and causes of blindness and visual impairment in an older population: the Rotterdam Study. Arch Ophthalmol. 1998;116:653–658.
- Wang Y, Fawzi AA, Tan O, Gil-Flamer J, Huang D. Retinal blood flow detection in diabetic patients by Doppler Fourier domain optical coherence tomography. Opt Express. 2009;17:4061–4073. doi:10.1364/OE.17.004061 [CrossRef]
- Logan JF, Rankin SJ, Jackson AJ. Retinal blood flow measurements and neuroretinal rim damage in glaucoma. Br J Ophthalmol. 2004;88:1049–1054. doi:10.1136/bjo.2003.034884 [CrossRef]
- Hafez AS, Bizzarro RL, Lesk MR. Evaluation of optic nerve head and peripapillary retinal blood flow in glaucoma patients, ocular hypertensives, and normal subjects. Am J Ophthalmol. 2003;136:1022–1031. doi:10.1016/S0002-9394(03)00632-9 [CrossRef]
- Berisha F, Feke GT, Hirose T, McMeel JW, Pasquale LR. Retinal blood flow and nerve fiber layer measurements in early-stage open-angle glaucoma. Am J Ophthalmol. 2008;146:466–472. doi:10.1016/j.ajo.2008.04.034 [CrossRef]
- Cuypers MH, Kasanardjo JS, Polak BC. Retinal blood flow changes in diabetic retinopathy measured with the Heidelberg scanning laser Doppler flowmeter. Graefes Arch Clin Exp Ophthalmol. 2000;238:935–941. doi:10.1007/s004170000207 [CrossRef]
- Deokule S, Vizzeri G, Boehm A, Bowd C, Weinreb RN. Association of visual field severity and parapapillary retinal blood flow in open-angle glaucoma. J Glaucoma. 2010;19:293–298.
- Wehbe H, Ruggeri M, Jiao S, Gregori G, Puliafito CA, Zhao W. Automatic retinal blood flow calculation using spectral domain optical coherence tomography. Opt Express. 2007;15:15193–15206. doi:10.1364/OE.15.015193 [CrossRef]
- Wang Y, Bower BA, Izatt JA, Tan O, Huang D. In vivo total retinal blood flow measurement by Fourier domain Doppler optical coherence tomography. J Biomed Opt. 2007;12:041215. doi:10.1117/1.2772871 [CrossRef]
- Wang Y, Lu A, Gil-Flamer J, Tan O, Izatt JA, Huang D. Measurement of total blood flow in the normal human retina using Doppler Fourier–domain optical coherence tomography. Br J Ophthalmol. 2009;93:634–637. doi:10.1136/bjo.2008.150276 [CrossRef]
- Wang Y, Bower BA, Izatt JA, Tan O, Huang D. Retinal blood flow measurement by circumpapillary Fourier domain Doppler optical coherence tomography. J Biomed Opt. 2008;13:064003. doi:10.1117/1.2998480 [CrossRef]
- Wang Y, Fawzi AA, Varma R, et al. Pilot study of optical coherence tomography measurement of retinal blood flow in retinal and optic nerve diseases. Invest Ophthalmol Vis Sci. 2011;52:840–845. doi:10.1167/iovs.10-5985 [CrossRef]
- Goebel W, Lieb WE, Ho A, Sergott RC, Farhoumand R, Grehn F. Color Doppler imaging: a new technique to assess orbital blood flow in patients with diabetic retinopathy. Invest Ophthalmol Vis Sci. 1995;36:864–870.
- Riva CE, Grunwald JE, Sinclair SH, Petrig BL. Blood velocity and volumetric flow rate in human retinal vessels. Invest Ophthalmol Vis Sci. 1985;26:1124–1132.
- Feke GT, Tagawa H, Deupree DM, Goger DG, Sebag J, Weiter JJ. Blood flow in the normal human retina. Invest Ophthalmol Vis Sci. 1989;30:58–65.
- Nicolela MT, Hnik P, Schulzer M, Drance SM. Reproducibility of retinal and optic nerve head blood flow measurements with scanning laser Doppler flowmetry. J Glaucoma. 1997;6:157–164. doi:10.1097/00061198-199706000-00004 [CrossRef]
- Garcia JP Jr, Garcia PT, Rosen RB. Retinal blood flow in the normal human eye using the canon laser blood flowmeter. Ophthalmic Res. 2002;34:295–299. doi:10.1159/000065600 [CrossRef]
- Yannuzzi LA, Rohrer KT, Tindel LJ, et al. Fluorescein angiography complication survey. Ophthalmology. 1986;93:611–617.
Values of Study Parameters
||Mean ± SD (Range) Values of Grader 1
||Mean ± SD (Range) Values of Grader 2
||Mean ± SD (Range) Absolute Difference
||ICC (95% CI)
||38.49 ± 10.84 (12.78–57.14)
||39.04 ± 10.65 (15.26–57.14)
||3.84 ± 3.70 (0–13.6)
|Venous area (mm2)
||47.27 ± 8.36 (30.35–61.01)
||50.18 ± 8.74 (31.14–67.84)
||3.59 ± 3.70 (0–10.95)
|Superior flow (μL/min)
||20.17 ± 6.6 (6.67–30.45)
||20.82 ± 6.75 (7.31–30.45)
||3.15 ± 3.65 (0–13.4)
|Inferior flow (μL/min)
||18.32 ± 5.9 (6.10–29.40)
||18.15 ± 6.14 (7.31–28.31)
||1.79 ± 1.74 (0–4.46)
|Superior area (mm2)
||25.16 ± 6.07 (16.9–40.63)
||26.46 ± 5.94 (18.99–44.18)
||2.04 ± 1.92 (0–6.71)
|Inferior area (mm2)
||22.12 ± 5.42 (13.29–35.0)
||23.72 ± 6.4 (11.08–40.01)
||2.60 ± 2.11 (0–6.87)
|Doppler angle (degree)
||5.06 ± 7.0 (−24.34–+28.3)
||5.22 ± 7.0 (−23.01–+31.69)
||1.26 ± 2.52 (0–28.83)
||0.033 ± 2.66 (−7.63–+78.89)
||− 0.008 ± 2.58 (−294.95–+78.08)
||0.16 ± 0.78 (0–20.71)