Ophthalmic Surgery, Lasers and Imaging Retina

The articles prior to January 2012 are part of the back file collection and are not available with a current paid subscription. To access the article, you may purchase it or purchase the complete back file collection here

Imaging: Clinical Science 

Between-Grader Repeatability of Tear Meniscus Measurements Using Fourier-Domain OCT in Patients With Dry Eye

Ethan H. Tittler, MD; Matthew C. Bujak, MD; Pho Nguyen, MD; Xinbo Zhang, PhD; Yan Li, PhD; Samuel C. Yiu, MD, PhD; David Huang, MD, PhD

Abstract

BACKGROUND AND OBJECTIVE:

To examine the between-grader repeatability of height, depth, and cross-sectional area measurements of the lower tear meniscus, using a Fourier-domain optical coherence tomography (OCT) system.

PATIENTS AND METHODS:

A total of 16 patients with dry eye had the lower tear meniscus of the right eye imaged twice in rapid succession. The tear meniscus height, depth, and cross-sectional area were measured by two masked graders using computer calipers. The between-grader variability, calculated using the pooled coefficient of variation (CV%), assessed the repeatability of the measurements.

RESULTS:

The between-grader CV% was 12.1%, 15.7%, and 19.5% for height, depth, and area, respectively. The between-image variability was 17.1%, 13.4%, and 35.4% for height, depth, and area, respectively. The overall intraclass correlation was 99%. There was no systematic bias between the two graders.

CONCLUSION:

Fourier-domain OCT demonstrates good between-grader and between-image repeatability in measuring the height, depth, and cross-sectional area of the tear meniscus in patients with dry eye. Measurement variability was primarily due to the difference between images rather than graders.

Abstract

BACKGROUND AND OBJECTIVE:

To examine the between-grader repeatability of height, depth, and cross-sectional area measurements of the lower tear meniscus, using a Fourier-domain optical coherence tomography (OCT) system.

PATIENTS AND METHODS:

A total of 16 patients with dry eye had the lower tear meniscus of the right eye imaged twice in rapid succession. The tear meniscus height, depth, and cross-sectional area were measured by two masked graders using computer calipers. The between-grader variability, calculated using the pooled coefficient of variation (CV%), assessed the repeatability of the measurements.

RESULTS:

The between-grader CV% was 12.1%, 15.7%, and 19.5% for height, depth, and area, respectively. The between-image variability was 17.1%, 13.4%, and 35.4% for height, depth, and area, respectively. The overall intraclass correlation was 99%. There was no systematic bias between the two graders.

CONCLUSION:

Fourier-domain OCT demonstrates good between-grader and between-image repeatability in measuring the height, depth, and cross-sectional area of the tear meniscus in patients with dry eye. Measurement variability was primarily due to the difference between images rather than graders.

From Keck School of Medicine, University of Southern California (EHT), Los Angeles, California; University of Toronto (MCB), Department of Ophthalmology and Vision Sciences, Toronto, Ontario, Canada; and Doheny Eye Institute (MCB, PN, XZ, YL, SCY, DH), Los Angeles, California.

Supported by R24EY13015, R01EY018184, research grant from Optovue, Inc., grant from Research to Prevent Blindness, Charles C. Manger, III, MD, Chair in Corneal Laser Surgery endowment.

Dr. Huang received stock options, patent royalty, speaker honorarium, and travel support from Optovue, Inc. (Fremont, CA), and receives royalties from the Massachusetts Institute of Technology for an optical coherence tomography patent licensed to Carl Zeiss Meditec, Inc. (Dublin, CA). Drs. Huang and Li received research grant support from Optovue. The remaining authors have no financial or proprietary interest in the materials presented herein.

Address correspondence to David Huang, MD, PhD, Casey Eye Institute–Marquam Hill, 3375 S.W. Terwilliger Blvd., Portland, OR 97239. E-mail: davidhuang@alum.mit.edu

Received: March 07, 2011
Accepted: June 14, 2011

Introduction

The clinical analysis of tear film on the ocular surface is usually achieved by collecting a detailed history, performing a slit-lamp examination with fluorescein dye, and implementing the Schirmer’s1 and tear breakup time tests.2 The lower eyelid tear meniscus has been estimated to contain 75% to 90% of the total tear volume on the ocular surface,3 thus tear meniscus volume may be a reasonable proxy for total tear volume. Advances in anterior segment imaging using optical coherence tomography4 (OCT) have allowed improved visualization and quantitative analysis of the lower eyelid tear meniscus.5–12 Fourier-domain OCT has greater speed and resolution than time-domain OCT systems and may provide more accurate and reproducible measurements.5

Previous investigators examining the tear meniscus of the lower eyelid using Fourier-domain OCT have demonstrated better single-grader repeatability, both within and between visits, than other OCT systems.5 The purpose of this study was to examine the repeatability of measurements performed by multiple graders (between-grader repeatability) and thus to further ascertain the utility of Fourier-domain OCT examination of the lower eyelid tear meniscus.

Patients and Methods

Sixteen patients with dry eye seen at the Doheny Eye Institute Dry Eye Clinic (Los Angeles, CA) were recruited for this study in compliance with the Health Insurance Portability and Accountability Act of 1996, following the tenets of the Declaration of Helsinki and with the approval of the institutional review board of the University of Southern California.

Patients were instructed not to use any eye drops within 2 hours prior to having their eyes scanned. The right lower eyelid margin of each patient, halfway between the medial and lateral canthus, was imaged by one certified technician using a Fourier-domain OCT system (RTVue; Optovue, Inc., Fremont, CA), taking a 6-mm vertical scan at 26,000 scans per second for a resolution of 5 μm, to a depth of 2.8 mm. Two scans were performed in rapid succession, 2 seconds after a blink.

Tear meniscus height, depth, and cross-sectional area were measured by two graders (grader 1 and grader 2) using computer calipers included in the Fourier-domain OCT system software (RTVue version 4.7.0.5). Tear meniscus height was estimated as a straight line connecting the points at which the anterior surface of the tear meniscus contacted the lower eyelid and the cornea (Fig. 1A). Tear meniscus depth was estimated as a straight line connecting the point at which the lower eyelid contacted any part of the globe and the approximate midpoint of the anterior surface of the tear meniscus (Fig. 1A). Tear meniscus cross-sectional area was estimated by creating a quadrilateral with one straight line to approximate the orbital surface, one straight line to approximate the interior surface of the lower eyelid, and two straight lines to approximate the anterior surface of the tear meniscus (Fig. 1B). This method of measuring the tear meniscus is similar to that described in a previous study.5 In the rare instance that the patient demonstrated redundant folds of bulbar or palpebral conjunctiva (conjunctivochalasis, Fig. 2), the cross-sectional area was measured such that the area of the conjunctiva that fell within the quadrilateral was approximately equal to the area of tear liquid outside the quadrilateral (Fig. 2B). The two graders took the tear meniscus measurements blinded to each other’s results.

Fourier-domain optical coherence tomography images of lower eyelid tear meniscus and method for measuring three parameters of the lower eyelid tear meniscus. (A) Height is measured as a straight line connecting the points at which the anterior surface of the meniscus makes contact with the cornea and the lower eyelid. Depth is measured as a straight line connecting the point at which the lower eyelid contacted any part of the globe and the approximate midpoint of the anterior surface of the tear meniscus. (B) Cross-sectional areas are measured by outlining the approximate borders of the tear meniscus using four lines total: two lines to approximate the anterior surface of the meniscus and one line to approximate both the lower eyelid and the orbital surfaces.

Figure 1. Fourier-domain optical coherence tomography images of lower eyelid tear meniscus and method for measuring three parameters of the lower eyelid tear meniscus. (A) Height is measured as a straight line connecting the points at which the anterior surface of the meniscus makes contact with the cornea and the lower eyelid. Depth is measured as a straight line connecting the point at which the lower eyelid contacted any part of the globe and the approximate midpoint of the anterior surface of the tear meniscus. (B) Cross-sectional areas are measured by outlining the approximate borders of the tear meniscus using four lines total: two lines to approximate the anterior surface of the meniscus and one line to approximate both the lower eyelid and the orbital surfaces.

Three identical Fourier-domain optical coherence tomography images of the lower eyelid tear meniscus of a patient with conjunctivochalasis, demonstrating three potential measuring techniques for estimating cross-sectional area and some possible sources of variability. (A) Area within lines is inclusive of nearly the entire meniscus, but also includes a large portion of bulbar conjunctiva. (B) Area within lines excludes the superior and posterior portions of the meniscus, but includes less bulbar conjunctiva. (C) Area within lines contains minimal conjunctiva, but excludes large portions of the meniscus.

Figure 2. Three identical Fourier-domain optical coherence tomography images of the lower eyelid tear meniscus of a patient with conjunctivochalasis, demonstrating three potential measuring techniques for estimating cross-sectional area and some possible sources of variability. (A) Area within lines is inclusive of nearly the entire meniscus, but also includes a large portion of bulbar conjunctiva. (B) Area within lines excludes the superior and posterior portions of the meniscus, but includes less bulbar conjunctiva. (C) Area within lines contains minimal conjunctiva, but excludes large portions of the meniscus.

For each eye, the mean values of the four measurements (by two graders on two images) were calculated for meniscus height, depth, and cross-sectional area, and then the overall mean and standard deviation were calculated based on these individual mean values. The between-grader repeatability and between-image repeatability were analyzed by fitting a mixed model separately for height, depth, and area. In each of the models, the total variance was broken into population variance, between-grader variance, and between-image variance. The repeatability was obtained by taking the square root of the respective variance values. Coefficient of variation (CV%) is another common way to describe measurement accuracy, which in our situation was calculated by dividing the repeatability by the overall mean. We also considered the intraclass correlation (ICC), which is simply the proportion of the class level (in our case patient level) variation among the total variance. It is worthwhile to point out that high patient level variation can inadvertently inflate the ICC number; hence, we must be careful to interpret and compare ICCs from different samples.

All statistical analysis was done using SAS software version 9.2 (SAS Institute, Cary, NC).

Results

Measurements by grader 1 and grader 2 for tear meniscus height (mean = 245 μm, standard deviation [SD] = 149 vs 236 μm, SD = 152 μm), depth (mean = 130 μm, SD = 82 vs 138 μm, SD = 79 μm), and area (mean = 0.018 mm2, SD = 0.020 vs 0.020 mm2, SD = 0.022 mm2) were similar to one another, but grader 1 had a larger average height and grader 2 had a larger average depth and area (Table 1). With both graders’ measurements pooled, average height, depth, and area were 240 μm (SD = 146 μm), 134 μm (SD = 78 μm), and 0.019 mm2 (SD = 0.020 mm2), respectively.

Lower Tear Meniscus Measurements, as Measured by Two Graders Viewing Images Gathered Using Fourier-Domain Optical Coherence Tomography

Table 1: Lower Tear Meniscus Measurements, as Measured by Two Graders Viewing Images Gathered Using Fourier-Domain Optical Coherence Tomography

The between-grader repeatability, measured as the CV%, was 12.1% for height, 15.7% for depth, and 19.5% for area (Table 2). The between-image repeatability, also measured as the CV%, was 17.1% for height, 13.4% for depth, and 35.4% for area. ICC was 99%.

Precision of the Lower Tear Meniscus Measurements, Assessed in Terms of Measurement Repeatabilitya

Table 2: Precision of the Lower Tear Meniscus Measurements, Assessed in Terms of Measurement Repeatability

Discussion

This study sought to examine the between-grader repeatability of tear meniscus measurements using a Fourier-domain OCT system. We demonstrated that this system can produce precise measurements of the lower eyelid tear meniscus, even when the measurements are taken by different graders. Measurements of meniscus height and depth had the greatest precision, demonstrated by their low CV% (12.1% and 15.7%, respectively). Measurements of cross-sectional area were comparatively less precise (CV% = 19.5%). However, the high overall ICC (99%), a statistic that takes into account the wide discrepancy in the tear measurements between patients, demonstrates the repeatability of the measurements by the two graders was high. To the best of our knowledge, this is the first study to consider the repeatability of measurements when multiple graders measure the same meniscus images using Fourier-domain OCT.

Our study looking at patients with dry eye, similar to an earlier study looking at normal patients,5 demonstrated that the between-image variability in the measurements of tear meniscus height and depth were comparatively low (17.1% and 13.4%, respectively), and that cross-sectional area measurements had the greatest variability (35.4%). This pattern of higher variability when measuring area is to be expected. The anterior boundary of the tear meniscus was well defined and easily visible in most images (Fig. 1A), making the height measurement relatively simple. However, occasionally the meniscus was so small that, even with a resolution of 5 microns, the pixilation of the image made it difficult to clearly identify the boundaries of the meniscus. On the other hand, measuring the depth was less well defined because it required the grader to place one end of the caliper on the approximate midpoint of the anterior boundary of the meniscus (Fig. 1A), thus relying on the graders’ best judgment.

The cross-sectional area was measured in a similar manner to an earlier study5 (Fig. 1B). This method of measurement is simple but, as may be expected, has comparatively high variability due to an inherent multiplication of the variability in approximating the lengths of the quadrilateral’s sides when calculating the area. Additionally, the shape of the anterior surface of the meniscus forced the graders to make an approximation of a curve using two straight lines. Where these two lines met, again, relied on the graders’ best judgment. Finally, in rare cases, patients had conjunctivochalasis, which altered the shape of the meniscus cross-section such that the lengths of all sides of the triangles could legitimately be questioned (Fig. 2). It should be noted, however, that despite these challenges, coefficients of variation were similar to those published in an earlier study using Fourier-domain OCT,5 and that the two observers’ measurements demonstrated remarkable agreement (ICC = 99%).

Our average tear meniscus height and cross-sectional area measurements were lower than several other published reports,5–8 but this was expected because the patients in our study had the diagnosis of dry eye. A study looking specifically at patients with dry eye demonstrated tear meniscus heights and cross-sectional areas that were approximately half that of our study,10 whereas another similar study demonstrated similar values to ours.13

Traditional methods for quantifying tear volume on the ocular surface include Schirmer’s test1 and the cotton thread test.14 Both tests have the benefits of low cost and simplicity, but have a major drawback in that they require the examiner to place foreign bodies between the eyelid and globe, causing irritation and reflex tearing. Physicians often prevent this reaction with anesthetic eye drops, adding yet another variable into the interpretation of the results, and the accuracy and precision of the tests has been questioned.15 Tear volume measurement using OCT may circumvent some of the drawbacks of the traditional methods of quantifying tear volume.

Because Fourier-domain OCT systems are now commercially available, the importance of knowing the inter-grader reproducibility of tear meniscus measurements cannot be overstated. Although the inter-grader reproducibility of tear meniscus measurements using Fourier-domain OCT is encouraging, future work in this area may benefit from measurement software that allows the area within any shape (with straight edges and hand-drawn curves) to be calculated. Especially in cases with conjunctivochalasis, this would require examiners to make fewer judgments and further objectify the results. Ultimately, a program that can recognize the borders and area within the tear meniscus may prove to be best, because it would provide the greatest objectivity, and perhaps the greatest accuracy and precision, when quantifying tear volume.

References

  1. Schirmer O. Studien zur physiologie und pathologie der tranen-absonderung und tranenabfuhr. Graefes Arch Clin Exp Ophthalmol. 1903;56:197–291.
  2. Lemp MA, Hamill JR Jr, . Factors affecting tear film breakup in normal eyes. Arch Ophthalmol. 1973;89:103–105.
  3. Holly FJ. Physical chemistry of the normal and disordered tear film. Trans Ophthalmol Soc U K. 1985;104:374–380.
  4. Huang D, Swanson EA, Lin CP, et al. Optical coherence tomography. Science. 1991;254:1178–1181. doi:10.1126/science.1957169 [CrossRef]
  5. Zhou S, Li Y, Lu AT, et al. Reproducibility of tear meniscus measurement by Fourier-domain optical coherence tomography: a pilot study. Ophthalmic Surg Lasers Imaging. 2009;40:442–447. doi:10.3928/15428877-20090901-01 [CrossRef]
  6. Johnson ME, Murphy PJ. The agreement and repeatability of tear meniscus height measurement methods. Optom Vis Sci. 2005;82:1030–1037. doi:10.1097/01.opx.0000192352.78935.e0 [CrossRef]
  7. Wang J, Aquavella J, Palakuru J, Chung S, Feng C. Relationships between central tear film thickness and tear menisci of the upper and lower eyelids. Invest Ophthalmol Vis Sci. 2006;47:4349–4355. doi:10.1167/iovs.05-1654 [CrossRef]
  8. Palakuru JR, Wang J, Aquavella JV. Effect of blinking on tear dynamics. Invest Ophthalmol Vis Sci. 2007;48:3032–3037. doi:10.1167/iovs.06-1507 [CrossRef]
  9. Bitton E, Keech A, Simpson T, Jones L. Variability of the analysis of the tear meniscus height by optical coherence tomography. Optom Vis Sci. 2007;84:903–908. doi:10.1097/OPX.0b013e3181560ba8 [CrossRef]
  10. Shen M, Li J, Wang J, et al. Upper and lower tear menisci in the diagnosis of dry eye. Invest Ophthalmol Vis Sci. 2009;50:2722–2726. doi:10.1167/iovs.08-2704 [CrossRef]
  11. Savini G, Goto E, Carbonelli M, Barboni P, Huang D. Agreement between stratus and visante optical coherence tomography systems in tear meniscus measurements. Cornea. 2009;28:148–151. doi:10.1097/ICO.0b013e31818526d0 [CrossRef]
  12. Wang Y, Zhuang H, Xu J, Wang X, Jiang C, Sun X. Dynamic changes in the lower tear meniscus after instillation of artificial tears. Cornea. 2010;29:404–408.
  13. Ibrahim OM, Dogru M, Takano Y, et al. Application of visante optical coherence tomography tear meniscus height measurement in the diagnosis of dry eye disease. Ophthalmology. 2010;117:1923–1929. doi:10.1016/j.ophtha.2010.01.057 [CrossRef]
  14. Kurihashi K, Yanagihara N, Honda Y. A modified Schirmer test: the fine-thread method for measuring lacrimation. J Pediatr Ophthalmol. 1977;14:390–397.
  15. Lamberts DW, Foster CS, Perry HD. Schirmer test after topical anesthesia and the tear meniscus height in normal eyes. Arch Ophthalmol. 1979;97:1082–1085.

Lower Tear Meniscus Measurements, as Measured by Two Graders Viewing Images Gathered Using Fourier-Domain Optical Coherence Tomography

GraderHeight (μm)Depth (μm)Area (mm2)
AverageSDAverageSDAverageSD
Grader 1245149130820.0180.020
Grader 2236152138790.0200.022
Pooled240146134780.0190.020

Precision of the Lower Tear Meniscus Measurements, Assessed in Terms of Measurement Repeatabilitya

VariableBetween-Grader RepeatabilityBetween-Image RepeatabilityBetween-Grader Variability (CV%)Between-Image Variability (CV%)
Height (μm)294112.117.1
Depth (μm)211815.713.4
Area (mm2)0.00370.006719.535.4
Authors

From Keck School of Medicine, University of Southern California (EHT), Los Angeles, California; University of Toronto (MCB), Department of Ophthalmology and Vision Sciences, Toronto, Ontario, Canada; and Doheny Eye Institute (MCB, PN, XZ, YL, SCY, DH), Los Angeles, California.

Supported by R24EY13015, R01EY018184, research grant from Optovue, Inc., grant from Research to Prevent Blindness, Charles C. Manger, III, MD, Chair in Corneal Laser Surgery endowment.

Dr. Huang received stock options, patent royalty, speaker honorarium, and travel support from Optovue, Inc. (Fremont, CA), and receives royalties from the Massachusetts Institute of Technology for an optical coherence tomography patent licensed to Carl Zeiss Meditec, Inc. (Dublin, CA). Drs. Huang and Li received research grant support from Optovue. The remaining authors have no financial or proprietary interest in the materials presented herein.

Address correspondence to David Huang, MD, PhD, Casey Eye Institute–Marquam Hill, 3375 S.W. Terwilliger Blvd., Portland, OR 97239. E-mail: davidhuang@alum.mit.edu

10.3928/15428877-20110812-05

Sign up to receive

Journal E-contents