From the Department of Ophthalmology and Visual Sciences, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin.
Supported by the National Institutes of Health grant EY06594 (R Klein, BEK Klein).
The authors have no financial or proprietary interest in the materials presented herein.
The authors thank Mr. Scott Brandenburg, Ms. Holly Cohn, Ms. Lisa Grady, Ms. Mary Kay Aprison, and Ms. Heidi Gutt for their assistance.
Address correspondence to Karl Jensen, MBA, University of Wisconsin–Madison Department of Ophthalmology & Visual Sciences, 610 North Walnut Street, 442 WARF, Madison, WI 53726-2336.
The 35-mm color slide has been used to document eye disease in ophthalmic practice and research for many years.1,2 Research centers such as those at the University of Wisconsin have accumulated large libraries of color slides representing longitudinal views of the retinas and lenses of study participants in epidemiological studies and clinical trials, as well as patients receiving clinical care for retinal disease. More recently, digital fundus photography has become a popular option,3 paralleling developments in the mainstream photography world.
In many cases, an image originally acquired on film is used as input to an automated or semi-automated process, necessitating digitization. At the University of Wisconsin, images originally acquired on film, in some cases from longitudinal studies spanning decades, are digitized for use in semi-automated assessments of retinal vessel diameters. Thirty-five millimeter color slides are digitized using general-purpose flatbed scanners with 35-mm adapters or dedicated 35-mm scanners. The latter are generally recommended for journalistic and scientific 35-mm conversion due to their greater spatial resolution and more consistent quality.4 The dedicated 35-mm film scanners have been widely used in projects where retinal or lens images were originally acquired on film and then digitized.5–8 At the University of Wisconsin, evaluation of the retinal vessel diameters is commonly performed on these images. An underlying assumption is that the digital scanning process is stable (ie, variations in light sensitivity of the 35-mm scanner do not affect the scientific results). The question of potential variations in the sensitivity of the 35-mm scanner has not been widely addressed in the vision research literature, although related issues have been discussed in the radiology field.9–11
Anecdotal evidence of instability of digital scanners led us to design an experiment where a color slide was repeatedly scanned and the resulting images compared. We used popular models of 35-mm scanners and assessed the impact of this variation on retinal vascular caliber gradings.
Patients and Methods
A given Early Treatment of Diabetic Retinopathy Field 1 slide, shot on Kodak Ektachrome 100 Plus film (Eastman Kodak Company, Rochester, NY) and centered on the optic disc, was scanned in four runs of 20 instances each on three different scanner models. The slide was of a healthy eye from one of the authors and of good photographic quality.
The scanners used were the Nikon LS-2000 (Nikon Corp., Tokyo, Japan), the Nikon 5000 (Nikon Corp.), and the Polaroid 4000 (Polaroid Corp., Waltham, MA). For the Nikon LS-2000, the Nikon Scan 2.5 software was used and the images were scanned at 2,700 dots per inch (dpi). For the Nikon 5000, the Nikon Scan 4.0 software was used and the images were scanned at 2,540 dpi. For the Polaroid 4000, the Polaroid Polacolor Insight 5.5 software was used and the images scanned at 3,000 dpi. Images were acquired at 8 bits per channel and saved in the Tagged Image File Format format.
The four runs were organized into sessions of two cold runs and two warm runs. Cold runs were defined as sessions where equipment was turned on for more than 30 minutes and less than 4 hours, and warm runs were defined as sessions where equipment was on for more than 4 hours. A Matlab (The Mathworks, Natick, MA) software program averaged the pixel color values across the entire image separately for the red, green, and blue channels. The result was three numbers for each image. The three numbers characterized the image and were used roughly to quantify the differences of the images. For our purposes, we looked for a tool that would identify cases of scanner drift and inconsistency. We had received reports of dramatic increase in scanner sensitivity in one of the channels, particularly the red channel, so we focused on color channel levels. The numerical results showed whether the results of two scans are identical and modeled the level of difference.
The resulting 240 images were processed using custom-built Matlab routines to determine the average pixel brightness of the red, green, and blue channels. The analysis looked at differences between the warm and cold runs, as well as differences within the runs, to determine whether the variations were due to temperature. Vessels within each image were measured three times using the Interactive Vessel ANalysis (IVAN) software12 and three variations of the IVAN grading protocol (The IVAN Grading Protocol, University of Wisconsin Ocular Epidemiology Reading Center internal document, 2006).
The IVAN grading protocol specifies the procedure for choosing the vessels to grade for width, where to measure them, and how to handle issues such as branching of the vessels within the evaluation zone. It delineates methods to adjust mis-readings by the automated process. For this experiment, we chose to focus on four specific retinal vessels: two arterioles and two venules. Three variations of the standard protocol were developed.
In the automated approach (Phase 1), the only grader intervention was to guide the IVAN software to evaluate the pre-selected vessels. Then, using a semi-automated approach (Phase 2), grader intervention included correction, if needed, of the automated placement of the grid over the optic nerve. The third phase used the standard IVAN grading protocol to a greater degree, in that the grader was permitted to truncate and otherwise modify the automated grading. The IVAN software attempts to find the optic nerve head and center the grid on it; this automatic centration occasionally fails and is manually corrected. Under our modified versions of the grading protocol, in Phase 1 we noted but did not correct the mis-centration. In Phase 2 and 3, we noted and corrected these errors.
Statistical analysis was performed with the SAS System version 9.1.3 (SAS Institute, Cary, NC). The mean, standard deviation, and coefficients of variation of the color channel levels were found using procedure MEANS, which performs simple statistics. The analysis of the effect of the warm-up period and the search for correlation between color channels and vessel caliber grading were performed using ANOVA (ANalysis of VAriance) in procedure Generalized Linear Model (GLM). ANOVA is robust to unbalanced data sets. For purposes of analysis, the luminance levels for each channel were stratified into low, medium, and high tertiles of equal size by scanner and these tertiles were regressed against the vessel widths.
The Nikon 5000 scanner had the lowest coefficient of variation for the blue and red channels, averaging each color channel across all runs (Table 1). The Nikon LS-2000 scanner had the lowest coefficient of variation for the green runs (Table 2). The standard deviation for the Nikon scanners ranged from 0.43 to 1.21 in the various channels, with higher standard deviations for the Polaroid scanner, ranging from 6.74 in the green channel to 9.01 in the red channel (Table 3). All channel intensity levels are based on eight-bit pixels and thus potentially can range from 0 to 255. Due to scanning error, the data for run 1 (cold) of the Nikon LS-2000 was discarded.
Table 1: Channel Levels in Nikon 5000 by Run, Comparison Between Runs
Table 2: Channel Levels in Nikon LS-2000 by Run, Comparison Between Runs
Table 3: Channel Levels in Polaroid 4000 by Run, Comparison Between Runs
ANOVA was used to compare the warm and cold runs by color channel. The data were missing for one Nikon LS-2000 cold run. Significant differences were found in the color channel levels, in most cases, between the two cold runs and between the two warm runs, as well as between the warm and cold runs, suggesting that although the overall variability was of interest, it did not relate to temperature.
The vessel caliber measurements were regressed against the categorized luminance values of the three channels among the three scanners, with all runs combined. The figure shows scatter plots of the vessel widths for the four measured vessels against the color channel levels for the Nikon 5000 scanner. Here, for simplicity, we show only the results from Phase 3 of the grading protocol and for the Nikon 5000 scanner. In these figures, the color channel levels are normalized to the average value for that scanner and color. Regression lines are displayed. The measured vessel widths seem to vary by approximately 3 μm in the bulk of the cases and the color channel levels vary by approximately five brightness levels. The data show no significant relation between the color channel level and the vessel width, except for some relation between the small arteriole and the green channel, which rises to the level of significance.
Figure. Nikon 5000 (Nikon Corp., Tokyo, Japan) Color Channel Levels Versus Vessel Width. Results from All Runs Are Plotted. Four Vessels Were Measured and Plotted Here: The Larger Arteriole (A), Smaller Arteriole (B), Larger Venule (C), and Smaller Venule (D). All Runs Are Adjusted for Color Channel Values.
In 3 of 80 scans from the Nikon 5000 scanner, the software incorrectly centered the grid to a large degree, necessitating re-centration of the grid by the grader. Notwithstanding the occasional incorrect centration, we found that the average coefficient of variation of the measured vessel widths was small; the coefficient of variation for the Nikon 5000 scans was 0.3 for the large arteriole and 0.4 for the large venule.
This study grew out of reported inconsistencies with the 35-mm scanners extensively used at the Ocular Epidemiology Reading Center. We decided to examine the reliability of the scanning process, investigating the physical consistency of the resulting images and the impact of any variation on our computer-assisted retinal vessel caliber grading. Our overall test was to scan the same slide repeatedly. Ideally, the result of these repeated scans would be a series of identical image files. Because the scanner and associated personal computers are electronic–mechanical devices, we expected some variation and found some in color channel levels. Interesting patterns were shown and may bear further investigation. However, they did not appear to correlate with the warm-up period and they did not significantly affect the vessel width measurements.
A review of the literature did not find extensive coverage of this topic in vision research publications, although many projects have digitized 35-mm slides in the course of their activity.5–8,13 In 1991, Cideciyan et al. developed a mathematical analysis and model of the overall noise produced by the process of retinal film photography and digital scanning in the course of research into retinitis pigmentosa.14 They described the mechanical operation of the Nikon LS-3500 scanner and developed an image processing technique to restore detail lost in the process. In 2000, Wilson et al. attempted to find the best set of scanning parameters to support human recognition and grading of retinal features such as vessels, cup and disc, and drusen.15
In the field of radiology oncology, several investigators have reported on the use of and issues associated with consumer-level flatbed and slide scanners for digitizing dosimetry film. Lynch et al. analyzed various performance characteristics of the Macbeth densitometer and the Epson and Microtek flatbed scanners used to read radiochromic film.9 They found that after repeated scans the flatbed scanners warmed the film itself, which in turn affected the values they were measuring, and that the scanner did not respond uniformly over the entire area of the film. Hupe and Brunzendorf examined the use of the Nikon 8000 ED slide scanner for radiochromic film dosimetry.10 This scanner used light-emitting diode illumination, which did not heat up the film as much as the lamps used in the flatbed scanners noted above. They also observed the non-uniform characteristics of the charge-coupled diode receptor array but found that the chief contributor to uncertainty in the process was the non-uniformity of the film itself. Paelinck et al. evaluated the Epson Pro 1680 Expression scanner and found a warm-up effect where the reading was slightly higher on the first scan. They did not find other forms of drift over time. Along with the other investigators, they noted the “response over the scanner field is not uniform.”11
Semiconductor theory indicates that warmer temperatures result in more noise in the electronic response of the optical sensor.16 The above researchers did not find the warm-up effects to be significant,9–11 and neither did we. Variations over time are illustrated in the figure. Note that the luminance of the color channels is measured on a 0 to 255 scale,17 so a change of 2.0 would be less than 1% of the entire range.
The Nikon scanners had lower coefficient of variation in the color channel levels, suggesting a better suitability for this type of application.
The core of our analysis was to repeatedly measure the width of certain vessels in the selected eye using variations of our IVAN vessel caliber reading protocol. We made certain modifications to focus precisely on potential vessel width variations. Our hypothesis was that the variations in color channel level, particularly where color levels varied with respect to each other, might influence either the automated or human finding of the vessel walls. As noted above, no clear-cut correlation was found.
We were also interested in any other insights the experiment offered about the vessel caliber grading process using the IVAN software. The IVAN software attempts to automatically detect the optic nerve head and center a grading grid on it. We would not expect any incorrect centration because all images were of the same slide and the original photograph quality was good. Nonetheless, we found a rate of 2.4% incorrectly centered grids. Despite this, we found that the average coefficient of variation of the measured vessel widths was small; for the Nikon 5000 scans, the coefficient of variation was 0.3 for the large artery and 0.4 for the large vein. In comparison, Newsom et al. found a coefficient of variation of 1.5% to 7.5% for automated vessel diameter measurement and 6% to 34% for an observer-driven measurement method.17
The film digitization process did not produce systematic bias that would influence the measurement of vessel caliber or the consequent calculation of summary variables based on individual vessel width measurement. The variations in the vessel caliber were low and lent credence to the reproducibility of the IVAN-based vessel width grading process.
- American Academy of Ophthalmology Retina Panel. Diabetic Retinopathy Preferred Practice Pattern®. San Francisco: American Academy of Ophthalmology; 2003.
- Klein R, Klein BE, Neider MW, Hubbard LD, Meuer SM, Brothers RJ. Diabetic retinopathy as detected using ophthalmoscopy, a nonmydriatic camera and a standard fundus camera. Ophthalmology. 1985;92:485–491.
- Gilchrist J. Computer processing of ocular photographs: a review. Ophthalmic Physiol Opt. 1987;7:379–386.
- Fulton W. A Few Scanning Tips. Plano, TX: Author; 1998.
- Duncan DD, Shukla OB, West SK, Schein OD. New objective classification system for nuclear opacification. J Opt Soc Am A Opt Image Sci Vis. 1997;14:1197–1204. doi:10.1364/JOSAA.14.001197 [CrossRef]
- Hubbard LD, Brothers RJ, King WN, et al. Methods for evaluation of retinal microvascular abnormalities associated with hypertension/sclerosis in the Atherosclerosis Risk in Communities Study. Ophthalmology. 1999;106:2269–2280. doi:10.1016/S0161-6420(99)90525-0 [CrossRef]
- Klein R, Klein BE, Marino EK, et al. Early age-related maculopathy in the cardiovascular health study. Ophthalmology. 2003;110:25–33. doi:10.1016/S0161-6420(02)01565-8 [CrossRef]
- Chapman N, Witt N, Gao X, et al. Computer algorithms for the automated measurement of retinal arteriolar diameters. Br J Ophthalmol. 2001;85:74–79. doi:10.1136/bjo.85.1.74 [CrossRef]
- Lynch BD, Kozelka J, Ranade MK, Li JG, Simon WE, Dempsey JF. Important considerations for radiochromic film dosimetry with flatbed CCD scanners and EBT GAFCHROMIC film. Med Phys. 2006;33:4551–4556. doi:10.1118/1.2370505 [CrossRef]
- Hupe O, Brunzendorf J. A novel method of radiochromic film dosimetry using a color scanner. Med Phys. 2006;33:4085–4094. doi:10.1118/1.2357019 [CrossRef]
- Paelinck L, De Neve W, De Wagter C. Precautions and strategies in using a commercial flatbed scanner for radiochromic film dosimetry. Phys Med Biol. 2007;52:231–242. doi:10.1088/0031-9155/52/1/015 [CrossRef]
- Mosher A, Klein BE, Klein R, Knudtson MD, Ferrier NJ. Comparison of retinal vessel measurements in digital vs film images. Am J Ophthalmol. 2006;142:875–878. doi:10.1016/j.ajo.2006.05.056 [CrossRef]
- Ferrier NJ. Automated identification of the anatomical features in slit lamp photographs of the lens. Invest Ophthalmol Vis Sci. 2002;43:E-Abstract 435.
- Cideciyan AV, Nagel JH, Jacobson SG. Modeling of high resolution digital retinal imaging engineering in Medicine and Biology Society. Proceedings of the Annual International Engineering in Medicine and Biology Society. 1991;13:264–266.
- Wilson M, Nemeth SC, Edwards A, Soliz P. Optimizing retinal image digitization for improved digital processing and visualization. In: Proceedings of the 14th IEEE Symposium on Computer-Based Medical Systems (CMBS’01), July 26-27, 2001, Bethesda, MD. Los Alamitos, CA: IEEE Computer Society; 2001:0091.
- Russ JC. The Image Processing Handbook, 2nd ed. Boca Raton, FL: CRC Press; 1995:28.
- Newsom RS, Sullivan PM, Rassam SM, Jagoe R, Kohner EM. Retinal vessel measurement: comparison between observer and computer driven methods. Graefes Arch Clin Exp Ophthalmol. 1992;230:221–225. doi:10.1007/BF00176292 [CrossRef]
Channel Levels in Nikon 5000 by Run, Comparison Between Runs
|Channel||Run 1 (Cold)||Run 2 (Cold)||PValue Cold 1 vs Cold 2||Run 3 (Warm)||Run 4 (Warm)||PValue Warm 3 vs Warm 4||PValue Warm vs Cold||Average Across All Runs|
| SD||0.19||0.12||< .001||0.61||0.79||.03||.22||0.56|
| SD||0.34||0.31||< .001||0.67||0.47||< .001||< .001||1.21|
| SD||0.03||0.01||< .001||0.52||0.52||< .001||< .001||0.43|
Channel Levels in Nikon LS-2000 by Run, Comparison Between Runs
|Channel||Run 1 (Cold)||Run 2 (Cold)||PValue Cold 1 Vs Cold 2||Run 3 (Warm)||Run 4 (Warm)||PValue Warm 3 Vs Warm 4||PValue Warm Vs Cold||Average Across All Runs|
| SD||*||0.38||NA||0.11||0.17||< .001||< .001||1.20|
| SD||*||0.11||NA||0.09||0.11||.43||< .001||0.62|
| SD||*||0.12||NA||0.08||0.10||< .001||< .001||1.01|
Channel Levels in Polaroid 4000 by Run, Comparison Between Runs
|Channel||Run 1 (Cold)||Run 2 (Cold)||PValue Cold 1 Vs Cold 2||Run 3 (Warm)||Run 4 (Warm)||PValue Warm 3 Vs Warm 4||PValue Warm Vs Cold||Average Across All Runs|
| SD||0.15||0.18||< .001||0.13||0.21||< .001||< .001||9.01|
| SD||0.23||0.42||< .001||0.25||0.43||< .001||< .001||6.74|
| SD||0.23||0.23||< .001||0.23||0.21||< .001||< .001||8.12|