Journal of Pediatric Ophthalmology and Strabismus

Original Article Supplemental Data

How Many Plusoptix S04 Measures Yield the Most Sensitive Amblyopia Screening?

Sandra Guimarães, MD, PhD; Andreia Soares, MD; Patrício Costa, PhD; Eduardo Silva, MD, PhD

Abstract

Purpose:

To analyze how the Plusoptix S04 (Plusoptix, Atlanta, GA), a widely used technology for amblyopia screening, predicts amblyopia.

Methods:

A total of 1,547 children aged 3 to 4 years underwent a non-invasive eye examination in a whole population screening. The Plusoptix was measured binocularly three consecutive times. Seven models were developed: the first (P1), second (P2), and third (P3) measure of Plusoptix, the mean of P1, P2, and P3 (Pmean123), the maximal value of P1, P2, and P3 (Pmax123), the mean of P1 and P2 (Pmean12), and the maximal value of P1 and P2 (Pmax12).

Results:

Internal consistency was excellent for sphere, cylinder (alpha = 0.92 vs 0.97), aniso-sphere, and aniso-cylinder (alpha = 0.87 vs 0.86). All models predicted amblyopia (P < .001); there was 2.3% new diagnosis and 5.1 events per variable. A bootstrap analysis with 1,000 samples confirmed the stability of the model. Astigmatism (P < .001; odds ratio [OR] = 2.1 to 3.6) and aniso-sphere (P < .001; OR = 3.6 to 9.0) predicted amblyopia. Although sphere, when measurable, did not predict amblyopia (P > .19; OR = 1.2 to 1.3), the likelihood of amblyopia presence was 100 times higher when the Plusoptix displayed “hyper.” All receiver operating characteristic (ROC) curves were significant (P < .001), with no differences between them (all P > .16). Adjusting the cut-off from ROC curves for the highest sensitivity, Pmax12 referred 31.9% and 16.1% fewer patients than P1 and Pmean12, respectively. A third measure revealed accommodation-related false-negative results; therefore, adding measurements showed no benefit.

Conclusions:

This was the first study to assess the Plusoptix's prediction of amblyopia. When aiming for maximal sensitivity, the Plusoptix refers many children (> 20% children in all models). The first measure refers more children than two measures. Non-measurable results must be considered abnormal because the chance of amblyopia presence was high.

[J Pediatr Ophthalmol Strabismus. 2019;56(5):305–312.]

Abstract

Purpose:

To analyze how the Plusoptix S04 (Plusoptix, Atlanta, GA), a widely used technology for amblyopia screening, predicts amblyopia.

Methods:

A total of 1,547 children aged 3 to 4 years underwent a non-invasive eye examination in a whole population screening. The Plusoptix was measured binocularly three consecutive times. Seven models were developed: the first (P1), second (P2), and third (P3) measure of Plusoptix, the mean of P1, P2, and P3 (Pmean123), the maximal value of P1, P2, and P3 (Pmax123), the mean of P1 and P2 (Pmean12), and the maximal value of P1 and P2 (Pmax12).

Results:

Internal consistency was excellent for sphere, cylinder (alpha = 0.92 vs 0.97), aniso-sphere, and aniso-cylinder (alpha = 0.87 vs 0.86). All models predicted amblyopia (P < .001); there was 2.3% new diagnosis and 5.1 events per variable. A bootstrap analysis with 1,000 samples confirmed the stability of the model. Astigmatism (P < .001; odds ratio [OR] = 2.1 to 3.6) and aniso-sphere (P < .001; OR = 3.6 to 9.0) predicted amblyopia. Although sphere, when measurable, did not predict amblyopia (P > .19; OR = 1.2 to 1.3), the likelihood of amblyopia presence was 100 times higher when the Plusoptix displayed “hyper.” All receiver operating characteristic (ROC) curves were significant (P < .001), with no differences between them (all P > .16). Adjusting the cut-off from ROC curves for the highest sensitivity, Pmax12 referred 31.9% and 16.1% fewer patients than P1 and Pmean12, respectively. A third measure revealed accommodation-related false-negative results; therefore, adding measurements showed no benefit.

Conclusions:

This was the first study to assess the Plusoptix's prediction of amblyopia. When aiming for maximal sensitivity, the Plusoptix refers many children (> 20% children in all models). The first measure refers more children than two measures. Non-measurable results must be considered abnormal because the chance of amblyopia presence was high.

[J Pediatr Ophthalmol Strabismus. 2019;56(5):305–312.]

Introduction

The Plusoptix S04 (Plusoptix, Atlanta, GA) is one of the most common amblyopia screening devices used worldwide.1 It is a photoscreening technology used by many pediatricians, primary care physicians, and screeners.

The definition of what is considered a disease or a screening goal changes over time, which reflects both the evolution in diagnostic tools and the change in health expectations from social and economic perspectives.2 Amblyopia screening strategies are currently under debate.1,3,4 The prevalence of amblyopia risk factors is much greater than that of amblyopia.5 For that reason, the temptation to use amblyopia risk factors to assess screening efficacy, either clinically or economically, is hard to resist. Most studies report their results regarding the effectiveness of amblyopia risk factor detection.6–14 However, because amblyopia risk factors are not equal to amblyopia, the American Association for Pediatric Ophthalmology and Strabismus (AAPOS) Vision Screening Committee recently recommended that studies should report their results respecting amblyopia diagnosis, rather than its risk factors.5 Nevertheless, in the last systematic review of the 40 fair-quality selected studies (n = 45,588 children) that assessed the diagnostic accuracy of different screening tests,1 only 4 studies used amblyopia as the target condition, and no studies used off-axis, red-reflex crescent photoscreening technology.1,15–18

Although the Plusoptix's performance for amblyopia risk factors has been largely demonstrated,5,11,19,20 its sensitivity and specificity in predicting amblyopia has not been studied. Moreover, having more than 10 years of experience with the Plusoptix, we learned that three consecutive measurements do not always report the same result (positive or negative) for a certain criterion. So, the questions that arise are: Is the child positive because the first measure was positive, although the two following measures were negative? Should we use the mean of various measures? Or, because the main problem seems to be accommodation bias, should we use the maximal (worst) value of various measures? Furthermore, all studies available, even those that assess amblyopia risk factors, do not state in their methodology what measure they use: the first, the second, a mean, or the worst. There is a need for standardization.

This was the first study to evaluate Plusoptix performance in amblyopia prediction. It also examined the best readings for that prediction, while answering the following questions: is it enough to have one Plusoptix measure or should we have more? And, if so, which is the best measure to use?

Patients and Methods

Data were collected prospectively (May 2014 to April 2016) in a whole-screening population with children aged 3 to 4 years who participated in a non-invasive eye examination at Hospital de Braga in Portugal. The study was approved by the Human Research Ethics Committee of Hospital de Braga, it adhered to the tenets of the Declaration of Helsinki, and written informed consent was obtained from the parents/guardians before examinations. Of the 1,547 children enrolled (96% participation rate), after the exclusion criteria were applied, 1,342 children were included in this analysis.

All measures and examinations were performed without correction by pediatric ophthalmologists/orthoptists trained in the protocol. The AAPOS 2013 guidelines for amblyopia risk factors were used for screening5: astigmatism or anisometropia of more than 1.50 diopters (D) for children aged 4 years and more than 2.00 D for children aged 3 years, and hyperopia of more than 3.50 D for children aged 3 years and more than 4.00 D for children aged 4 years. The Plusoptix was measured binocularly three consecutive times. The Plusoptix is one of the few infrared, off-axis red-reflex internally interpreted photoscreeners currently available. When the Plusoptix did not display a numeric value for each eye, but rather a description such as “hyperopia,” “myopia,” or no value displayed in one or both eyes (because of strabismus, visual axis obstruction, or any other problem that could lead to a non-measurement by the device), it was considered as non-measurable. If the child was distracted, the device automatically stopped measuring after 20 seconds and a new measurement was tried and considered the first one.

Visual acuity was measured monocularly with single surrounded tumbling E. Visual acuity was recorded as the smallest optotype size that the child identified in all four directions to prevent a misdiagnosis of meridional amblyopia. After a brief training with binocular visual acuity, testing was conducted monocularly, always starting with the right eye. The eye was occluded with a non-adhesive paper occluder, explaining to the child that it was a “pirate game.” The child could use either his or her fingers or a plastic E to indicate the direction of the letter.21

Stereoacuity was measured with the Randot circles stereo-test. The cover test, Hirschberg test, ocular movements, biomicroscopy, and funduscopy were also assessed. In this first screening assessment, all children with: AAPOS criteria for amblyopia risk factors (presence of at least one AAPOS criterion for screening in any of the three Plusoptix measures was considered positive in the current study), strabismus, visual axis obstruction, or any other abnormality in the complete eye examination, and uncorrected visual acuity of 0.6 (decimal scale) or worse were reassessed on a second day when the cycloplegic refraction and best corrected visual acuity were assessed.

Amblyopia diagnosis was established based on Multi-Ethnic Pediatric Eye Disease Study (MEPEDS)22 criteria for both best corrected visual acuity and amblyopia risk factors (assessed by autorefraction under cycloplegia) as follows: bilateral amblyopia criteria = best corrected visual acuity of worse than 0.4 (decimal) for 3-year-old children and worse than 0.5 (decimal) for 4-year-old children; amblyopia risk factors = bilateral ametropia (SE ≥ +4.00 D; SE ≤ −6.00 D; and ≥ 2.50 D in astigmatism) or bilateral evidence of past or current visual axis obstruction.

Unilateral amblyopia criteria were: best corrected visual acuity difference of two or more logarithm of the minimum angle of resolution lines, with 0.6 (decimal) or less in the worst eye plus presence of amblyopia risk factors: anisometropia (SE ≥ +1.00 D; SE ≤ −3.00 D; ≥ 1.50 D of astigmatism); strabismus; or evidence of past or current visual axis obstruction.

Abnormal eye movements in different gaze positions (eg, alphabetic syndromes, Duane or Brown syndromes, cranial nerve palsies, or any other problem in the different positions of gaze compensated by torticollis) were not considered amblyopia risk factors. When both unilateral and bilateral criteria were present, amblyopia was considered unilateral.

Inclusion and Exclusion Criteria

We excluded 205 children. Exclusion criteria were history of glasses or amblyopia treatment (n = 33); abnormal uncorrected visual acuity in first evaluation but neither best corrected visual acuity nor cycloplegia were assessed (binocular, n = 5; unilateral, n = 3); fundus or biomicroscopy abnormalities (n = 9); incomplete data due to no collaboration in visual acuity (n = 42), no cooperation on complete eye examination with a visual acuity of 0.7 (decimal) or better (n = 44), no collaboration on complete eye examination with visual acuity of 0.6 (decimal) or worse but better than MEPEDS (n = 55), or data recording missing (n = 14). All other children were included.

Study Design and Statistics

This was a cross-sectional study. Data were entered in a Microsoft Office Excel (Microsoft Corporation, Redmond, WA) spreadsheet and exported to SPSS software (version 24; IBM Corporation, Armonk, NY) for statistical analysis. A logistic regression model was created to predict amblyopia based on seven different Plusoptix models: three consecutive measures (P1, P2, or P3), the mean of three consecutive measures (Pmean123), the maximal (worst) of three consecutive measures (Pmax123), the mean of the first two measures (Pmean12), and the maximal (worst) of the first two measures (Pmax12). When the Plusoptix could not measure, because either the values exceeded the device range or the device “could not read,” two categorical variables were created: non-measurable hyperopia and non-measurable other. The regression model was performed with two stepwise blocks: block 1 with the two categorical non-measurable variables and block 2 with sphere (sph), cylinder (cyl), difference of sphere (dif_shp), and difference of cylinder (dif_cyl) between the right and left eye.

Sensitivity, specificity, positive predicted value, and negative predicted value were calculated for each of the seven models, ROC curves were compared, and false-negative results were calculated. ROC curves were then used to select the cut-off values needed to not miss one true diagnosis (100% sensitivity). Sensitivity, specificity, positive predicted value, and negative predicted value were recalculated. The percentage of children who required observation was calculated by dividing the number of children predicted to have amblyopia in each model by the total number of children.

Cronbach's alpha was considered acceptable if alpha was 0.7 or greater.239

Results

Of the 1,342 children enrolled, there were slightly more children from public than private schools (54.4% vs 45.6%), more boys than girls (51.2% vs 48.8%), and more 3 year olds than 4 years olds (50.7% vs 49.3%). Cronbach's alpha was used to estimate the reliability of the three consecutive Plusoptix measures, and it showed acceptable consistency values for sphere (alpha = 0.92), cylinder (alpha = 0.97), and SE (alpha = 0.90) and differences between eyes for sphere (alpha = 0.87), cylinder (alpha = 0.86), and SE (alpha = 0.86). There were no differences between examiners regarding sphere (P = .07; eta-squared = 0.004), cylinder (P = .84; eta-squared = 0.000), dif_sphere (P = .47; chi-square = 0.001), dif_cylinder (P = .38; eta-squared = 0.002), or non-measurable values (chi-square = 0.151, df = 2, P = .93).

Logistic regression by blocks (Table A, available in the online version of this article) revealed that all models were statistically significant for amblyopia prediction, with hit rates between 97.8% and 98.2% for all models: P1 (chi-square = 145.073, df = 6, P < .001, Nagelkerke R2 = 0.52); P2 (chi-square = 133.440, df = 6, P < .001, Nagelkerke R2 = 0.48); P3 (chi-square = 117.380, df = 6, P < .001, Nagelkerke R2 = 0.43); Pmean123 (chi-square = 139.272, df = 6, P < .001, Nagelkerke R2 = 0.50); Pmax123 (chi-square = 141.143, df = 6, P < .001, Nagelkerke R2 = 0.51); Pmean12 (chi-square = 140.783, df = 6, P < .001, Nagelkerke R2 = 0.51); and Pmax12 (chi-square = 146.180, df = 6, P < .001; Nagelkerke R2 = .52).

Logistic Regression by Blocks Using Plusoptix S04 Amblyopia PredictionaLogistic Regression by Blocks Using Plusoptix S04 Amblyopia Predictiona

Table A:

Logistic Regression by Blocks Using Plusoptix S04 Amblyopia Prediction

Astigmatism (P < .001; OR = 2.1 to 3.6) and difference in sphere between eyes (P < .001; OR = 3.6 to 9.0) were predictors of amblyopia in all models. Although sphere was not a predictor (P > .19; OR = 1.2 to 1.3), we found a high OR (P < .001; OR = 96.7 to 100.3) when the Plusoptix displayed “hyper,” due to the device's range measurement. Differences in cylinders between both eyes were not predictive (P > .223) for the same reason. The regression coefficients for the original regression and bootstrap models are shown in Table A.

The ROC curves for each model were statistically significant to predict amblyopia (P < .001), with no statistically significant differences between them (P > .16) (Table 1, Figure 1).

ROC Curve of the 7 Models Using the Plusoptix S04 for Amblyopia Prediction

Table 1:

ROC Curve of the 7 Models Using the Plusoptix S04 for Amblyopia Prediction

Receiver operating characteristic (ROC) curve plots comparing the seven models for amblyopia (not amblyopia risk factors) prediction. Reference line for 0.5 cut-off. P1 = first Plusoptix (Plusoptix, Atlanta, GA) measure; P2 = second Plusoptix measure; P3 = third Plusoptix measure; Pmean123 = mean of P1, P2, and P3; Pmax123 = maximal value of P1, P2, and P3; Pmean12 = mean of P1 and P2; Pmax12 = maximal value of P1 and P2

Figure 1.

Receiver operating characteristic (ROC) curve plots comparing the seven models for amblyopia (not amblyopia risk factors) prediction. Reference line for 0.5 cut-off. P1 = first Plusoptix (Plusoptix, Atlanta, GA) measure; P2 = second Plusoptix measure; P3 = third Plusoptix measure; Pmean123 = mean of P1, P2, and P3; Pmax123 = maximal value of P1, P2, and P3; Pmean12 = mean of P1 and P2; Pmax12 = maximal value of P1 and P2

Sensitivity, specificity, positive predictive values, and negative predictive values are presented in Table 2. For the standard cut-off of 0.5, the number of children whose amblyopia diagnosis would be missed would be between 64.5% and 74.2%, depending on the model used. The lower and upper confidence intervals from all models varied from 45% to 88.1% for a 95% significance. To overcome false-negative results, we looked for 100% sensitivity in each model and adjusted the cut-offs based on each ROC curve. If we aimed to not lose one true diagnosis based on one measure only, the percentage of children who required an ophthalmologic evaluation to confirm a diagnosis of amblyopia was 30.2%, 20.9%, and 92% when considering P1, P2, or P3, respectively (Table 3).

Sensitivity, Specificity, PPV, and NPV for the 7 Models

Table 2:

Sensitivity, Specificity, PPV, and NPV for the 7 Models

Comparison Between the 7 Models After Cut-off is Adapted Based on the ROC Curve for 100% Sensitivitya

Table 3:

Comparison Between the 7 Models After Cut-off is Adapted Based on the ROC Curve for 100% Sensitivity

Discussion

As recommended in 2013 by the AAPOS Vision Screening Committee,5 instruments that detect amblyopia should report results using amblyopia presence as the gold standard. The current study, based on the Plusoptix's performance, has two innovative aspects: it uses amblyopia prediction (instead of amblyopia risk factors) and it evaluates internal consistency and states which measure is best for a higher sensitivity.

When using the Plusoptix to predict amblyopia, we can use any of the seven models because their area under the curve is high (0.93 to 0.97), and no statistically significant difference was found between them (all P > .16). Because we are predicting amblyopia diagnosis and not amblyopia risk factors, it seems reasonable that our aim should be not to fail one true diagnosis, (ie, to have no false-negative results). So, when we adjusted the ROC curve cutoff for 100% sensitivity, we found that, regarding single measurements, P1 and P2 performed better than P3. Moreover, the maximal (worst) value (ie, the most abnormal) between P1 and P2, performed better than P1 alone, diminishing the number of children who need an ophthalmologic observation from 30.2% to 20.6%, meaning 144 fewer children to reevaluate per 1,500 screened, for which we state that the first measure alone is insufficient.

Because P2 performed only slightly worse than Pmax12 (0.3 percentage points more referrals), if we need to use one measure only, P2 may be used instead of Pmax12. Adding a third measure increases this referral percentage drastically as shown in Table 3. When we analyzed the reason why, we found a low cut-off in the P3 ROC curve that led to a low specificity of 8% in contrast to P1 and P2 (71% vs 81%). That was mainly due to a child whose hyperopia was revealed in the first two measures, but on the third measure, he accommodated and skipped the amblyogenic risk factors criteria. Nevertheless, if we consider that child an outlier, the specificity of P3 is still lower than P1 and P2 (4.1 vs 8.4 percentage points) and referral rates are higher, meaning P3 is not a good measure to predict amblyopia. We speculate that repeating the test continuously may give the child the false idea that he or she “has not done it well” in previous measures and he or she may need to concentrate, which would induce the child to accommodate to focus and see better as a way of trying to “perform better” in the examination, giving rise to false negative results. The first two measures may have the “surprise defocus effect” when measuring refraction in a child less concentrated on focusing.

Plusoptix measures were not influenced by the examiner (all P ≥ .07, with 0.0% to 0.4% being explained by examiner), but it is relevant to note that all examiners were experienced with the device.

When aiming for higher sensitivity, the mean of various measures does not yield better results than the maximal (worst) value found between those measures. For example, because P3 alone had worse results, we learned by excluding it that the maximal (worst) reading of the other two measures (Pmax12) refers 16.1% fewer children than the mean between the first two measures (Pmean12). Additionally, even if we change the best corrected visual acuity threshold for amblyopia diagnosis and consider amblyopia as an amblyopia risk factor plus best corrected visual acuity of 0.5 (decimal) or worse, based on our normative,21 Pmax12 also referred 22.5% fewer children than Pmean12. Moreover, we reviewed our data and tested the same regressions regarding the best (closest to normal) results (P12min and P123min) and compared them with the maximal (worst) (most abnormal) results (P12max and P123max). We found we had to refer 21.8% versus 20.6% (for P12min vs P12max) and 64.6% versus 20.3% (for P123min vs P123max). This means that considering the best measure refers more children than the worst.

Therefore, we argue for the use of the worst values of the first two measures, in either clinical practice or new research Plusoptix guidelines. Furthermore, non-measurable results have high odd ratios prediction for amblyopia, and we state that these results should always be included in future Plusoptix studies, otherwise valuable information may be missed.

We performed additional analyses using a lower sensitivity criterion (96.8%) and found Pmax12 to be better than the first measure alone (84% specificity for Pmax12 vs 80.8% for P1) and Pmean12 to be similar to Pmax12 (84%).

Having a “silver standard” rather than a gold standard validation is important for cost-effectiveness.24,25 Although our “eye screening examination” included an almost complete pediatric ophthalmology eye examination (eg, photoscreening, uncorrected monocular visual acuity, stereoacuity, cover test, versions and ductions, slit lamp, and funduscopy), it was not intended to be the final screening model. Instead, it was a way to develop a silver standard, cost-effective method, which we are currently assessing in a different study. Future reports from our research group will use this silver standard for amblyopia screening.

A limitation of the current study is that a large number of models can potentially increase the risk of Type I errors, which makes interpretating the results difficult and potentially impacts the statistical decision (ie, reject or not reject the null hypothesis; statistical significance). Several adjustments can be done (such as, Holm-Bonferroni, Holm-Sidak, and Benjamini-Hochberg) to address any possible “false-positive result.” In the current study, our approach was parallel with P values to consider the practical significance (effect size), which according to Cohen is found to be “medium” for specificity, sensitivity, and hit rate pseudo R2 (> 0.4 for all models).

Although larger samples and more events are always preferred, there are situations (eg, low prevalence diseases such as amblyopia) where confounding cannot be addressed without violating the classic rule of thumb of 10 or more events per variable.26 We had 2.3% children with amblyopia (n = 31) (ie, 5.1 events per variable for 6 variables). In this case, some authors state that, although results should be interpreted with caution, systematic devaluing of those results, in particular statistically significant associations, from any model with five or more events per variable or more does not appear to be justified.26,27

Regarding calibration in the bootstrap analysis, the small bias and standard error values, the fact that all B values are inside the confidence intervals, and the fact that statistical significance for all variables is maintained confirms the stability of the model.

Although we aimed to have a sound definition of amblyopia (using best corrected visual acuity with cyclo-autorefraction and MEPEDS criteria), we do not know if retesting best corrected visual acuity after recovery from cycloplegia would have helped children pass the vision criteria. Furthermore, of the 205 children excluded, we speculate that two children with amblyopia may not have been included (they did not cooperate in visual acuity assessment but both children had high ametropia and one had Down syndrome).

External validation was not addressed in this study and should be addressed in future studies.

We do not know if more than three measures would change the results related to accommodation bias, but it seems unreasonable to have an unlimited number of measures in clinical practice. It should also be noted that we used the Plusoptix model S04. Because there are newer upgraded devices, we sent an e-mail to the Plusoptix manufacturer prior to our study. They confirmed that: “all models (S04, S09, and S12) take the same readings and work with an identical measurement algorithm. The main difference is speed and device layout. S09 works two times faster than the S04 and S12 is 1.5 times faster than S09. S04 and S09 are stationary and S12 is a portable (ie, battery powered) device” (Plusoptix, personal communication, March 12, 2016). The differences in time acquisition are related to the device and not to the time the examiner takes to find the exact point where the Plusoptix measures (approximately 1 meter, but you must make movements back and forward to be in the correct position for Plusoptix to automatically start measuring). This time may influence the child's focus/defocus and, thus, accommodation bias. We also do not know if professionals who are not ophthalmologists would change the accommodation bias because the time to get the device in the right place is different when you are familiar with it.

Furthermore, some children with positive ametropia (either hyperopia or astigmatic) can accommodate and see well (have no amblyopia), whereas other children accommodate almost never or only “part-time.” The fixation targets of the Plusoptix S04 are designed to maximally allow relaxed accommodation: the targets used for the child are small twinkling lights around a central black circle. For amblyopia risk factor screening, relaxing accommodation is important to increase sensitivity (not to skip positive ametropia masked by accommodation). However, for amblyopia screening, relaxing accommodation in children with ametropia without amblyopia (who accommodate and see well) may lead to false-positive results. We do not know if newer Plusoptix versions (that use a “smiling doll picture” for the child to fixate on and could have some accommodative stimulus) will reduce false-positive results of amblyopia. Studies about this issue are needed.

The Plusoptix is an excellent instrument for amblyopia screening, based on its ease of use, high internal consistency, and good results for predicting amblyogenic risk factors.11,14,19,28–31 Nevertheless, when aiming for maximal sensitivity for amblyopia diagnosis, the number of children who need to be referred with the Plusoptix S04 is still high. Further studies using the same amblyopia criteria are needed to compare other devices and/or strategies.

Conclusions

Two measures of the Plusoptix are better than the first measure alone for amblyopia screening in 3- and 4-year-old children. Adding a third measure has no advantage and, if used alone, a third measure may skip children with amblyopia because of accommodation bias. Considering the highest sensitivity, the maximal value over two measures of Plusoptix (Pmax12), and the maximal value over the three measures (Pmax123) are the best models, referring fewer children for an ophthalmologic consultation. Therefore, screeners using the Plusoptix should be advised to use the maximal (worst) value (ie, the most abnormal) of consecutive measures and be discouraged to only use one measure.

References

  1. Jonas DE, Amick HR, Wallace IF, et al. Vision screening in children aged 6 months to 5 years: evidence report and systematic review for the US Preventive Services Task Force. JAMA. 2017;318:845–858. doi:10.1001/jama.2017.9900 [CrossRef]28873167
  2. Scully JL. What is a disease?EMBO Rep. 2004;5:650–653. doi:10.1038/sj.embor.7400195 [CrossRef]15229637
  3. Solebo AL, Cumberland PM, Rahi JS. Whole-population vision screening in children aged 4–5 years to detect amblyopia. Lancet. 2015;385:2308–2319. doi:10.1016/S0140-6736(14)60522-5 [CrossRef]
  4. Donahue SP. The 2017 US Preventive Services Task Force report on preschool vision screening. JAMA Ophthalmol. 2017;135:1021–1022. doi:10.1001/jamaophthalmol.2017.3373 [CrossRef]28873119
  5. Donahue SP, Arthur B, Neely DE, et al. POS Vision Screening Committee. Guidelines for automated preschool vision screening: a 10-year, evidence-based update. J AAPOS. 2013;17:4–8. doi:10.1016/j.jaapos.2012.09.012 [CrossRef]23360915
  6. Kinori M, Molina I, Hernandez EO, et al. The PlusoptiX photoscreener and the Retinomax autorefractor as community-based screening devices for preschool children. Curr Eye Res. 2018;43:654–658. doi:10.1080/02713683.2018.1437453 [CrossRef]29424565
  7. Huang D, Chen X, Zhang X, et al. Pediatric vision screening using the plusoptiX A12C photoscreener in Chinese preschool children aged 3 to 4 years. Sci Rep. 2017;7:2041. doi:10.1038/s41598-017-02246-6 [CrossRef]28515427
  8. Rosenfield M, Ciuffreda KJ. Evaluation of the SVOne Handheld autorefractor in a pediatric population. Optom Vis Sci. 2017;94:159–165. doi:10.1097/OPX.0000000000000999 [CrossRef]
  9. Asare AO, Malvankar-Mehta MS, Makar I. Community vision screening in preschoolers: initial experience using the Plusoptix S12C automated photoscreening camera. Can J Ophthalmol. 2017;52:480–485. doi:10.1016/j.jcjo.2017.02.002 [CrossRef]28985808
  10. Ruão M, Almeida I, Leitão R, et al. Photoscreening for amblyogenic risk factors in 1-year-olds: results from a single center in Portugal over a 9-year period. J AAPOS. 2016;20:435–438. doi:10.1016/j.jaapos.2016.06.003 [CrossRef]27647116
  11. Fogel-Levin M, Doron R, Wygnanski-Jaffe T, Ancri O, Ben Zion I. A comparison of plusoptiX A12 measurements with cycloplegic refraction. J AAPOS. 2016;20:310–314. doi:10.1016/j.jaapos.2016.04.006 [CrossRef]27422572
  12. Lowry EA, de Alba Campomanes AG. Efficient referral thresholds in autorefraction-based preschool screening. Am J Ophthalmol. 2015;159:1180–1187.e3. doi:10.1016/j.ajo.2015.02.012 [CrossRef]25728859
  13. Lowry EA, Wang W, Nyong'o O. Objective vision screening in 3-year-old children at a multispecialty practice. J AAPOS. 2015;19:16–20. doi:10.1016/j.jaapos.2014.09.008 [CrossRef]25727580
  14. Arnold RW, Armitage MD. Performance of four new photoscreeners on pediatric patients with high risk amblyopia. J Pediatr Ophthalmol Strabismus. 2014;51:46–52. doi:10.3928/01913913-20131223-02 [CrossRef]
  15. Afsari S, Rose KA, Pai AS, et al. Diagnostic reliability and normative values of stereoacuity tests in preschool-aged children. Br J Ophthalmol. 2013;97:308–313. doi:10.1136/bjophthalmol-2012-302192 [CrossRef]23292927
  16. Barry JC, König HH. Test characteristics of orthoptic screening examination in 3 year old kindergarten children. Br J Ophthalmol. 2003;87:909–916. doi:10.1136/bjo.87.7.909 [CrossRef]12812897
  17. Jost RM, Stager D Jr, Dao L, Katz S, McDonald R, Birch EE. High specificity of the Pediatric Vision Scanner in a private pediatric primary care setting. J AAPOS. 2015;19:521–525. doi:10.1016/j.jaapos.2015.09.004 [CrossRef]26691030
  18. Ciner E, Carter A, Ying GS, Maguire M, Kulp MTVision in Preschoolers Study Group. Comparison of the Retinomax and Palm-AR auto-refractors: a pilot study. Optom Vis Sci. 2011;88:830–836.21516050
  19. Crescioni M, Miller JM, Harvey EM. Accuracy of the Spot and Plusoptix photoscreeners for detection of astigmatism. J AAPOS. 2015;19:435–440. doi:10.1016/j.jaapos.2015.07.284 [CrossRef]26486025
  20. Singman E, Matta N, Tian J, Silbert D. A comparison of referral criteria used by the plusoptiX photoscreener. Strabismus. 2013;21:190–194. doi:10.3109/09273972.2013.811606 [CrossRef]23978147
  21. Guimaraes S, Fernandes T, Costa P, Silva E. Should tumbling E go out of date in amblyopia screening? Evidence from a population-based sample normative in children aged 3–4 years. Br J Ophthalmol. 2018;102:761–766. doi:10.1136/bjophthalmol-2017-310691 [CrossRef]
  22. Varma R, Deneen J, Cotter S, et al. Multi-Ethnic Pediatric Eye Disease Study Group. The multi-ethnic pediatric eye disease study: design and methods. Ophthalmic Epidemiol. 2006;13:253–262. doi:10.1080/09286580600719055 [CrossRef]16877284
  23. Hair JF, Anderson RE, Tatham RL, Black W. Multivariate Data Analysis, 5th ed. Upper Saddle River, NJ: Prentice Hall; 1998.
  24. Arnold RW, Stark L, Leman R, Arnold KK, Armitage MD. Tent photoscreening and patched HOTV visual acuity by school nurses: validation of the ASD-ABCD protocol (Anchorage School District-Alaska Blind Child Discovery program). Binocul Vis Strabismus Q. 2008;23:83–94.
  25. Leman R, Clausen MM, Bates J, Stark L, Arnold KK, Arnold RW. A comparison of patched HOTV visual acuity and photoscreening. J Sch Nurs. 2006;22:237–243. doi:10.1177/10598405050220040901 [CrossRef]16856779
  26. Vittinghoff E, McCulloch CE. Relaxing the rule of ten events per variable in logistic and Cox regression. Am J Epidemiol. 2007;165:710–718. doi:10.1093/aje/kwk052 [CrossRef]
  27. Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996;49:1373–1379. doi:10.1016/S0895-4356(96)00236-3 [CrossRef]8970487
  28. Yilmaz I, Ozkaya A, Alkin Z, Ozbengi S, Yazici AT, Demirok A. Comparison of the Plusoptix A09 and Retinomax K-Plus 3 with retinoscopy in children. J Pediatr Ophthalmol Strabismus. 2015;52:37–42. doi:10.3928/01913913-20141230-06 [CrossRef]25643369
  29. Singman E, Matta N, Tian J, Brubaker A, Silbert D. A comparison of the PlusoptiX S04 and A09 photoscreeners. Strabismus. 2013;21:85–87. doi:10.3109/09273972.2013.786735 [CrossRef]23713927
  30. Arnold RW, Arnold AW, Armitage MD, Shen JM, Hepler TE, Woodard TL. Pediatric photoscreeners in high risk patients 2012: a comparison study of Plusoptix, Iscreen and SPOT. Binocul Vis Strabolog Q Simms Romano. 2013;28:20–28.23521032
  31. Nathan NR, Donahue SP. Modification of Plusoptix referral criteria to enhance sensitivity and specificity during pediatric vision screening. J AAPOS. 2011;15:551–555. doi:10.1016/j.jaapos.2011.08.008 [CrossRef]22153399

ROC Curve of the 7 Models Using the Plusoptix S04 for Amblyopia Prediction

MeasurementAUCSEP95% CI
P1.963.012< .001.952 to .973
P2964.010< .001.952 to .973
P3.930.030< .001.915 to .943
Pmean123.968.009< .001.957 to .976
Pmax123.968.009< .001.957 to .977
Pmean12.967.009< .001.956 to .976
Pmax12.970.009< .001.959 to .978

Sensitivity, Specificity, PPV, and NPV for the 7 Models

CharacteristicP1P2P3Pmean123Pmax123Pmean12Pmax12
Sensitivity35.5%32.3%25.8%32.3%32.3%35.5%32.3%
Specificity99.7%99.5%99.5%99.5%99.5%99.6%99.6%
PPV73.3%58.8%57.1%62.5%62.5%68.8%66.7%
NPV98.5%98.4%98.3%98.4%98.4%98.5%98.4%
Missed diagnosisa64.5% (CI: 45.4% to 80.8%)67.7% (CI: 48.6% to 83.3%)74.2% (CI: 55.4% to 88.1%)67.7% (CI: 48.6% to 83.3%)67.7% (CI: 48.6% to 83.3%)64.5% (CI: 45.4% to 80.8%)67.7% (CI: 48.6% to 83.3%)

Comparison Between the 7 Models After Cut-off is Adapted Based on the ROC Curve for 100% Sensitivitya

CharacteristicP1P2P3Pmean123Pmax123Pmean12Pmax12
Sensitivity100%100%100%100%100%100%100%
Specificity71.5%81.0%8.2%80.8%81.5%77.2%81.3%
PPV7.7%11.1%2.5%11.0%11.3%9.4%11.2%
NPV100%100%100%100%100%100%100%
Missed diagnosisb0000000
No. of children needed to be observed (n = 1,342)30.2% (n = 405)20.9% (n = 280)92% (n = 1,234)21.1% (n = 283)20.3% (n = 273)24.6% (n = 329)20.6% (n = 276)

Logistic Regression by Blocks Using Plusoptix S04 Amblyopia Predictiona

MeasurementBlock 1Block 2


NM_hyperNM_otherSphereCylinderAniso-sphAniso-cyl
P1
  B4.5713.185.242−1.0562.195.570
  SE1.240.888.187.226.384.468
   P< .001< .001.194< .001< .001.223
  OR96.66724.1671.274.3488.9771.768
Bootstrap for Variables in the Equation, Based on 945 Samples

  Bias4.761−2.013.056−.027−.007.022
  SE12.7127.245.291.298.492.589
   P.001.001.310.001.001.292
  Lower 95% CI−17.430−17.465−.166−1.6641.204−.524
  Upper 95% CI25.4385.070.950−.4103.2121.820
P2
  B4.6073.404.204−1.3071.888−.145
  SE1.241.757.174.237.348.360
   P< .001< .001.241< .001< .001.687
  OR100.23130.0691.226.2716.609.865
Bootstrap for Variables in the Equation, Based on 957 Samples

  Bias5.639−.805.054.030−.029.122
  SE12.6864.890.253.296.466.518
   P.001.001.398.001.001.721
  Lower 95% CI−17.399−17.253−.172−1.829.914−1.029
  Upper 95% CI25.3905.332.803−.6432.8411.090
P3
  B4.6073.221.197−1.0951.274.484
  SE1.241.734.194.224.379.444
   P< .001< .001.308< .001.001.276
  OR100.23125.0581.218.3343.5771.623
Bootstrap for Variables in the Equation, Based on 948 Samples

  Bias4.773−1.076.109.030−.112.086
  SE12.7144.730.354.315.559.570
   P.001.001.545.001.007.335
  Lower 95% CI−17.399−17.246−.238−1.672.023−.569
  Upper 95% CI25.4914.6261.089−.3242.1021.734
Pmean123
  B4.5713.185.226−1.1722.029.368
  SE1.240.888.181.232.396.473
   P< .001< .001.214< .001< .001.437
  OR96.66724.1671.253.3107.6081.445
Bootstrap for Variables in the Equation, Based on 948 Samples

  Bias4.761−2.013.086−.001−.023.032
  SE12.7127.245.304.310.527.592
   P.001.001.390.001.001.475
  Lower 95% CI−17.430−17.465−.165−1.786.918−.797
  Upper 95% CI25.4385.0701.019−.5153.0131.608
Pmax123
  B4.5723.186.280−1.2411.771.299
  SE1.240.888.181.241.352.433
   P< .001< .001.122< .001< .001.490
  OR96.74124.1851.323.2895.8771.348
Bootstrap for Variables in the Equation, Based on 945 Samples

  Bias4.122−1.952.087.022−.049.056
  SE12.9737.603.297.327.425.510
   P.001.001.255.001.001.553
  Lower 95% CI−17.429−17.469−.118−1.856.835−.731
  Upper 95% CI25.4365.4371.066−.5562.5811.293
Pmean12
  B4.6073.404.075−1.5731.936−.334
  SE1.241.757.166.179.378.350
   P< .001< .001.650< .001< .001.341
  OR100.23120.2321.078.2076.931.716
Bootstrap for Variables in the Equation, Based on 956 Samples

  Bias5.639−.805.114.146.077.228
  SE12.6864.890.275.330.479.661
   P.001.001.746.001.001.551
  Lower 95% CI−17.399−17.253−.261−1.8931.046−1.351
  Upper 95% CI25.3905.332.814−.6292.9831.219
Pmax12
  B4.6083.404.235−1.2311.949.392
  SE1.241.757.179.235.359.434
   P< .001< .001.191< .001< .001.367
  OR100.30830.0921.264.2927.0241.480
Bootstrap for Variables in the Equation, Based on 957 Samples

  Bias4.766−1.021.058−.030.012−.011
  SE12.7074.941.282.292.463.515
   P.001.001.349.001.001.400
  Lower 95% CI−17.399−17.286−.165−1.824.963−.712
  Upper 95% CI25.4394.994.916.6352.8981.365
Authors

From the Life and Health Sciences Research Institute (ICVS), School of Medicine, University of Minho, Braga; Portugal (SG, PC); the ICVS/3B's–PT Government Associate Laboratory, Braga/Guimarães, Portugal (SG, PC); the Department of Ophthalmology, Hospital de Braga, Braga, Portugal (SG, AS); the Clinical Academic Center Braga, Braga, Portugal (PC); the Centro Cirurgico de Coimbra, Coimbra, Portugal (ES); and the Department of Ophthalmology, Centro Hospitalar Lisboa Norte, Lisbon, Portugal (ES).

The authors have no financial or proprietary interest in the materials presented herein.

Correspondence: Sandra Guimarães, MD, PhD, Serviço de Oftalmologia, Hospital de Braga, Sete Fontes–São Victor, 4710-243 Braga, Portugal. E-mail: sandraguimaraes.eye@gmail.com

Received: May 01, 2019
Accepted: July 01, 2019

10.3928/01913913-20190702-01

Sign up to receive

Journal E-contents