Journal of Pediatric Ophthalmology and Strabismus

Original Article 

Evaluation of a Remote Telemedicine Screening System for Severe Retinopathy of Prematurity

Brett A. Begley, BA; Joseph Martin, CO, COMT; Geoffrey T. Tufty, MD; Donny W. Suh, MD

Abstract

Purpose:

To evaluate the validity of remote telemedicine screening for retinopathy of prematurity (ROP) in a population of at-risk preterm infants in Iowa and South Dakota.

Methods:

The medical records for all preterm infants screened for ROP at neonatal intensive care units (NICUs) in Sioux City, Iowa, and Sioux Falls, South Dakota, from September 1, 2017, to July 31, 2018, were retrospectively reviewed. The RetCam Shuttle (Natus Medical Inc., Pleasanton, CA) was used to capture retinal images, which were posted on a secure server for evaluation by a pediatric ophthalmologist. Infants with suspected ROP approaching the criteria for treatment with anti-vascular endothelial growth factor (VEGF) medications were transferred to the Children's Hospital and Medical Center NICU in Omaha, Nebraska, where a comprehensive examination was performed and treatment was administered when indicated. The remaining infants received an outpatient comprehensive examination by one of two pediatric ophthalmologists within 2 weeks of discharge.

Results:

A total of 124 telemedicine examinations were performed on 35 infants during the study period. Remote telemedicine screening for referral-warranted ROP using the RetCam Shuttle had a sensitivity of 100%, specificity of 97%, positive predictive value of 66.7%, and negative predictive value of 100%. Of the three infants transferred for referral-warranted ROP, two required treatment with anti-VEGF medications. Good outcomes were noted in all cases, and no patients progressed beyond stage 3 ROP.

Conclusions:

Telemedicine screening reliably detected referral-warranted ROP in at-risk premature infants at two remote sites, with no poor outcomes during the 11-month period. These results demonstrate the validity and utility of remote telemedicine screening for ROP.

[J Pediatr Ophthalmol Strabismus. 2019;56(3):157–161.]

Abstract

Purpose:

To evaluate the validity of remote telemedicine screening for retinopathy of prematurity (ROP) in a population of at-risk preterm infants in Iowa and South Dakota.

Methods:

The medical records for all preterm infants screened for ROP at neonatal intensive care units (NICUs) in Sioux City, Iowa, and Sioux Falls, South Dakota, from September 1, 2017, to July 31, 2018, were retrospectively reviewed. The RetCam Shuttle (Natus Medical Inc., Pleasanton, CA) was used to capture retinal images, which were posted on a secure server for evaluation by a pediatric ophthalmologist. Infants with suspected ROP approaching the criteria for treatment with anti-vascular endothelial growth factor (VEGF) medications were transferred to the Children's Hospital and Medical Center NICU in Omaha, Nebraska, where a comprehensive examination was performed and treatment was administered when indicated. The remaining infants received an outpatient comprehensive examination by one of two pediatric ophthalmologists within 2 weeks of discharge.

Results:

A total of 124 telemedicine examinations were performed on 35 infants during the study period. Remote telemedicine screening for referral-warranted ROP using the RetCam Shuttle had a sensitivity of 100%, specificity of 97%, positive predictive value of 66.7%, and negative predictive value of 100%. Of the three infants transferred for referral-warranted ROP, two required treatment with anti-VEGF medications. Good outcomes were noted in all cases, and no patients progressed beyond stage 3 ROP.

Conclusions:

Telemedicine screening reliably detected referral-warranted ROP in at-risk premature infants at two remote sites, with no poor outcomes during the 11-month period. These results demonstrate the validity and utility of remote telemedicine screening for ROP.

[J Pediatr Ophthalmol Strabismus. 2019;56(3):157–161.]

Introduction

Retinopathy of prematurity (ROP) is a leading cause of preventable blindness in children worldwide. Approximately 15 million infants are born preterm every year, and the number is rising according to the World Health Organization.1 Both younger gestational ages and lower birth weights are correlated with an increase in the incidence and severity of ROP. In 2005, the Early Treatment for Retinopathy of Prematurity trial found that ROP developed in approximately 68% of preterm infants weighing less than 1,251 g, of which more than one-third were severe cases of the disease.2 Because the survival rate of preterm infants is increasingly prevalent in both developed and developing countries, the demand for ROP care is expanding.3,4 Traditionally, the gold standard approach for ROP screening involves a clinical examination performed in person by an ophthalmologist with binocular indirect ophthalmoscopy. However, with fewer adequately trained ophthalmologists to care for these infants, disease management strategies are limited, especially in areas distant from major medical centers, pediatric ophthalmologists, or retina specialists.5,6

Telemedicine is an emerging method to lessen the burden of ROP screening through the remote interpretation of digital retinal images by an expert.7–20 The RetCam Shuttle (Natus Medical Inc., Pleasanton, CA) has been implemented recently in telemedicine screening for ROP. The purpose of this retrospective study was to evaluate the validity of remote telemedicine screening for ROP in an at-risk preterm infant population in Iowa and South Dakota.

Patients and Methods

The medical records for all preterm infants screened for ROP at UnityPoint Health-St. Luke's NICU in Sioux City, Iowa, and Avera Health in Sioux Falls, South Dakota, from September 1, 2017, to July 31, 2018, were retrospectively reviewed. This study was approved by the institutional review board at the University of Nebraska Medical Center, which granted a waiver of consent for retrospective analysis of screening data. Additionally, the authors obtained data transfer agreements and institutional reliance agreements from UnityPoint Health and Avera Health.

UnityPoint Health-St. Luke's NICU is a level II facility, Avera McKennan's NICU is a level IIIB facility, and both institutions accept and care for all preterm infants in their surrounding communities, including extremely premature (< 28 weeks' gestational age at birth) and extremely low birth weight (< 1,000 g) infants. At both institutions, the timing of the initial ROP screening examination was determined based on guidelines published jointly by the American Academy of Pediatrics (AAP) Section on Ophthalmology, American Academy of Ophthalmology (AAO), American Association for Pediatric Ophthalmology and Strabismus (AAPOS), and American Academy of Certified Orthoptists (AACO).21 Sex, gestational age at birth, birth weight, and multiple birth data (single born, monozygotic twins, dizygotic twins, or triplets) were obtained from the delivery records for each patient from both hospitals.

At UnityPoint Health-St. Luke's NICU, a team including two registered nurses/clinical staff educators was responsible for positioning the infants, monitoring vital signs, and capturing images. Similarly, Avera McKennan's NICU employed a team including one orthoptist, one neonatal nurse practitioner, and one registered nurse who worked together to screen preterm infants for ROP. At each site, the infants' pupils were dilated by instilling cyclopentolate 0.2% and phenylephrine 1% (Cyclomydril; Alcon Laboratories, Inc., Fort Worth, TX) 30 to 60 minutes before screening, with feeding discontinued 2 hours before and after examination in accordance with aspiration precaution guidelines. Proparacaine 0.5% was instilled in each eye as a topical anesthetic immediately before examination, and hydroxypropyl methylcellulose 2.5% was used to couple the digital camera lens to the cornea to provide adequate exposure for photography. In each eye, the capture of five clearly focused retinal fields was attempted using the RetCam Shuttle with a 130° lens until adequate quality was obtained: (1) optic nerve centered, (2) optic nerve superior, (3) optic nerve inferior, (4) optic nerve temporal, and (5) optic nerve nasal. Throughout each examination, vital signs, oxygen saturation, and cardiopulmonary status were closely monitored for signs of possible bradycardia or apnea. Signs of either condition prompted halting of the examination.

The digital retinal images were posted on a secure, password-controlled server for evaluation by a pediatric ophthalmologist at Children's Hospital and Medical Center in Omaha, Nebraska. The pediatric ophthalmologist prepared, signed, and returned written reports for each patient within 24 hours. In the current study, referral-warranted ROP was defined as any Early Treatment Retinopathy of Prematurity22 ROP disease type 2 or greater, any zone II stage 2 eyes with pre-plus disease, and any plus disease.

Infants who appeared to have ROP approaching the criteria for treatment with anti-vascular endothelial growth factor (VEGF) medications were transferred to the Children's Hospital and Medical Center NICU, where a comprehensive examination was performed and treatment administered when indicated. All other infants received an outpatient comprehensive examination by one of two pediatric ophthalmologists, at either Children's Hospital and Medical Center in Omaha, or Sanford Eye Center and Optical in Sioux Falls, within 2 weeks of discharge.

Results

A total of 124 telemedicine examinations were performed on 35 infants during the study period for an average of 3.5 examinations per infant. For all infants screened, the mean gestational age at birth was 29.9 weeks (range: 25.7 to 32.7 weeks), and the mean birth weight was 1,335 g (range: 605 to 2,745 g). Of the 35 infants, 3 (8.6%) infants were transferred for referral-warranted ROP. The mean post-menstrual age at transfer was 34.8 weeks (range: 31.3 to 38.3 weeks).

In this study, remote telemedicine screening for referral-warranted ROP using the RetCam Shuttle had a sensitivity of 100%, specificity of 97%, positive predictive value of 66.7%, and negative predictive value of 100% (Table 1). Good outcomes were observed in all 4 eyes treated with anti-VEGF medications; active retinopathy resolved and no progression beyond stage 3 ROP was noted. For the remaining 32 infants who were screened but not transferred, outpatient clinical binocular indirect ophthalmoscopy was performed within 2 weeks of discharge from the NICU. All of these infants were observed until the documentation of complete retinal maturation.

RetCam Shuttle vs Clinical Examination Findings for Detecting Referral-Warranted ROP

Table 1:

RetCam Shuttle vs Clinical Examination Findings for Detecting Referral-Warranted ROP

Discussion

The 11-month results for the ROP telemedicine screening system in this study were highly favorable. The low positive predictive value (66.7%) reflects the expanded diagnostic criteria used to define referral-warranted ROP, resulting in a tendency for overcalling cases of potential disease on imaging to prioritize patient safety. Consequently, in this study, one “false positive” infant was identified and transferred during the study period. The quality of the photographs was not a major influence in the physicians' decision to transfer the patient in this case, and repeat images were obtained in all cases in which image quality was questionable. It is likely that the positive predictive value would increase with more experience for the technicians and pediatric ophthalmologists in obtaining and interpreting the images. The negative predictive value of 100% suggests a low probability of missing clinically significant ROP, and the high true negative count (32 infants) was a result of ensuring that all 35 infants underwent a follow-up clinical examination by one of two pediatric ophthalmologists. Most important, although it is possible for the disease to have developed and spontaneously regressed, no cases of referral-warranted ROP went undetected, reinforcing the safety and validity of telemedicine screening for clinically significant ROP.

Remote telemedicine screening for ROP was officially recognized for the first time by the AAP, AAO, AAPOS, and AACO in 2013.21 As a result, telemedicine has become more widely accepted in recent years as a cost-effective, timely approach to ROP detection. In 2008, Jackson et al.23 reported that the costs per quality-adjusted life year gained were $3,193 for telemedicine compared to $5,617 for standard ophthalmoscopy. In 2016, Quinn et al.24 found that the average turnaround time from image submission to grading results was 10.1 ± 11.3 hours, with 95.5% of those returned within 24 hours. In our study, the turnaround period was 6 hours or less, on average. Together, these studies suggest that telemedicine is more cost-effective than binocular indirect ophthalmoscopy in the long term and can provide timely feedback to NICUs regarding cases of ROP that may require treatment. Additionally, telemedicine allows for a record of previous examinations to be saved for comparison purposes and for consulting new cases with other physicians. In 2018, Brown et al.25 concluded that a fully automated algorithm was able to diagnose plus disease in ROP with comparable or better accuracy than human experts. Thus, if implemented without compromising the quality of patient care, telemedicine screening and automated diagnosis of plus disease in ROP may have potential applications to alleviate shortages in the workforce, financial costs, and time constraints.6,23–25

Although several publications support telemedicine as a reliable method for detecting ROP,7–16 only a few studies to date have evaluated the real-world situation of screening without the safety net of a simultaneous examination with binocular indirect ophthalmoscopy. In 2000, Schwartz et al.17 reported ROP with plus disease was accurately identified in 95% of eyes, and pre-threshold, threshold, and stage 4 or 5 ROP was correctly detected in 89% of eyes. In 2009, Lorenz et al.18 noted 100% sensitivity in detecting suspected treatment-requiring ROP and an 82.4% positive predictive value in identifying treatment-requiring ROP. In a 2012 study, Weaver and Murdock19 reported 100% sensitivity and 96.3% specificity for detecting type 1 ROP.19 In 2015, the Stanford University Network for Diagnosis of Retinopathy of Prematurity initiative noted 100% sensitivity and 95% specificity in a 6-year retrospective analysis.20 Although few in number, these studies indicate that telemedicine is a safe, reliable, and accurate method of screening for ROP when indirect ophthalmoscopy is not available.

Telemedicine screening for ROP has the potential to improve access to care for all infants no matter where they are born. In our 11-month evaluation, there were no cases of missed type 1 ROP and no adverse anatomical outcomes. Importantly, remote interpretations did not replace in-person examinations in our study; rather, they served to complement screening by allowing one pediatric ophthalmologist to prioritize the highest-risk infants at multiple medical centers and administer treatment as needed. Furthermore, with regard to accessibility, all patients with ROP seen at the two participating NICU facilities in Iowa and South Dakota could be remotely evaluated via telemedicine screening at one central location in Nebraska. Therefore, the expansion of this high-volume telemedicine screening for ROP may be warranted, especially in rural areas where the number of specialists may be limited.

Telemedicine programs are also being established in other medical specialties. In psychiatry, Williams et al.26 found that college students who were offered web-based depression screening and psychiatric consultation rated the experience positively and useful in their understanding of depression. In 2016, a cardiology study by de Araújo et al.27 noted the establishment of a telemedicine screening program in northeast Brazil showed a significant increase in the incidence of detecting nearly all forms of congenital heart disease. In pathology, Huang et al.28 reported that telepathology consultation of frozen sections in rural China had a mean turnaround time within 30 minutes, a definitive diagnosis reached in more than 90% of cases, and extremely low false-positive and false-negative rates.28 Overall, these studies highlight the growing body of evidence supporting telemedicine as a promising solution to address challenges in health care reform in other medical fields outside of ophthalmology.

A limitation of this study was its relatively small sample size. Based on the decision to screen weekly, it is likely that infants in the current study underwent more examinations than necessary. This choice was made to prioritize patient safety by limiting the possibility of failing to detect a cases of ROP requiring treatment. Of note, based on vital signs, oxygen saturation, and cardiopulmonary status, the infants seemed to tolerate telemedicine examinations better than indirect ophthalmoscopy, where the use of depressors can deteriorate these objective findings more quickly. The average time needed to capture useable images for both eyes was approximately 5 to 10 minutes per infant, and the telemedicine screening examinations also offered greater flexibility in terms of scheduling for both infants and staff members.

However, the ability to save and store digital retinal images for later use has the potential to improve clinical efficacies. In particular, telemedicine systems offer the opportunity for many ROP specialists to collaborate in monitoring patients during disease manifestations, treatments, and outcomes and to establish objective guidelines for patient management. Additional challenges of telemedicine screening for ROP include the upfront costs and suboptimal quality of images of the peripheral retina (zone III) associated with the Ret-Cam. Nevertheless, as the affordability, resolution, and speed of retinal camera technology continue to improve, the prospects of telemedicine screening will become increasingly feasible for real-world ROP screenings. Moreover, the learning curve for non-physician technicians to perform the telemedicine examinations was relatively minimal. After 3 hours of hands-on training by the pediatric ophthalmologists, a total of eight non-physician technicians were trained without difficulty.

This study demonstrates the validity of remote telemedicine screening for ROP. Telemedicine screening in Iowa and South Dakota detected referral-warranted ROP with no adverse outcomes. Future studies are warranted to evaluate whether remote telemedicine screening for ROP has an impact on reducing ergonomic injury, improving health care accessibility, and lowering financial costs compared to indirect ophthalmoscopic examination.

References

  1. March of Dimes, The Partnership for Maternal, Newborn, and Child Health, Save the Children, World Health Organization. In: Howson CP, Kinney MV, Lawn JE, eds. Born Too Soon: The Global Action Report on Preterm Birth. Geneva, Switzerland: World Health Organization; 2012.
  2. Good WV, Hardy RJ, Dobson V, et al. The incidence and course of retinopathy of prematurity: findings from the early treatment for retinopathy or prematurity study. Pediatrics. 2005;116:15–23. doi:10.1542/peds.2004-1413 [CrossRef]
  3. Gilbert C. Retinopathy of prematurity: a global perspective of the epidemics, population of infants at risk and implications for control. Early Hum Dev. 2008;84:77–82. doi:10.1016/j.earlhumdev.2007.11.009 [CrossRef]
  4. Gilbert C, Fielder A, Gordillo L, et al. Characteristics of infants with severe retinopathy of prematurity in countries with low, moderate, and high levels of development: implications for screening programs. Pediatrics. 2005;115:e518–e525. doi:10.1542/peds.2004-1180 [CrossRef]
  5. Altersitz K, Piechock IM. Survey: physicians being driven away from ROP treatment. Ocular Surgery News. August15, 2006.
  6. Kemper AR, Wallace DK. Neonatologists' practices and experiences in arranging retinopathy of prematurity screening services. Pediatrics. 2007;120:527–531. doi:10.1542/peds.2007-0378 [CrossRef]
  7. Ells AL, Holmes JM, Astle WF, et al. Telemedicine approach to screening for severe retinopathy of prematurity: a pilot study. Ophthalmology. 2003;110:2113–2117. doi:10.1016/S0161-6420(03)00831-5 [CrossRef]
  8. Chiang MF, Keenan JD, Starren J, et al. Accuracy and reliability of remote retinopathy of prematurity diagnosis. Arch Ophthalmol. 2006;124:322–327. doi:10.1001/archopht.124.3.322 [CrossRef]
  9. Chiang MF, Wang L, Busuioc M, et al. Telemedical retinopathy of prematurity diagnosis: accuracy, reliability, and image quality. Arch Ophthalmol. 2007;125:1531–1538. doi:10.1001/archopht.125.11.1531 [CrossRef]
  10. Wu C, Petersen RA, VanderVeen DK. RetCam imaging for retinopathy of prematurity screening. J AAPOS. 2006;10:107–111. doi:10.1016/j.jaapos.2005.11.019 [CrossRef]
  11. Photographic Screening for Retinopathy of Prematurity (Photo-ROP) Cooperative Group. The photographic screening for retinopathy of prematurity study (photo-ROP): primary outcomes. Retina. 2008;28:47S–54S. doi:10.1097/IAE.0b013e31815e987f [CrossRef]
  12. Dhaliwal C, Wright E, Graham C, McIntosh N, Fleck BW. Wide-field digital retinal imaging versus binocular indirect ophthalmoscopy for retinopathy of prematurity screening: a two-observer prospective, randomised comparison. Br J Ophthalmol. 2009;93:355–359. doi:10.1136/bjo.2008.148908 [CrossRef]
  13. Dai S, Chow K, Vincent A. Efficacy of wide-field digital retinal imaging for retinopathy of prematurity screening. Clin Exp Ophthalmol. 2011;39:23–29.
  14. Quinn GE, Ying GS, Daniel E, et al. Validity of a telemedicine system for the evaluation of acute-phase retinopathy of prematurity. JAMA Ophthalmol. 2014;132:1178–1184. doi:10.1001/jamaophthalmol.2014.1604 [CrossRef]
  15. Morrison D, Bothun ED, Ying GS, Daniel E, Baumritter A, Quinn G. Impact of number and quality of retinal images in a telemedicine screening program for ROP: results from the e-ROP study. J AAPOS. 2016;20:481–485. doi:10.1016/j.jaapos.2016.08.004 [CrossRef]
  16. Biten H, Redd TK, Moleta C, et al. Diagnostic accuracy of ophthalmoscopy vs telemedicine in examinations for retinopathy of prematurity. JAMA Ophthalmol. 2018;136:498–504. doi:10.1001/jamaophthalmol.2018.0649 [CrossRef]
  17. Schwartz SD, Harrison SA, Ferrone PJ, Trese MT. Telemedical evaluation and management of retinopathy of prematurity using a fiberoptic digital fundus camera. Ophthalmology. 2000;107:25–28. doi:10.1016/S0161-6420(99)00003-2 [CrossRef]
  18. Lorenz B, Spasovska K, Elflein H, Schneider N. Wide-field digital imaging based telemedicine for screening for acute retinopathy of prematurity (ROP): six years of results of a multicentre field study. Graefes Arch Clin Exp Ophthalmol. 2009;247:1251–1262. doi:10.1007/s00417-009-1077-7 [CrossRef]
  19. Weaver DT, Murdock TJ. Telemedicine detection of type 1 ROP in a distant neonatal intensive care unit. J AAPOS. 2012;16:229–233. doi:10.1016/j.jaapos.2012.01.007 [CrossRef]
  20. Wang SK, Callaway NF, Wallenstein MB, Henderson MT, Leng T, Moshfeghi DM. SUNDROP: six years of screening for retinopathy of prematurity with telemedicine. Can J Ophthalmol. 2015;50:101–106. doi:10.1016/j.jcjo.2014.11.005 [CrossRef]
  21. Fierson WM. Screening examination of premature infants for retinopathy of prematurity. Pediatrics. 2013;131:189–195. doi:10.1542/peds.2012-2996 [CrossRef]
  22. Early Treatment for Retinopathy of Prematurity Cooperative Group. Revised indications for the treatment of retinopathy of prematurity: results of the early treatment for retinopathy of prematurity randomized trial. Arch Ophthalmol. 2003;121:1684–1694. doi:10.1001/archopht.121.12.1684 [CrossRef]
  23. Jackson KM, Scott KE, Graff Zivin J, et al. Cost-utility analysis of telemedicine and ophthalmoscopy for retinopathy of prematurity management. Arch Ophthalmol. 2008;126:493–499. doi:10.1001/archopht.126.4.493 [CrossRef]
  24. Quinn GE, Ying GS, Repka MX, et al. Timely implementation of a retinopathy of prematurity telemedicine system. J AAPOS. 2016;20:425–430. doi:10.1016/j.jaapos.2016.06.007 [CrossRef]
  25. Brown JM, Campbell JP, Beers A, et al. Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks. JAMA Ophthalmol. 2018;136:803–810. doi:10.1001/jamaophthalmol.2018.1934 [CrossRef]
  26. Williams A, Larocca R, Chang T, et al. Web-based depression screening and psychiatric consultation for college students: a feasibility and acceptability study. Int J Telemed Appl. 2014;2014:580786.
  27. de Araújo JS, Regis CT, Gomes RG, Mourato FA, Mattos SD. Impact of telemedicine in the screening for congenital heart disease in a center from northeast Brazil. J Trop Pediatr. 2016;62:471–476.
  28. Huang Y, Lei Y, Wang Q, et al. Telepathology consultation for frozen section diagnosis in China. Diagn Pathol. 2018;13:29. doi:10.1186/s13000-018-0705-0 [CrossRef]

RetCam Shuttle vs Clinical Examination Findings for Detecting Referral-Warranted ROP

RetCam ShuttleClinical

Positive (+)Negative (−)Total
Positive (+)213
Negative (−)03232
Total23335
Authors

From the Department of Ophthalmology and Visual Sciences, Truhlsen Eye Institute, University of Nebraska Medical Center, Omaha, Nebraska (BAB, DWS); Children's Hospital and Medical Center, Omaha, Nebraska (DWS); Avera McKennan Hospital and University Health Center, Sioux Falls, South Dakota (JM); and Sanford Children's Hospital, Sioux Falls, South Dakota (GTT).

The authors have no financial or proprietary interest in the materials presented herein.

Correspondence: Brett A. Begley, BA, 985540, University of Nebraska Medical Center, Omaha, NE 68198-5540. E-mail: brett.begley@unmc.edu

Received: November 20, 2018
Accepted: January 14, 2019

10.3928/01913913-20190215-01

Sign up to receive

Journal E-contents