Ophthalmic Surgery, Lasers and Imaging Retina

Clinical Science Open Access

Use of Corticosteroids in the Treatment of Patients With Diabetic Macular Edema Who Have a Suboptimal Response to Anti-VEGF: Recommendations of an Expert Panel

Carl D. Regillo, MD; David G. Callanan, MD; Diana V. Do, MD; Howard F. Fine, MD, MHSc; Nancy M. Holekamp, MD; Baruch D. Kuppermann, MD, PhD; Michael A. Singer, MD; Rishi P. Singh, MD

  • Ophthalmic Surgery, Lasers and Imaging Retina. 2017;48(4):291-301
  • https://doi.org/10.3928/23258160-20170329-03
  • Posted April 19, 2017

Abstract

BACKGROUND AND OBJECTIVE:

Guidance on the use of corticosteroids in the treatment of diabetic macular edema (DME) is lacking. This study aimed to develop a clinically recommended treatment paradigm for DME with emphasis on the role of corticosteroids.

PATIENTS AND METHODS:

An expert panel of nine retinal specialists in the United States developed consensus recommendations for DME treatment through a modified Delphi process.

RESULTS:

The panelists typically use intravitreal injections of vascular endothelial growth factor (VEGF) antagonists as first-line treatment of DME and switch patients with an inadequate response to anti-VEGF therapy (failure of best-corrected visual acuity to improve to 20/40 or better because of edema after three to six monthly injections, or a less-than-50% reduction in excess macular thickness after three to four monthly injections) to intravitreal corticosteroid treatment.

CONCLUSION:

Intravitreal corticosteroids have a potentially useful role in the treatment of patients with DME who have an inadequate response to intravitreal anti-VEGF therapy.

[Ophthalmic Surg Lasers Imaging Retina. 2017;48:291–301.]

Abstract

BACKGROUND AND OBJECTIVE:

Guidance on the use of corticosteroids in the treatment of diabetic macular edema (DME) is lacking. This study aimed to develop a clinically recommended treatment paradigm for DME with emphasis on the role of corticosteroids.

PATIENTS AND METHODS:

An expert panel of nine retinal specialists in the United States developed consensus recommendations for DME treatment through a modified Delphi process.

RESULTS:

The panelists typically use intravitreal injections of vascular endothelial growth factor (VEGF) antagonists as first-line treatment of DME and switch patients with an inadequate response to anti-VEGF therapy (failure of best-corrected visual acuity to improve to 20/40 or better because of edema after three to six monthly injections, or a less-than-50% reduction in excess macular thickness after three to four monthly injections) to intravitreal corticosteroid treatment.

CONCLUSION:

Intravitreal corticosteroids have a potentially useful role in the treatment of patients with DME who have an inadequate response to intravitreal anti-VEGF therapy.

[Ophthalmic Surg Lasers Imaging Retina. 2017;48:291–301.]

Introduction

Diabetic macular edema (DME) is characterized by accumulation of fluid and retinal thickening within the macula. The pathogenesis of DME is multifactorial. Vascular endothelial growth factor (VEGF) has a central role in the breakdown of the blood-retinal barrier that leads to DME,1 but other pro-inflammatory factors and processes also appear to be involved.2,3 DME is the most common cause of vision loss in patients with diabetes.4 It is estimated to affect approximately one in 25 individuals with diabetes older than age 40 in the United States,5 and the number of cases of DME can be expected to increase because of the growing prevalence of diabetes and its complications.6

The American Academy of Ophthalmology Preferred Practice Patterns-recommended first-line treatment for center-involved DME is intravitreal anti-VEGF therapy, with or without adjunct focal laser treatment.7 Ranibizumab 0.3 mg (Lucentis; Genentech, South San Francisco, CA) and aflibercept 2 mg (Eylea; Regeneron, Tarrytown, NY) are U.S. Food and Drug Administration (FDA)-approved anti-VEGF treatments for DME. Bevacizumab 1.25 mg (Avastin; Genentech, South San Francisco, CA) is commonly used via an intravitreal injection in an off-label fashion for treatment of DME; a formulation approved for systemic cancer therapy is repackaged in syringes by a compounding pharmacy for off-label intravitreal ophthalmic use. Many studies have demonstrated that intravitreal anti-VEGF treatment has a favorable tolerability profile and is effective in reducing central retinal thickness and improving best-corrected visual acuity (BCVA) in patients with DME.8 Even with monthly administration of anti-VEGF, however, the response to anti-VEGF therapy is suboptimal in a substantial proportion of patients.9–11 After 2 years of monthly ranibizumab 0.3 mg injections for the treatment of DME in the RISE/RIDE registration trials, central foveal thickness remained greater than 250 μm on time-domain optical coherence tomography (OCT) in 24.8% of patients, and BCVA was worse than 20/40 in 42.8% of patients.9

Corticosteroids are also available for the treatment of DME. Although intraocular corticosteroid treatment is commonly associated with adverse effects of cataract and increases in intraocular pressure (IOP), intravitreal corticosteroids have demonstrated efficacy in improving central retinal thickness and BCVA in DME and, therefore, are viable treatment options.7,12,13 In the United States, dexamethasone intravitreal implant 0.7 mg (DEX implant) (Ozurdex; Allergan plc, Dublin, Ireland) is FDA-approved for treatment of DME, and intravitreal fluocinolone acetonide insert (FAc) (Iluvien; Alimera Sciences, Alpharetta, GA) is approved for treatment of DME in patients who have been previously treated with a course of corticosteroids and did not have a clinically significant rise in IOP. In addition, intravitreal triamcinolone acetonide (TA) has been used off-label for many years and has demonstrated beneficial effects in patients with DME.14

There is a need for guidelines on how DME should be treated to provide patients with the best possible outcomes when the response to anti-VEGF therapy is inadequate. To address this need, a panel of clinical experts in retinal ophthalmology was brought together to develop consensus recommendations for DME treatment through a modified Delphi process. The objective was to develop a clinically recommended treatment paradigm for DME, with particular focus on how corticosteroids fit within the treatment paradigm.

Patients and Methods

The Delphi technique is a widely used and accepted method for building consensus from experts in a field through iterations of a survey and anonymous feedback; in each round of the survey, the participants review and assess the results and feedback from the previous round, then respond to the survey again, and the process is repeated until consensus is reached.15 A modified Delphi approach incorporating a face-to-face meeting led by a moderator has been used in previous studies to develop recommendations for the assessment and management of ophthalmic diseases16–21 and was used in this study to develop a recommended treatment paradigm for DME. A panel of nine retina specialists with expertise in the treatment of diabetic retinopathy and DME were invited and agreed to participate in the development of the treatment guidelines. After a literature review was conducted to identify current treatment patterns in DME, a survey including 22 multipart, multiple-choice, and open-ended questions regarding treatment scenarios and clinical decision-making was developed by Endpoint Outcomes and Allergan (See Appendix A below). The questions were designed to elucidate key elements of the treatment paradigm including assessment of the response to anti-VEGF therapy, the role of corticosteroids in the treatment paradigm, differentiation of available corticosteroid treatment options, and corticosteroid use and side effects.

The panel responded to the survey in two formal rounds of feedback. The first round of the survey was conducted on the SurveyMonkey online survey platform ( www.surveymonkey.com). Nine panelists participated in the survey and provided anonymous feedback. Following the first round, the results were summarized and provided to the panelists. Eight of the panelists then participated in the second round of the survey at a “real-time” Delphi panel meeting on June 4, 2016, in Philadelphia, Pa. At the meeting, the anonymous summary results of the first round were presented to the panelists. Answers to open-ended questions in the first round were used to frame multiple-choice answers to survey questions for the second round (See Appendix B below). For each survey question, if consensus was not reached in the first round, the results were discussed and debated by the panelists. The panelists then voted for their chosen response. An electronic audience response system (ResponseCards; Turning Technologies, Youngstown, OH) was used to capture anonymous votes during the Delphi panel discussion, and all panelists were required to vote before results were displayed. Discussion was continued and voting was repeated until consensus was reached, or it was clear that it was impossible to achieve consensus. During this process, modification of survey questions and possible responses, based on opinions expressed during the discussion, was allowed to facilitate the achievement of consensus. Consensus was defined as agreement by at least six of the nine panelists in the first round (66.7%), or by at least six of eight panelists at the Delphi panel meeting (75%).

Results

The panelists reached consensus on many issues in the treatment of DME (Table 1) and developed a recommended treatment paradigm for center-involved DME in patients who have an incomplete response to anti-VEGF treatment (Figure).

�A;Delphi Panel Consensus on Issues in the Treatment of DME

Table 1:

Delphi Panel Consensus on Issues in the Treatment of DME

�A;Recommended treatment paradigm for center-involved DME. BCVA = best-corrected visual acuity; DME = diabetic macular edema; VEGF = vascular endothelial growth factor.

Figure.

Recommended treatment paradigm for center-involved DME. BCVA = best-corrected visual acuity; DME = diabetic macular edema; VEGF = vascular endothelial growth factor.

The key aspects of the treatment paradigm are as follows.

Disease Classification

The majority of panelists (75%; six of eight) agreed that clinical decision-making on whether and how to treat DME begins with the classification of the DME as center-involved or non-center-involved.

First-line Treatment for Center-Involved DME: Anti-VEGF Therapy

Members of the expert panel typically use intravitreal bevacizumab, ranibizumab, and aflibercept as first-line treatment for center-involved DME. In the first round (online) survey, there was consensus agreement (77.8%; seven of nine) that if there was a suboptimal response to anti-VEGF injections early in treatment (after three or fewer injections), the patient is typically switched to another anti-VEGF agent. In the discussion at the Delphi meeting, panelists clarified that insurance coverage and reimbursement can affect the choice of anti-VEGF agent initially used. When the early response to anti-VEGF is poor, they may switch to a different anti-VEGF agent that is believed to be more efficacious, but they generally limit the total time onanti-VEGF-only therapy to 6 months for patients who have an inadequate response.

The response to anti-VEGF treatment is variable among patients. All panelists agreed that the relative roles of VEGF and other inflammatory cytokines in the edema is the most important factor contributing to different levels of patient response to anti-VEGF. The panelists also all agreed that the chronicity of the edema is a second key factor contributing to different levels of patient response to anti-VEGF.

There was unanimous agreement that long-term DME results in poorer outcomes. Panelists stated that chronic DME may be less responsive to anti-VEGF therapy and is likely to involve inflammatory pathways and cytokines other than VEGF that can cause tissue damage. Photoreceptor damage and permanent loss of visual acuity can occur in chronic DME. Prompt treatment of center-involved DME with anti-VEGF is recommended for optimal outcomes.

Evaluating the Response to Anti-VEGF Therapy

All panelists agreed that the most important determinant of an ideal response to anti-VEGF therapy is the resolution of edema. Most of the panelists (87.5%; seven of eight) further agreed that improvement in visual acuity is the second most important determinant of an ideal response to anti-VEGF therapy. Improvement in visual acuity is important, but it is a secondary factor in evaluation of the treatment response because other factors beyond DME can affect vision.

A patient can be determined to be an inadequate responder to anti-VEGF therapy based on anatomic criteria, or a combination of BCVA and anatomic criteria. Almost all (87.5%; seven of eight) of the panelists agreed that a DME patient who after three to four monthly anti-VEGF injections has a less than 50% reduction from baseline in the excess macular thickness (defined as thickness over the upper limit of normal) is an inadequate responder to anti-VEGF therapy. In addition, the majority of panelists (75%; six of eight) agreed that a DME patient whose BCVA after three to six monthly anti-VEGF injections has failed to improve to 20/40 or better because of edema is an inadequate responder to anti-VEGF therapy.

The panel was unable to agree on a value for the central subfield thickness on OCT that indicates an inadequate response to anti-VEGF therapy. Although the central subfield thickness measurement is valuable for evaluation of changes in retinal thickness over time and after treatment, qualitative assessment of the persistence of edema, which is associated with visual acuity loss, may drive treatment decisions.

Almost all panelists (87.5%; seven of eight) agreed that they are prepared to tolerate persistent edema for 4 to 6 months in patients who are receiving monthly anti-VEGF injections. Panelists agreed that waiting longer than 6 months to move on to other modalities of treatment is potentially detrimental. It was noted that DME may be present for months before patients are seen by a clinician and initiate treatment, so the persistence of edema through 6 months of treatment may indicate the development of chronic DME, which involves inflammatory pathways and cytokines other than VEGF and potentially has reduced responsiveness to anti-VEGF therapy. Panelists cited clinical data (the EARLY Study) showing that the BCVA outcomes of patients in the DRCR.net Protocol I study after 3 years of ranibizumab 0.5 mg treatment could be predicted by results at week 12.11 These results support a change in therapy when edema persists and BCVA does not improve after 3 months of ranibizumab treatment, because most patients who had less-than-five-letter BCVA gain from baseline after 3 months continued to have less-than-five-letter BCVA gain from baseline after 3 years of ranibizumab administration.

Reasons to Change Therapy

In the first round of the survey, all panelists reported that they were somewhat or very likely to use treatment options in addition to anti-VEGF if monthly injections were too burdensome for the patient (ie, there was poor compliance with frequent injections). After discussion at the Delphi panel meeting, the majority of panelists (62.5%; five of eight) agreed that they typically would be likely to consider switching anti-VEGF treatment to other therapy if monthly injections are too burdensome. Panelists emphasized that to facilitate compliance, it is important to discuss with patients the importance of frequent injections to preserve vision.

The panelists agreed that persistent edema and lack of improvement in BCVA are important factors driving decisions to change therapy. In the first round of the survey, all panelists reported that they would be somewhat or very likely to incorporate additional treatment modalities beyond anti-VEGF if fluid was still present on the retina, and all but one of the panelists responded that they would be somewhat or very likely to consider switching or stopping anti-VEGF treatment in this scenario. Similarly, all panelists reported that they would be somewhat or very likely to incorporate additional treatment modalities beyond anti-VEGF if there was lack of anatomical improvement (eg, in retinal thickness). In the second round, consensus was reached regarding switching treatment, and 75% (six of eight) of panelists agreed that they typically would be likely to consider switching from anti-VEGF to another treatment modality if there was a lack of anatomical improvement. Consensus was reached in the first round of the survey with respect to the effect of lack of improvement in BCVA on treatment decisions. Most panelists (77.8%; seven of nine) reported that they would be very likely to incorporate additional treatment modalities in patients who had no improvement in BCVA with anti-VEGF therapy, and almost all panelists (88.9%; eight of nine) reported that they would be somewhat or very likely to switch or stop anti-VEGF treatment.

Most panelists (87.5%; seven of eight) also agreed that they typically would be likely to consider switching from anti-VEGF to another treatment modality in patients who have had a recent stroke or cardiovascular event.

The panelists were unable to reach consensus on the relative importance of treating the VEGF component of DME compared with the inflammatory component of DME. However, all panelists agreed that it is more important to treat the inflammatory component of DME than the VEGF component of DME after it has been established that the response to anti-VEGF therapy is suboptimal.

Treatment When the Response to Anti-VEGF Is Inadequate

The panelists agreed unanimously that they would be most likely to switch a patient who is an inadequate responder after three to six monthly injections of anti-VEGF agents to corticosteroid treatment. All panelists reported they are most likely to switch an anti-VEGF inadequate responder to DEX implant treatment, and their second most likely choice for corticosteroid therapy is intravitreal TA (Triesence; Alcon, Fort Worth, TX).

All panelists further agreed that laser photocoagulation should be used as an adjunct treatment in patients with center-involved DME, but should not be considered a rescue treatment. Panelists agreed that the rescue treatment for patients with suboptimal anti-VEGF response is a corticosteroid.

Role of Corticosteroids in the Treatment Paradigm

Consensus was reached that intravitreal corticosteroid treatment is appropriate in multiple scenarios of DME (Table 2). Phakic as well as pseudophakic patients who have an inadequate response to anti-VEGF therapy can be candidates for corticosteroid treatment, and corticosteroid treatment can be used in patients with severe edema and those who have undergone vitrectomy. Corticosteroid treatment can also be considered in DME patients with co-existing glaucoma as long as IOP is controlled on one or two medications and the optic nerve is healthy. Several panelists indicated that patients who have had successful filtering surgery can be candidates for corticosteroid therapy as long as the cup-to-disc ratio does not exceed 0.8.

�A;Delphi Panel Consensus on Candidates for Corticosteroid Treatment of DME

Table 2:

Delphi Panel Consensus on Candidates for Corticosteroid Treatment of DME

Patient Monitoring

Most panelists (77.8%; seven of nine) reported that their patients who are on anti-VEGF therapy are monitored monthly. There was no consensus regarding the frequency of monitoring for patients on corticosteroid-only therapy. Two panelists reported that they monitor patients monthly, four monitor patients at intervals of 6 to 8 weeks, and two monitor 1 month after the corticosteroid injection and 2 to 3 months afterward, depending on the IOP. During subsequent discussion, it was pointed out that these results indicate that the panelists typically monitor patients on corticosteroid-only therapy every 1 to 2 months. Patients who receive a sustained-release implant may be monitored at 4 to 6 weeks after their first corticosteroid injection, then subsequently at extended intervals not exceeding 3 months.

For patients receiving a combination of anti-VEGF and corticosteroid treatment, the frequency of monitoring is dependent on whether patients are new to or established on the combination treatment. Most panelists (75%; six of eight) agreed that patients who are new to combination therapy should be monitored every 1 to 2 months, and almost all panelists (87.5%; seven of eight) agreed that patients who are established on combination therapy should be monitored every 2 months.

Choosing Among Corticosteroid Options

All panelists agreed that they would prefer to use a corticosteroid approved by the FDA for DME treatment (DEX implant or FAc) when considering efficacy, safety, and duration of effect, and all panelists reported that they use an FDA-approved corticosteroid in most cases. In their comments on the first-round survey, panelists indicated that they usually treat patients with DEX implant and sometimes use the FAc implant after DEX implant treatment in patients who are not steroid responders, who have shown a good response to multiple DEX implants, and who need long-term treatment.

Some panelists reported that they occasionally use a nonapproved corticosteroid (TA) when lack of insurance coverage or cost is an issue.

Management of the Side Effects of Corticosteroid Treatment

All panelists agreed that side effects of corticosteroid treatment are monitored in conjunction with efficacy, and no additional office visits are scheduled to monitor for side effects, unless a problem is detected and requires follow-up. Panelists commented that side effects of cataract are manageable and side effects of increases in IOP are generally manageable. The panelists reached consensus on how they treat increased IOP in patients on corticosteroid therapy. For patients who have elevated IOP of 25 mm Hg or less, six panelists (75%) agreed that they would prescribe a single topical IOP-lowering medication, and for patients with IOP between 26 mm Hg and 30 mm Hg, almost all panelists (87.5%; seven of eight) agreed that they would prescribe a fixed-combination IOP-lowering eye drop. For patients with IOP higher than 30 mm Hg, all panelists agreed that they would prescribe a fixed-combination IOP-lowering eye drop or refer the patient to a glaucoma specialist, and most agreed that they would both prescribe the fixed combination and make the referral.

Discussion

The introduction of anti-VEGF treatment has greatly improved visual outcomes in patients with DME, but a significant proportion of patients have an incomplete response to anti-VEGF treatment.9–11 Corticosteroids are a rational approach to the treatment of DME because inflammatory mediators and pathways in addition to VEGF appear to be involved in the development of DME. However, there has been limited practical guidance on when and how to use corticosteroids in the treatment regimen. The consensus recommendation of this panel is that corticosteroids be used in patients with an inadequate response to anti-VEGF treatment, and guidance is provided on how to identify patients who are inadequate responders to anti-VEGF, and when to progress to corticosteroid therapy.

The recommendation of the panel is that center-involved DME with some degree of decreased visual acuity should be treated promptly with anti-VEGF therapy. Most patients with DME respond well to anti-VEGF treatment. Currently bevacizumab is used more frequently than ranibizumab or aflibercept for treatment of ophthalmic disease in clinical practice22 because of its lower cost. In the DRCR.net Protocol T study comparing these anti-VEGF agents in patients with DME, after 1 year of treatment, aflibercept was more effective than ranibizumab or bevacizumab in improving BCVA in patients with baseline BCVA worse than 20/40, whereas the anti-VEGF agents showed comparable efficacy in patients with baseline BCVA of 20/32 to 20/40.23 After 2 years of treatment, aflibercept and ranibizumab provided similar improvement in BCVA, and aflibercept continued to be more effective than bevacizumab in improving BCVA in patients with baseline BCVA worse than 20/40.24 Although the efficacy of switching among anti-VEGF therapies has not been well studied,25 favorable outcomes have been reported after patients with suboptimal response switched from bevacizumab to ranibizumab,26 or from bevacizumab or ranibizumab to aflibercept.27 Therefore, if a patient has a poor early response to anti-VEGF, and the initial anti-VEGF agent used was not aflibercept because of considerations such as cost and reimbursement, the clinician may consider switching the patient to aflibercept. The consensus of the panel, however, is that whether or not there is a switch among anti-VEGF therapies, continued edema should be tolerated for no longer than 6 months.

There are three fundamental reasons why a patient with an inadequate response to anti-VEGF therapy typically should remain on anti-VEGF treatment for no longer than 6 months before proceeding to a corticosteroid. First, there is evidence that anti-VEGF treatment is most effective in early disease.28–30 In the RISE/RIDE registration studies of ranibizumab, patients in the initial sham group who crossed over to ranibizumab treatment at year 2 had less favorable outcomes after 12 months of ranibizumab treatment than patients in the initial ranibizumab groups,28 and patients with a longer duration of DME at baseline required more as-needed ranibizumab injections during the open-label study extension after year 3,29 suggesting that prompt treatment of shorter-term DME with ranibizumab results in better outcomes. Similarly, in the VIVID/VISTA registration studies of aflibercept, patients in the initial laser treatment group achieved a smaller gain in BCVA after receiving rescue aflibercept treatment compared with patients in the initial aflibercept treatment groups, suggesting that delaying aflibercept treatment results in poorer outcomes and that aflibercept treatment is more effective in shorter-term DME.30 Second, continuing anti-VEGF treatment when the early response to treatment is poor usually does not result in favorable outcomes. The EARLY study analysis of results from the DRCR.net Protocol I study showed that the majority of patients who had a less-than-five-letter gain in BCVA from baseline at week 12 (after three monthly ranibizumab injections) continued to have less-than-five-letter gains in BCVA from baseline at week 156 (after 3 years of monthly ranibizumab injections).11 Third, delaying effective treatment increases the risk of permanent vision loss, because chronicity of DME is associated with foveal atrophy and photoreceptor damage.31

Members of the panel agreed that the balance of VEGF versus other inflammatory cytokines appears to be an important factor contributing to the different levels of anti-VEGF response observed in patients with DME. The importance of this balance goes hand-in-hand with the importance of the chronicity of DME in the variability observed in patient response to anti-VEGF treatment, because it is generally believed that DME is primarily mediated by VEGF in its early phase, whereas the involvement of other cytokines and inflammatory mechanisms becomes more important in chronic disease.2 Low-grade inflammation has been postulated to be responsible for the retinal damage that occurs in chronic DME.2

It may be important to treat the inflammatory component of DME in patients who do not respond adequately to anti-VEGF treatment. Studies have demonstrated the effectiveness of corticosteroid treatment in long-standing and refractory DME. In the MEAD registration studies for DEX implant, the treatment effects of DEX implant relative to sham were similar in subgroups defined by the duration of DME at baseline: DEX implant had beneficial effects in longer-term (duration of more than 3 years) as well as short-term (less than 1 year in duration) DME.12,32 Furthermore, in the FAME registration studies for FAc, the percentage of patients with 15-letter or greater gains in BCVA from baseline was superior with FAc relative to sham only in the subgroup of patients with long-standing DME (duration of 3 years or longer),33 consistent with an important role of inflammatory pathways amenable to corticosteroid treatment in the pathophysiology of persistent DME.

The panel recommends that a patient who has a less than 50% reduction from baseline in the excess macular thickness after three to four monthly anti-VEGF injections, or whose BCVA has not improved to 20/40 after three to six monthly injections because of edema, be switched to corticosteroid treatment. The definitions of inadequate response include a range in the number of anti-VEGF injections, because the decision of whether to change treatment should take into account the trajectory of the response. A patient with continual improvement in edema after three or four sequential injections typically would be re-treated with anti-VEGF, whereas a patient with no improvement in edema after three or four anti-VEGF injections typically would be switched to a corticosteroid.

The panelists recommend that patients on anti-VEGF therapy receive frequent (monthly) injections and have monthly assessments to evaluate the trajectory of the response to treatment and identify inadequate responders. However, they recognize that some practitioners may see patients monthly for anti-VEGF injections but wait until after three to four injections before doing a full examination and OCT, rather than doing full exams at every visit. With this approach, the decision of whether to continue anti-VEGF treatment or switch to a corticosteroid is still made 3 to 4 months after the initial anti-VEGF injection, consistent with the suggestion from the EARLY study that this is an appropriate timeframe for deciding whether to switch therapy, since BCVA outcomes at week 12 in the Protocol I study were predictive of long-term outcomes.11 Although this approach does not consider the trajectory of the treatment response, there are cost savings associated with fewer examinations, and there is little risk of overtreatment, since edema is unlikely to resolve after one to three anti-VEGF injections. The panelists also recognize that the frequency of anti-VEGF injections required can be burdensome for patients,34,35 but in the experience of the panelists, patients are likely to adhere to frequent injections if they are educated on the alternative of risk of permanent loss of vision. The consensus opinion of the panel is that frequent monitoring is also needed for patients who have switched to corticosteroid implant treatment, but the recommended interval between visits for these patients is longer (1 to 2 months) than that recommended for patients on anti-VEGF therapy.

The panel's consensus choice of corticosteroid for treatment of DME in inadequate responders to anti-VEGF is DEX implant, primarily because DEX implant has a more favorable safety profile compared with FAc and intravitreal TA. The increases in IOP associated with intraocular corticosteroid use are less frequent, less severe, and more easily managed with DEX implant than with FAc or intravitreal TA.12,13,36–38 Accordingly, the FDA-approved indication for FAc in DME is limited to patients who have been demonstrated to have no clinically significant IOP response to a corticosteroid. Disadvantages of use of intravitreal TA include its lack of FDA approval for treatment of DME and the possible need for more frequent injections. DEX implant has been demonstrated to be effective in patients with previous vitrectomy,39 patients with previously treated DME,40 and patients with DME that has been demonstrated to be resistant to anti-VEGF treatment.41–45

The primary role of corticosteroids in the treatment paradigm for DME is as alternative treatment after a suboptimal anti-VEGF response, but corticosteroids may also be considered for treatment in patients who discontinue anti-VEGF therapy because of systemic safety concerns. Although intravitreal anti-VEGF treatment is generally believed to be systemically safe, it has been difficult to determine whether anti-VEGF might increase the risk of stroke, arteriothrombotic events, or cardiovascular events because studies have been underpowered and patients with DME may be at increased risk of these events regardless of treatment.46 However, the majority of panelists agreed that they typically would likely consider a change in therapy if a patient being treated with anti-VEGF had a recent stroke or cardiovascular event. In a recent meta-analysis that pooled systemic safety data from patients treated with ranibizumab 0.5 mg or aflibercept in clinical studies, the risk of cerebrovascular accidents and vascular deaths was significantly higher in patients treated monthly for 2 years with anti-VEGF injections than in patients treated with sham,47 suggesting that frequent anti-VEGF injections over the long term may increase the risk of serious systemic adverse events.

There is a lack of evidence from clinical studies concerning some aspects of care in DME, such as the benefits of using combination therapy with an anti-VEGF agent and a corticosteroid. In their discussion at the Delphi meeting, panelists indicated that they generally use combination anti-VEGF and corticosteroid therapy only for patients with severe and persistent DME. Furthermore, based on their clinical experience, the panelists believe that a patient with a suboptimal response to anti-VEGF has better improvement in retinal drying if the patient is switched to a corticosteroid than if the patient is switched to a different anti-VEGF agent, but supporting data from clinical trials are lacking.

The clinically recommended treatment paradigm for DME presented here includes recommendations for the medical treatment of DME, and it was developed with the understanding that the recommendations should be general and applicable in 80% of cases. The authors recognize that there are exceptions and that there is a need for individualized patient care, as well as a need for surgery in cases of epiretinal membrane or vitreomacular traction. Some patients will lose vision irretrievably even if the recommendations are followed. The recommendations are based on evidence from reported studies and the clinical experience of the panelists, and they may be modified when more data become available. It is anticipated that results of the ongoing DRCR.net Protocol U study ( clinicaltrials.gov identifier: NCT01945866), comparing ranibizumab monotherapy to combination therapy with ranibizumab and DEX implant in patients with an incomplete response to three monthly injections of ranibizumab, may further help guide corticosteroid use in the treatment of DME.

References

  1. Boyer DS, Hopkins JJ, Sorof J, Ehrlich JS. Anti-vascular endothelial growth factor therapy for diabetic macular edema. Ther Adv Endocrinol Metab. 2013;4(6):151–169. doi:10.1177/2042018813512360 [CrossRef]
  2. Romero-Aroca P, Baget-Bernaldiz M, Pareja-Rios A, Lopez-Galvez M, Navarro-Gil R, Verges R. Diabetic macular edema pathophysiology: Vasogenic versus inflammatory. J Diabetes Res. 2016; 2016:2156273. doi:10.1155/2016/2156273 [CrossRef]
  3. Dugel PU, Bandello F, Loewenstein A. Dexamethasone intravitreal implant in the treatment of diabetic macular edema. Clin Ophthalmol. 2015;9:1321–1335. doi:10.2147/OPTH.S79948 [CrossRef]
  4. Lee R, Wong TY, Sabanayagam C. Epidemiology of diabetic retinopathy, diabetic macular edema and related vision loss. Eye Vis (Lond). 2015;2:17. doi:10.1186/s40662-015-0026-2 [CrossRef]
  5. Varma R, Bressler NM, Doan QV, et al. Prevalence of and risk factors for diabetic macular edema in the United States. JAMA Ophthalmol. 2014;132(11):1334–1340. doi:10.1001/jamaophthalmol.2014.2854 [CrossRef]
  6. Holekamp NM. Overview of diabetic macular edema. Am J Manag Care. 2016; 22(10 Suppl):s284–s291.
  7. American Academy of Ophthalmology Preferred Practice Pattern (PPP) Retina/Vitreous Panel. Diabetic retinopathy PPP – Updated 2016. https://www.aao.org/preferred-practice-pattern/diabetic-retinopathy-ppp-updated-2016. Accessed November 15, 2016.
  8. Bandello F, Cicinelli MV, Parodi MB. Anti-VEGF molecules for the management of diabetic macular edema. Curr Pharm Des. 2015;21(32):4731–4737. doi:10.2174/1381612821666150909095756 [CrossRef]
  9. Nguyen QD, Brown DM, Marcus DM, et al. Ranibizumab for diabetic macular edema: Results from 2 phase III randomized trials: RISE and RIDE. Ophthalmology. 2012;119(4):789–801. doi:10.1016/j.ophtha.2011.12.039 [CrossRef]
  10. Brown DM, Schmidt-Erfurth U, Do DV, et al. Intravitreal aflibercept for diabetic macular edema: 100-week results from the VISTA and VIVID studies. Ophthalmology. 2015;122(10):2044–2052. doi:10.1016/j.ophtha.2015.06.017 [CrossRef]
  11. Gonzalez VH, Campbell J, Holekamp NM, et al. Early and long-term responses to anti-vascular endothelial growth factor therapy in diabetic macular edema: analysis of Protocol I data. Am J Ophthalmol. 2016;172:72–79. doi:10.1016/j.ajo.2016.09.012 [CrossRef]
  12. Boyer DS, Yoon YH, Belfort R Jr, et al. Three-year, randomized, sham-controlled trial of dexamethasone intravitreal implant in patients with diabetic macular edema. Ophthalmology. 2014;121(10):1904–1914. doi:10.1016/j.ophtha.2014.04.024 [CrossRef]
  13. Campochiaro PA, Brown DM, Pearson A, et al. Sustained delivery fluocinolone acetonide vitreous inserts provide benefit for at least 3 years in patients with diabetic macular edema. Ophthalmology. 2012;119(10):2125–2132. doi:10.1016/j.ophtha.2012.04.030 [CrossRef]
  14. Elman MJ, Aiello LP, Diabetic Retinopathy Clinical Research Network et al. Randomized trial evaluating ranibizumab plus prompt or deferred laser or triamcinolone plus prompt laser for diabetic macular edema. Ophthalmology. 2010;117(6):1064–1077.e5. doi:10.1016/j.ophtha.2010.02.031 [CrossRef]
  15. Hsu CC, Sanford BA. The Delphi technique: Making sense of consensus. Practical Assessment, Research & Evaluation. 2007;12(10):1–8. http://pareonline.net/getvn.asp?v=12&n=10. Accessed November 15, 2016.
  16. Behrens A, Doyle JJ, Stern L, et al. Dysfunctional tear syndrome: A Delphi approach to treatment recommendations. Cornea. 2006;25(8):900–907. doi:10.1097/01.ico.0000214802.40313.fa [CrossRef]
  17. Lee PP, Sultan MB, Grunden JW, Cioffi GAIOP Consensus Panel. Assessing the importance of IOP variables in glaucoma using a modified Delphi process. J Glaucoma. 2010;19(5):281–287.
  18. Ferris FL 3rd, Wilkinson CP, Bird A, et al. Clinical classification of age-related macular degeneration. Ophthalmology. 2013;120(4):844–851.e7. doi:10.1016/j.ophtha.2012.10.036 [CrossRef]
  19. Ohno-Matsui K, Kawasaki R, Jonas JB, et al. International photographic classification and grading system for myopic maculopathy. Am J Ophthalmol. 2015;159(5):877–883. doi:10.1016/j.ajo.2015.01.022 [CrossRef]
  20. Bandello F, Midena E, Menchini U, Lanzetta P. Recommendations for the appropriate management of diabetic macular edema: Light on DME survey and consensus document by an expert panel. Eur J Ophthalmol. 2016;26(3):252–261. doi:10.5301/ejo.5000736 [CrossRef]
  21. Rodrigues IA, Sprinkhuizen SM, Barthelmes D, et al. Defining a minimum set of standardized patient-centered outcome measures for macular degeneration. Am J Ophthalmol. 2016;168:1–12. doi:10.1016/j.ajo.2016.04.012 [CrossRef]
  22. Parikh R, Ross JS, Sangaralingham LR, Adelman RA, Shah ND, Barkmeier AJ. Trends of anti-vascular endothelial growth factor use in ophthalmology among privately insured and Medicare Advantage patients. Ophthalmology. 2017;124(3):352–358. doi:10.1016/j.ophtha.2016.10.036 [CrossRef]
  23. Wells JA, Glassman AR, Diabetic Retinopathy Clinical Research Network et al. Aflibercept, bevacizumab, or ranibizumab for diabetic macular edema. N Engl J Med. 2015;372(13):1193–1203. doi:10.1056/NEJMoa1414264 [CrossRef]
  24. Wells JA, Glassman AR, Ayala AR, et al. Aflibercept, bevacizumab, or ranibizumab for diabetic macular edema: Two-year results from a comparative effectiveness randomized clinical trial. Ophthalmology. 2016;123(6):1351–1359. doi:10.1016/j.ophtha.2016.02.022 [CrossRef]
  25. Do DV, Nguyen QD, Vitti R, et al. Intravitreal aflibercept injection in diabetic macular edema patients with and without prior anti-vascular endothelial growth factor treatment: Outcomes from the phase 3 program. Ophthalmology. 2016;123(4):850–857. doi:10.1016/j.ophtha.2015.11.008 [CrossRef]
  26. Fechter C, Frazier H, Marcus WB, Farooq A, Singh H, Marcus DM. Ranibizumab 0.3 mg for persistent diabetic macular edema after recent, frequent, and chronic bevacizumab: The ROTATE trial. Ophthalmic Surg Lasers Imaging Retina. 2016;47(11):1–18. doi:10.3928/23258160-20161031-07 [CrossRef]
  27. Lim LS, Ng WY, Mathur R, et al. Conversion to aflibercept for diabetic macular edema unresponsive to ranibizumab or bevacizumab. Clin Ophthalmol. 2015;9:1715–1718. doi:10.2147/OPTH.S81523 [CrossRef]
  28. Brown DM, Nguyen QD, Marcus DM, et al. Long-term outcomes of ranibizumab therapy for diabetic macular edema: The 36-month results from two phase III trials: RISE and RIDE. Ophthalmology. 2013; 120(10):2013–2022. doi:10.1016/j.ophtha.2013.02.034 [CrossRef]
  29. Wykoff CC, Elman MJ, Regillo CD, Ding B, Lu N, Stoilov I. Predictors of diabetic macular edema treatment frequency with ranibizumab during the open-label extension of the RIDE and RISE Trials. Ophthalmology. 2016;123(8):1716–1721. doi:10.1016/j.ophtha.2016.04.004 [CrossRef]
  30. Heier JS, Korobelnik JF, Brown DM, et al. Intravitreal aflibercept for diabetic macular edema: 148-week results from the VISTA and VIVID studies. Ophthalmology. 2016;123(11):2376–2385. doi:10.1016/j.ophtha.2016.07.032 [CrossRef]
  31. Channa R, Sophie R, Khwaja AA, et al. Factors affecting visual outcomes in patients with diabetic macular edema treated with ranibizumab. Eye (Lond). 2014;28(3):269–278. doi:10.1038/eye.2013.245 [CrossRef]
  32. Loewenstein AMEAD Study Group. MEAD: Diabetic macular edema trial subanalysis. Paper presented at: American Academy of Ophthalmology Annual Meeting. ; October 18–21, 2014. ; Chicago, IL. .
  33. Cunha-Vaz J, Ashton P, Iezzi R, et al. Sustained delivery fluocinolone acetonide vitreous implants: Long-term benefit in patients with chronic diabetic macular edema. Ophthalmology. 2014;121(10):1892–1903. doi:10.1016/j.ophtha.2014.04.019 [CrossRef]
  34. Shea AM, Curtis LH, Hammill BG, et al. Resource use and costs associated with diabetic macular edema in elderly persons. Arch Ophthalmol. 2008;126(12):1748–1754. doi:10.1001/archopht.126.12.1748 [CrossRef]
  35. Wallick CJ, Hansen RN, Campbell J, Kiss S, Kowalski JW, Sullivan SD. Comorbidity and health care resource use among commercially insured non-elderly patients with diabetic macular edema. Ophthalmic Surg Lasers Imaging Retina. 2015;46(7):744–751. doi:10.3928/23258160-20150730-09 [CrossRef]
  36. Diabetic Retinopathy Clinical Research Network. A randomized trial comparing intravitreal triamcinolone acetonide and focal/grid photocoagulation for diabetic macular edema. Ophthalmology. 2008;115(9):1447–1459.e10. doi:10.1016/j.ophtha.2008.06.015 [CrossRef]
  37. Elman MJ, Bressler NM, Qin H, et al. Expanded 2-year follow-up of ranibizumab plus prompt or deferred laser or triamcinolone plus prompt laser for diabetic macular edema. Ophthalmology. 2011;118(4):609–614. doi:10.1016/j.ophtha.2010.12.033 [CrossRef]
  38. Kiddee W, Trope GE, Sheng L, et al. Intraocular pressure monitoring post intravitreal steroids: A systematic review. Surv Ophthalmol. 2013;58(4):291–310. doi:10.1016/j.survophthal.2012.08.003 [CrossRef]
  39. Boyer DS, Faber D, Gupta S, et al. Dexamethasone intravitreal implant for treatment of diabetic macular edema in vitrectomized patients. Retina. 2011;31(5):915–923. doi:10.1097/IAE.0b013e318206d18c [CrossRef]
  40. Augustin AJ, Kuppermann BD, Lanzetta P, et al. Dexamethasone intravitreal implant in previously treated patients with diabetic macular edema: Subgroup analysis of the MEAD study. BMC Ophthalmol. 2015;15:150. doi:10.1186/s12886-015-0148-2 [CrossRef]
  41. Alshahrani ST, Dolz-Marco R, Gallego-Pinazo R, Diaz-Llopis M, Arevalo JFKKESH International Collaborative Retina Study Group. Intravitreal dexamethasone implant for the treatment of refractory macular edema in retinal vascular diseases: Results of the KKESH International Collaborative Retina Study Group. Retina. 2016;36(1):131–136. doi:10.1097/IAE.0000000000000616 [CrossRef]
  42. Zhioua I, Semoun O, Lalloum F, Souied EH. Intravitreal dexamethasone implant in patients with ranibizumab persistent diabetic macular edema. Retina. 2015;35(7):1429–1435. doi:10.1097/IAE.0000000000000490 [CrossRef]
  43. Totan Y, Güler E, Guraǧaç FB. Dexamethasone intravitreal implant for chronic diabetic macular edema resistant to intravitreal bevacizumab treatment. Curr Eye Res. 2016;41(1):107–113. doi:10.3109/02713683.2014.1002048 [CrossRef]
  44. Gutiérrez-Benítez L, Millan E, Arias L, Garcia P, Cobos E, Caminal M. Dexamethasone intravitreal implants for diabetic macular edema refractory to ranibizumab monotherapy or combination therapy. Arch Soc Esp Oftalmol. 2015;90(10):475–480. doi:10.1016/j.oftal.2015.04.003 [CrossRef]
  45. Lazic R, Lukic M, Boras I, et al. Treatment of anti-vascular endothelial growth factor-resistant diabetic macular edema with dexamethasone intravitreal implant. Retina. 2014;34(4):719–724. doi:10.1097/IAE.0b013e3182a48958 [CrossRef]
  46. Avery RL. What is the evidence for systemic effects of intravitreal anti-VEGF agents, and should we be concerned?Br J Ophthalmol. 2014;98Suppl 1:i7–10. doi:10.1136/bjophthalmol-2013-303844 [CrossRef]
  47. Avery RL, Gordon GM. Systemic safety of prolonged monthly anti-vascular endothelial growth factor therapy for diabetic macular edema: A systematic review and meta-analysis. JAMA Ophthalmol. 2016;134(1):21–29. doi:10.1001/jamaophthalmol.2015.4070 [CrossRef]


 

 

 

 



 

 

 

 



Delphi Panel Consensus on Issues in the Treatment of DME

Issue Consensus Opinion/Recommendation
Classification of DME Center-involved or non-center-involved
First-line treatment for center-involved DME Intravitreal anti-VEGF injections
Ideal response to anti-VEGF treatment <list-item>

Resolution of edema (primary)

</list-item><list-item>

Improvement in visual acuity (secondary)

</list-item>
Most important factor contributing to different levels of patient response to anti-VEGF The balance of VEGF versus other inflammatory cytokines
Second most important factor contributing to different levels of patient response to anti-VEGF The chronicity of the DME
Long-term DME Results in poorer outcomes
Treatment decision when early response to anti-VEGF (after three or more injections) is suboptimal Switch to a different anti-VEGF agent if initial anti-VEGF agent was not the most efficacious available
Definition of inadequate response to anti-VEGF therapy BCVA worse than 20/40 after three to six monthly anti-VEGF injections because of edema, or less-than-50% reduction in excess macular thickness after three to four monthly anti- VEGF injections
How long to tolerate persistent edema 4 to 6 months
Reasons to incorporate additional treatment modalities beyond anti-VEGF <list-item>

Fluid still present on the retina

</list-item><list-item>

Lack of anatomical improvement (eg, CRT)

</list-item><list-item>

Lack of improvement in BCVA

</list-item><list-item>

Monthly injections are too burdensome (ie, poor compliance)

</list-item>
Reasons to potentially consider switching or stopping anti-VEGF treatment <list-item>

Fluid still present on the retina

</list-item><list-item>

Lack of anatomical improvement (eg, CRT)

</list-item><list-item>

Lack of improvement in BCVA

</list-item><list-item>

Monthly injections are too burdensome (ie, poor compliance)

</list-item><list-item>

Recent stroke or cardiovascular event

</list-item>
Importance of treating the inflammatory versus the VEGF component of DME More important to treat the inflammatory component after a suboptimal response to anti-VEGF therapy
Treatment decision when patient has an inadequate response to anti-VEGF therapy Switch to a corticosteroid (consensus choice is DEX implant)
Choice of corticosteroid FDA-approved corticosteroid is preferred
Use of macular laser in center-involved DME Laser used as adjunct (not rescue) treatment
Frequency of monitoring <list-item>

Monthly for patients on anti-VEGF

</list-item><list-item>

Every 1 to 2 months for patients on corticosteroid

</list-item><list-item>

Every 1 to 2 months for patients newly on combination of anti-VEGF and corticosteroid

</list-item><list-item>

Every 2 months for patients established on combination of anti-VEGF and corticosteroid

</list-item>
Management of IOP increases during corticosteroid treatment <list-item>

Treat elevated IOP of 30 mm Hg or less with topical IOP-lowering medication (single medication or fixed combination)

</list-item><list-item>

For patients with IOP higher than 30 mm Hg, treat with fixed-combination IOP-lowering eye drop and/or refer the patient to a glaucoma specialist

</list-item>

Delphi Panel Consensus on Candidates for Corticosteroid Treatment of DME

Patient Scenario Definite or Likely Candidate for Corticosteroid Treatment Unlikely Candidate for Corticosteroid Treatment
Inadequate responder to anti-VEGF after three to six injections
Inadequate responder to anti-VEGF who is pseudophakic
Inadequate responder to anti-VEGF who is phakic and older than 60 years old
Inadequate responder to anti-VEGF who is phakic and younger than 60 years old
Patient in need of rescue
Patient who is scheduled to undergo cataract surgery
Patient with a history of vitrectomy
Patient with persistent DME
Patient with severe edema
Suboptimal response to anti-VEGF treatment after three injections
Patient who is resistant to laser photocoagulation
Patient who had successful filtration surgery to control IOP
Patient with POAG well controlled on one glaucoma drop
Patient with POAG well controlled on two glaucoma drops (status of optic nerve not specified)
Patient with POAG well controlled on two glaucoma drops and a healthy optic nerve

Save
Authors

From the Retina Service of Wills Eye Hospital, Mid Atlantic Retina, Philadelphia (CDR); Texas Retina Associates, Arlington, Texas (DGC); Byers Eye Institute, Stanford University School of Medicine, Palo Alto, CA (DVD); NJ Retina, New Brunswick, NJ (HFF); Rutgers UMDNJ Department of Ophthalmology, New Brunswick, NJ (HFF); Pepose Vision Institute, Chesterfield, MO (NMH); Gavin Herbert Eye Institute, University of California, Irvine, CA (BDK); Medical Center Ophthalmology Associates, San Antonio, TX (MAS); Cole Eye Institute, Cleveland (RPS).

This study was sponsored by Allergan plc, Dublin, and conducted by Endpoint Outcomes, Boston, MA. The study sponsor had the opportunity to review the manuscript but could not make changes in the manuscript; the content of the manuscript was fully controlled by and approved by the authors.

The authors report no proprietary interests in the products described in the article. Dr. Regillo is a consultant for Alcon, Allergan, Bayer, Genentech, Iconic, and Novartis and has received research grant support from Accucela, Alcon, Alimera, Allergan, Bayer, Genentech, GSK, Iconic, Novartis, Ophthotech, and Regeneron. Dr. Callanan is a consultant for AbbVie, Alcon, Allergan, Allgenesis, Forsight, Graybug, Regeneron, pSivida, and Santen. Dr. Do is a consultant for Allergan, Genentech, and Santen and has received research funding from Allergan, Genentech, Regeneron, and Santen. Dr. Fine is a consultant for Alimera, Allergan, Genentech, and Regeneron and has received research grant support from Alcon, Alimera, Allergan, Genentech, GSK, and Regeneron. Dr. Holekamp is a consultant for Alimera Sciences, Allergan, Genentech, Katalyst, Novartis, and Regeneron and has received research grant support from Alimera Sciences, Allergan, Genentech, and Opthotech. Dr. Kuppermann is a consultant for Acucela, Aerpio, Alcon, Alimera, Allegro, Allergan, Ampio, Catalyst, Dose, Eleven Biotherapeutics, Genentech, Glaukos, Lumenis, Novartis, Ophthotech, Regeneron, and ThromboGenics and has received research grant support from Alcon, Alimera, Allegro, Allergan, Apellis, Genentech, GSK, J-Cyte, Neurotech, Ohr, Ophthotech, Regeneron, and ThromboGenics. Dr. Singer is a consultant for Aerpio, Alimera, Allergan, Ampio, Genentech, and Santen and has received research grant support from Acucela, Aerpio, Alcon, Alimera, Allergan, Ampio, Genentech, Ophthotech, and Regeneron. Dr. Singh is a consultant for Alcon, Genentech, Optos, Regeneron, Shire, and Zeiss and has received research grant support from Alcon, Apellis, Ophthotech, Regeneron, and Roche.

The authors thank Chris Evans, Nate Johnson, and Nicole Lyn of Endpoint Outcomes for their contributions to the study. Writing and editorial assistance was provided to the authors by Kate Ivins, PhD, of Evidence Scientific Solutions, Philadelphia, and funded by Allergan plc. All authors met the ICMJE authorship criteria. Neither honoraria nor payments were made for authorship.

Address correspondence to Carl D. Regillo, MD, Wills Eye Hospital, 840 Walnut Street, Suite 1020, Philadelphia, PA 19107; email: cregillo@midatlanticretina.com.

This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International (https://creativecommons.org/licenses/by/4.0). This license allows users to copy and distribute, to remix, transform, and build upon the article, for any purpose, even commercially, provided the author is attributed and is not represented as endorsing the use made of the work.
Received: February 07, 2017
Accepted: March 20, 2017

10.3928/23258160-20170329-03

Sign up to receive

Journal E-contents