Journal of Refractive Surgery

The articles prior to January 2011 are part of the back file collection and are not available with a current paid subscription. To access the article, you may purchase it or purchase the complete back file collection here

Editorial 

Graphic Reporting of Outcomes of Refractive Surgery

Dan Z. Reinstein, MD, MA(Cantab), FRCSC, FRCOphth; George O. Waring, MD, FACS, FRCOphth

Abstract

From London Vision Clinic, London, United Kingdom; the Department of Ophthalmology, St Thomas’ Hospital – Kings College, London, United Kingdom; the Department of Weill Medical College of Cornell University, New York, NY; Centre Hospitalier National d’Ophtalmologie, Paris, France (Reinstein); and InView, Atlanta, Ga (Waring).

A simple internet search using search terms such as “refractive surgery outcomes analysis software” will readily provide links to current commercially available software packages.

Correspondence: Dan Z. Reinstein, MD, London Vision Clinic, 138 Harley St, London W1G 7LA, United Kingdom. Tel: 44 207 224 1005; Fax: 44 207 224 1055; E-mail: dzr@londonvisionclinic.com

In a mature medical specialty such as refractive surgery, the value of publishing the outcomes of a particular surgical technique, study, or case series is maximized if these outcomes are directly comparable to other publications. This can best be achieved by peer-reviewed journals adhering to a universal standard or format for reporting outcomes. Such standardization pervades scientific manuscript publishing: structured abstracts, four component text categories (introduction, methods, results, conclusions), reference formats, table layouts and headings, figure legends—all allowing a reader to more quickly assimilate an article.

In 2000, George O. Waring IIII MD, in association with the editorial staffs of the Journal of Refractive Surgery and the Journal of Cataract and Refractive Surgery, published an article titled “Standard Graphs for Reporting Refractive Surgery,”1 which set out a concise six-graph format for reporting clinical outcomes, covering the four main areas of accuracy, efficacy, safety, and stability. In 2006, Eydelman et al in the Astigmatism Project Group, American National Standards Institute (ANSI) Z80.11 Working Group on Laser Systems for Corneal Reshaping described a standard for reporting cylinder vector analysis,2 which is also an essential part of reporting refractive surgery outcomes.

The Standard Graphs for Reporting Refractive Surgery1 are instantly recognizable, help the reader quickly find some basic information for which he/she is looking for, and are easier to interpret visually than numerical details in the tables or text. Without this protocol, comparing outcomes among studies becomes more difficult and time consuming. Authors who choose to present graphic data in their own unique format, make it more challenging, impractical, less accurate, or even impossible to use their study for comparison with others.

Some examples of inferior quality reporting include:Accuracy of the refractive outcome is sometimes shown only as a cumulative percentage (eg, the number of eyes with a spherical equivalent refractive outcome within ±0.50 diopters [D]), whereas the results should indicate the percentage of over- and undercorrections as set out in the Standard Graphs for Reporting Refractive Surgery.1Some authors report visual acuity as a mean and standard deviation logMAR values; however, this method can hide outliers. Although a mean value can be of some use, it must be supplemented by the efficacy bar chart to show the cumulative percentage of eyes with uncorrected distance visual acuity (UDVA) at each Snellen line of vision as defined in the Standard Graphs for Reporting Refractive Surgery.1 Refractive surgeons are used to thinking about efficacy in terms of the percentage of eyes that can see 20/16 or better, 20/20 or better, etc. The efficacy bar chart reports UDVA results in a transparent manner and enables direct comparison among studies in which the Standard Graphs for Reporting Refractive Surgery have been used. Efficacy of UDVA outcomes is often reported incorrectly; many studies do not present the postoperative UDVA results in the context of preoperative corrected distance visual acuity (CDVA). For example, a series of phakic intraocular lens cases for extreme myopia might have a baseline CDVA of 20/20 or better in only 60% of eyes, which is necessary information to interpret the…

From London Vision Clinic, London, United Kingdom; the Department of Ophthalmology, St Thomas’ Hospital – Kings College, London, United Kingdom; the Department of Weill Medical College of Cornell University, New York, NY; Centre Hospitalier National d’Ophtalmologie, Paris, France (Reinstein); and InView, Atlanta, Ga (Waring).

*:

A simple internet search using search terms such as “refractive surgery outcomes analysis software” will readily provide links to current commercially available software packages.

Correspondence: Dan Z. Reinstein, MD, London Vision Clinic, 138 Harley St, London W1G 7LA, United Kingdom. Tel: 44 207 224 1005; Fax: 44 207 224 1055; E-mail: dzr@londonvisionclinic.com

In a mature medical specialty such as refractive surgery, the value of publishing the outcomes of a particular surgical technique, study, or case series is maximized if these outcomes are directly comparable to other publications. This can best be achieved by peer-reviewed journals adhering to a universal standard or format for reporting outcomes. Such standardization pervades scientific manuscript publishing: structured abstracts, four component text categories (introduction, methods, results, conclusions), reference formats, table layouts and headings, figure legends—all allowing a reader to more quickly assimilate an article.

Current Standards

In 2000, George O. Waring IIII MD, in association with the editorial staffs of the Journal of Refractive Surgery and the Journal of Cataract and Refractive Surgery, published an article titled “Standard Graphs for Reporting Refractive Surgery,”1 which set out a concise six-graph format for reporting clinical outcomes, covering the four main areas of accuracy, efficacy, safety, and stability. In 2006, Eydelman et al in the Astigmatism Project Group, American National Standards Institute (ANSI) Z80.11 Working Group on Laser Systems for Corneal Reshaping described a standard for reporting cylinder vector analysis,2 which is also an essential part of reporting refractive surgery outcomes.

Current Usage of the Standard Graphs

The Standard Graphs for Reporting Refractive Surgery1 are instantly recognizable, help the reader quickly find some basic information for which he/she is looking for, and are easier to interpret visually than numerical details in the tables or text. Without this protocol, comparing outcomes among studies becomes more difficult and time consuming. Authors who choose to present graphic data in their own unique format, make it more challenging, impractical, less accurate, or even impossible to use their study for comparison with others.

Some examples of inferior quality reporting include:

  • Accuracy of the refractive outcome is sometimes shown only as a cumulative percentage (eg, the number of eyes with a spherical equivalent refractive outcome within ±0.50 diopters [D]), whereas the results should indicate the percentage of over- and undercorrections as set out in the Standard Graphs for Reporting Refractive Surgery.1
  • Some authors report visual acuity as a mean and standard deviation logMAR values; however, this method can hide outliers. Although a mean value can be of some use, it must be supplemented by the efficacy bar chart to show the cumulative percentage of eyes with uncorrected distance visual acuity (UDVA) at each Snellen line of vision as defined in the Standard Graphs for Reporting Refractive Surgery.1 Refractive surgeons are used to thinking about efficacy in terms of the percentage of eyes that can see 20/16 or better, 20/20 or better, etc. The efficacy bar chart reports UDVA results in a transparent manner and enables direct comparison among studies in which the Standard Graphs for Reporting Refractive Surgery have been used. Efficacy of UDVA outcomes is often reported incorrectly; many studies do not present the postoperative UDVA results in the context of preoperative corrected distance visual acuity (CDVA). For example, a series of phakic intraocular lens cases for extreme myopia might have a baseline CDVA of 20/20 or better in only 60% of eyes, which is necessary information to interpret the number (%) of eyes with UDVA of 20/20 or better after surgery. It can also be laborious for the reader if the graph displays only the distinct percentage of eyes seeing at a particular level, rather than cumulative percentages.1 This applies also to manuscripts in which UDVA efficacy is reported in the text or table only, without a bar chart.
  • Safety, in terms of the change in Snellen lines of CDVA, is another important statistic that must be presented, in particular the percentage of eyes that lose two or more lines. The loss of one line is within normal biological variability, especially when one considers that this can be the loss of only three letters. Some authors only report the mean logMAR CDVA before and after treatment, which is inappropriate, because the change in CDVA in the worst cases is hidden.

The graphs presented in the Figure are standardized by their format, but not by the amount of refractive error dealt with. For example, the sample scattergram is for mild to moderate myopia, but in a paper reporting phakic intraocular lenses for myopia the x and y axes would be expanded out to approximately 20.00 to 25.00 D of myopia; similary, if a hyperopic series of eyes is reported, clearly the x and y axes would have different values. This principle applies to other graphs as well where some expansion or contraction of the values on the x and y axes may be appropriate. Nevertheless, with similar formats in different articles comparison of outcomes would be much easier for readers.

Standard Graphs for Reporting Refractive Surgery (2009). Linear Regression Analysis and the Coefficient of Determination (R2) Have Been Added to the Attempted vs Achieved Correction Scatterplot, and the Stability Plot Is Represented as a Continuous Timescale. UDVA = Uncorrected Distance Visual Acuity, CDVA = Corrected Visual Acuity [note. A Simple Internet Search Using Search Terms Such as “refractive Surgery Outcomes Analysis Software” Will Readily Provide Links to Current Commercially Available Software Packages.]

Figure. Standard Graphs for Reporting Refractive Surgery (2009). Linear Regression Analysis and the Coefficient of Determination (R2) Have Been Added to the Attempted vs Achieved Correction Scatterplot, and the Stability Plot Is Represented as a Continuous Timescale. UDVA = Uncorrected Distance Visual Acuity, CDVA = Corrected Visual Acuity [note. A Simple Internet Search Using Search Terms Such as “refractive Surgery Outcomes Analysis Software” Will Readily Provide Links to Current Commercially Available Software Packages.]

It is almost a decade since these graphic standards for reporting refractive surgery outcomes were first published; discouragingly, my own survey of the articles published in the Journal of Refractive Surgery in 2008 revealed that only approximately one-third presented these graphs in full (~8/26 articles). This figure is quite low for the journal encouraging the use of these graphs, and is probably even lower in other peer-reviewed ophthalmic journals. The editorial board of the Journal of Refractive Surgery has agreed to impose a stricter policy that requires the use of the Standard Graphs for Reporting Refractive Surgery1 in all studies that are amenable to this format of reporting. As part of this process, we have made some amendments to further improve the scope of this standard protocol.

Additions to the Standard Graphs

Attempted vs Achieved Scatterplot

Within the attempted versus achieved spherical equivalent refraction (SEQ) scatterplot, the following additional parameters will be included.

Linear Regression Analysis. A regression trend line provides a visual indication of the general tendency towards over- or undercorrection and is described mathematically by the linear regression equation (consisting of slope [m] and intercept [c]; achieved SEQ = m attempted SEQ + c). The desired outcome where the achieved SEQ is equal to the attempted SEQ for the entire range of refractions treated is described by the equation y = x, ie, a slope of 1 and intercept of 0. Therefore, the closer the slope and intercept are to these values, the more accurate the results. The slope and intercept are values that can be compared among studies to describe concisely the accuracy of the refractive correction. The value of the slope, in particular, immediately provides useful information; a slope less than 1 indicates that higher refractions tended to undercorrect, whereas a slope greater than 1 indicates that higher refractions tended to overcorrect. Finally, the intercept value provides an indicator as to a generalized over- or undercorrection tendency for the data set as a whole.

Coefficient of Determination (R2). This statistic is calculated as part of the linear regression analysis and denotes the strength of the correlation between the attempted and achieved SEQ change. Statistically, the co-efficient of determination represents the proportion (or percentage) of the total variation in the achieved SEQ change that can be explained by the regression equation. In the context of refractive surgery outcomes, it isolates the scatter in the results and represents the best potential accuracy assuming a perfect nomogram. The coefficient of determination can be any value between 0 and 1, with a stronger correlation (ie, less scatter) described by values closer to 1. Comparison of the coefficient of determination among studies provides a simple but effective way of comparing the scatter of results even when comparing studies in which the overall nomogram was not perfect in one or more studies being compared.

Accuracy Histogram

In the original publication of the Standard Graphs for Reporting Refractive Surgery,1 the histogram “bins” are displayed as “−0.50 to 0.00” and “+0.1 to +0.50.” However, the problem with this setup is that eyes that are plano postoperatively fall into the “−0.50 to 0.00” bin. Therefore, if the achieved spherical equivalent refraction change is the same as the attempted change in a large percentage of patients, the accuracy histogram appears skewed to the left. This would imply a trend of undercorrection even though most of the eyes were actually exactly on target. To solve this problem, it seems to make sense to include a central group “−0.13 to +0.13,” which would include all eyes where the achieved sphere is 100% accurate and the cylinder is corrected to within 0.25 D of the target (manifest refraction usually is measured to the nearest 0.25 D for sphere and cylinder, therefore, spherical equivalent is measured to the nearest 0.125 D). This central bin then represents the percentage of eyes with an ideal refractive result.

Stability Plot

Within the stability plot, the x-axis should be represented as a continuous timescale, rather than as regular discrete bins. Plotting the data according to a timescale helps the reader interpret and evaluate the refractive stability as the change in refraction is displayed in the context of the different time points reported. The common follow-up routine is to see the patient at 1 day, 2 weeks, 3 months, 6 months, and 12 months. Therefore, the gap between the time points gets larger as the time after surgery increases, something which is not obvious if discrete bins are used for each time point. In corneal refractive surgery, it is common for the refraction to initially overcorrect before regressing in the early postoperative period and stabilizing thereafter. If regular discrete bins are used—such as 1 day, 2 weeks, and 3, 6, and 12 months—this effect can appear to occur over a longer period because the data are equally spaced whereas the time points are actually closer together in the early postoperative period. Representing the x-axis as a timescale allows an instant visual impression of the refractive stability over time. An example of the refractive stability after LASIK in a myopic population is shown in the Figure.

The Figure demonstrates the Standard Graphs for Reporting Refractive Surgery (2009) for a sample population, including these amendments. The standard graphs are published in virtually every issue of the Journal of Refractive Surgery and can be found online at www.journalofrefractivesurgery.com/PDFs/graphs.pdf. These graphs can be created easily using standard spreadsheet software programs such as Microsoft Excel (Microsoft Corp, Redmond, Wash) as well as other software programs designed specifically for refractive surgery outcomes analysis. The graphs shown in the Figure herein were generated using Microsoft Excel 2003, based on the 1-year follow-up data from a consecutive series of 232 eyes treated for myopia up to −8.50 D by a single surgeon (D.Z.R.) with the MEL80 excimer laser (Carl Zeiss Meditec, Jena, Germany) and VisuMax femtosecond laser system (Carl Zeiss Meditec) (personal communication, Dan Z. Reinstein, MD, 2009). This spreadsheet is offered for free download from www.londonvisionclinic.com/refractivesurgeryoutcomes.

It is the hope of the editors of the Journal of Refractive Surgery that other ophthalmic journals reporting refractive surgery outcomes will vigorously encourage their authors to include these six standard graphs in the presentation of their results.

References

  1. Waring GO III, . Standard graphs for reporting refractive surgery. J Refract Surg. 2000;16:459–466. Erratum in: J Refract Surg. 2001;17:294.
  2. Eydelman MB, Drum B, Holladay J, Hilmantel G, Kezirian G, Durrie D, Stulting RD, Sanders D, Wong B. Standardized analyses of correction of astigmatism by laser systems that reshape the cornea. J Refract Surg. 2006;22:81–95.
Authors

From London Vision Clinic, London, United Kingdom; the Department of Ophthalmology, St Thomas’ Hospital – Kings College, London, United Kingdom; the Department of Weill Medical College of Cornell University, New York, NY; Centre Hospitalier National d’Ophtalmologie, Paris, France (Reinstein); and InView, Atlanta, Ga (Waring).

Correspondence: Dan Z. Reinstein, MD, London Vision Clinic, 138 Harley St, London W1G 7LA, United Kingdom. Tel: 44 207 224 1005; Fax: 44 207 224 1055; E-mail: dzr@londonvisionclinic.com

10.3928/1081597X-20091016-01

Sign up to receive

Journal E-contents