Athletic Training and Sports Health Care

Original Research 

Consistency and Agreement of Two Devices for Running Speed

David Hovsepian, DPT; Stacey A. Meardon, PT, PhD, ATC, CSCS; Thomas W. Kernozek, PhD, FACSM

Abstract

Consistency and agreement of the Garmin Forerunner 305 (device A) and Polar RS800cx (device B) for overground running speed was investigated. Participants ran on a 200-m track. Photoelectric timing lights were used to calculate the criterion speeds for 3 conditions: 6-m straight, 6-m curved, and 1 lap. Consistency within device and level of agreement of each device with photoelectric timing lights was assessed with intraclass correlations (ICC Model 2). For consistency, device A demonstrated ICC values of 0.84 to 0.99 and device B demonstrated ICC values of 0.99 to 1.0. Device B had better levels of agreement than device A with the timing lights (ICC = 0.98–0.99 versus ICC = 0.87–0.96). Differences in speed relative to the timing lights ranged from <0.01 to 0.08 m/s for device B and −0.03 to −0.25 m/s for device A. Device B may be preferred when accuracy is needed over both short and long running distances. [Athletic Training & Sports Health Care. 2014;6(2):67–72.]

Dr Hovsepian and Dr Kernozek are from the Department of Health Professions, Physical Therapy Program, and the La Crosse Institute for Movement Science, University of Wisconsin-La Crosse, La Crosse, Wisconsin; and Dr Meardon is from East Carolina University, Physical Therapy Department, Greenville, North Carolina.

The authors have disclosed no potential conflicts of interest, financial or otherwise.

The authors thank the University of Wisconsin-La Crosse Graduate Studies for their financial support of this project in the form of a funded Research, Service and Educational Leadership Grant to Dr Hovsepian.

Address correspondence to Thomas W. Kernozek, PhD, FACSM, University of Wisconsin-La Crosse, Department of Health Professions, Physical Therapy Program, Health Science Center, 1300 Badger Street, La Crosse, WI 54601; e-mail: kernozek.thom@uwlax.edu.

Received: July 30, 2013
Accepted: December 11, 2013
Posted Online: March 13, 2014

Abstract

Consistency and agreement of the Garmin Forerunner 305 (device A) and Polar RS800cx (device B) for overground running speed was investigated. Participants ran on a 200-m track. Photoelectric timing lights were used to calculate the criterion speeds for 3 conditions: 6-m straight, 6-m curved, and 1 lap. Consistency within device and level of agreement of each device with photoelectric timing lights was assessed with intraclass correlations (ICC Model 2). For consistency, device A demonstrated ICC values of 0.84 to 0.99 and device B demonstrated ICC values of 0.99 to 1.0. Device B had better levels of agreement than device A with the timing lights (ICC = 0.98–0.99 versus ICC = 0.87–0.96). Differences in speed relative to the timing lights ranged from <0.01 to 0.08 m/s for device B and −0.03 to −0.25 m/s for device A. Device B may be preferred when accuracy is needed over both short and long running distances. [Athletic Training & Sports Health Care. 2014;6(2):67–72.]

Dr Hovsepian and Dr Kernozek are from the Department of Health Professions, Physical Therapy Program, and the La Crosse Institute for Movement Science, University of Wisconsin-La Crosse, La Crosse, Wisconsin; and Dr Meardon is from East Carolina University, Physical Therapy Department, Greenville, North Carolina.

The authors have disclosed no potential conflicts of interest, financial or otherwise.

The authors thank the University of Wisconsin-La Crosse Graduate Studies for their financial support of this project in the form of a funded Research, Service and Educational Leadership Grant to Dr Hovsepian.

Address correspondence to Thomas W. Kernozek, PhD, FACSM, University of Wisconsin-La Crosse, Department of Health Professions, Physical Therapy Program, Health Science Center, 1300 Badger Street, La Crosse, WI 54601; e-mail: kernozek.thom@uwlax.edu.

Received: July 30, 2013
Accepted: December 11, 2013
Posted Online: March 13, 2014

Studying movement characteristics outside the laboratory in a more natural running environment has the potential to advance our understanding of running injury and the efficacy of rehabilitation. Portable instrumentation, such as accelerometers, can be used to measure loads and movement patterns in the real world. Accelerometer-based variables associated with running injury include peak impacts,1,2 shock attenuation,3,4 and stride time variability.5 All these variables can be measured outside the laboratory using accelerometers and are speed dependent.6,7 Thus, gait speed must be held relatively constant from stride to stride, or at least accurately monitored, to adequately describe the potential influence of injury or intervention on these movement characteristics. This is particularly challenging when running on ground, which requires the use of continual feedback on pace with a portable device that is both reliable and valid over short and long distances.

Portable accelerometry-based systems have been developed to assess locomotor speed.8,9 Some of these lightweight devices are housed within a wristband computer system and a footpod, requiring both a wristwatch and footpod attached to the shoe. However, knowledge related to the validity of such devices is limited. One such device, the Polar RS800sd with S3 footpod (Polar Electro Inc, Lake Success, New York), has been found to be valid for running speeds ranging from 3.33 to 5.0 m/s on a treadmill.8 However, this device has not been tested during overground running. Overground running conditions have the potential to alter stride characteristics, introducing greater performance variability10; thus, the validity of these accelerometry-based devices during overground running needs to be assessed for use in investigations of injury or intervention on movement characteristics.

The purpose of this study was to assess the level of agreement of 2 commercially available running computers: the Garmin Forerunner 305 with footpod (device A; Garmin International Inc, Olathe, Kansas) and the Polar RS800cx with S3 footpod (device B; Polar Electro Inc), compared with a criterion measure obtained from photoelectric timing lights during overground running. It was hypothesized that 1) both wristband computer systems would be consistent and demonstrate good agreement with timing lights in measuring running speed indoors for both short and long distances, and 2) the level of agreement would be influenced by the curved portions of the track.

Method

University institutional review board approval was obtained for this study, and all participants provided written informed consent prior to participation. Thirteen runners (mean age = 25.3 ± 2.5 years, mean weight = 65.5 ± 9.4 kg, and mean height = 177 ± 8.1 cm) free from injury participated in this study. All participants were older than 18 years and self-reported running an average of at least 10 miles per week during the past year.

Participants were fitted with both portable measuring devices (device A and device B); wrist watches were worn on one arm and footpods were placed closely together on the dorsum of the right shoe per manufacturer recommendations. Footpods were calibrated per manufacturer instructions for each participant prior to the study by having the participants run a known distance (400 m) at their preferred pace. The output sampling rate of both devices was 1 Hz.

Running speed was measured over 6 m on the straight and curved portions of a 200-m indoor track with photoelectric timing lights for device A and device B. In addition, speed over 200 m was recorded. The 6-m distance was chosen to capture the accuracy of devices over a short distance, and the 200-m distance was chosen to assess these data over longer distances. Running speed on the curved portion of the track was examined to better understand the effects of curvature on the validity of running speed reported by portable devices. To obtain 6-m gait speed, photoelectric timing lights were placed 6 m apart on both the straight (6-m straight) and curved (6-m curved) portions of the track. For each condition (6-m straight, 6-m curved, and lap), participants ran 7 laps at their preferred 5k pace. Time to traverse the 6-m straight, 6-m curved, and 200-m conditions were recorded from the timing lights. A minimum 5-minute rest period between conditions was provided to minimize fatigue. Condition order was randomized to ensure that fatigue did not unduly influence the study outcomes.

Prior to each condition, the running computers and a stopwatch were manually synchronized with another stopwatch so that the stopwatch and both of the running computers were turned on simultaneously. After 2 minutes, participants were given a signal to begin running. Times at which participants crossed the timing lights were recorded from the stopwatch to timestamp the data for later analysis.

Using the respective manufacturer’s software, data from the wristband computers were exported for analysis and timestamped using the recorded times at which participants passed through the timing light areas of the track. For each trial, device data were averaged over the time window associated with crossing the timing lights. Intraclass correlation coefficients (ICC Model 2) were performed in the SPSS statistical package, version 20 (IBM, Aramonk, New York) to determine the consistency for each device across 7 laps. In addition, absolute agreement between each device and the criterion measure was determined using ICC Model 2. Pearson Bland-Altman plots were created, using the MedCalc version 12.7.0.0 (MedCalc Software, Ostend, Belgium) to visually analyze the relationship between the observed differences and the magnitude of measurement, as well as to identify any systematic bias.11,12 The 95% confidence interval (CI) of the mean difference was used to demonstrate the magnitude of systematic differences. A significant systematic difference occurred when zero (ie, no difference) fell beyond the reported 95% CI of the mean difference between the timing lights and device.

Results

Participants ran at their preferred 5k training pace, ranging between 2.9 to 5.1 m/s (average pace = 3.49 ± 0.53 m/s). This pace represents the range of paces often used by recreational runners. Level of consistency within each device ranged from 0.84 to 1.00 (Table 1). Both devices demonstrated excellent levels of agreement with the photoelectric timing lights across all 3 conditions, with ICC values ranging from 0.87 to 0.99 (Table 1). For device A, the lowest levels of consistency and agreement were observed in the 6-m straight condition (ICC = 0.84 and 0.87). Device B was equally consistent across all conditions (ICC = 0.99 to 1.00) and demonstrated the lowest levels of agreement with the timing lights in the 6-m curved condition (ICC = 0.98).

Intraclass Correlation Values for Reliability and Validity, With Lower-Bound and Upper-Bound 95% Confidence Intervals

Table 1:

Intraclass Correlation Values for Reliability and Validity, With Lower-Bound and Upper-Bound 95% Confidence Intervals

The Figure illustrates the Bland-Altman plots of the differences in running speed between each device and the timing lights. No clear relationship between the differences and magnitude of speed was observed. Analysis of the graphs for bias reveals a tendency for device A to underestimate running speed and for device B to overestimate running speed in the 6-m conditions. Both devices tended to slightly underestimate speed in the 200-m condition. Limits of agreement for device A in the straight, curved, and lap condition were −0.80 to 0.29 m/s, −0.68 to 0.46 m/s, and −0.42 to 0.36 m/s, respectively. For device B, limits of agreement were −0.14 to 0.14 m/s, −0.16 to 0.32 m/s and −0.16 to 0.11 m/s for the straight, curved, and lap conditions, respectively.

The Bland-Altman plots above illustrate the differences between the devices and the timing lights plotted against the averages of running speed obtained from the timing lights and the respective device. Horizontal lines are drawn at the mean differences (solid line) and at the upper limits and lower limits of agreement (dashed lines), which are defined as the mean difference plus and minus 1.96 times the standard deviation of the differences. 95% confidence intervals of the average difference are represented by the dash dot lines.

Figure.

The Bland-Altman plots above illustrate the differences between the devices and the timing lights plotted against the averages of running speed obtained from the timing lights and the respective device. Horizontal lines are drawn at the mean differences (solid line) and at the upper limits and lower limits of agreement (dashed lines), which are defined as the mean difference plus and minus 1.96 times the standard deviation of the differences. 95% confidence intervals of the average difference are represented by the dash dot lines.

Mean differences and the 95% CI of the differences are listed in Table 2 and illustrated in the Bland-Altman plots. A mean difference of zero fell outside the 95% CI of the observed mean difference between the timing lights and device A in the 6-m straight condition and between the timing lights and device B in the 6-m curved condition. This indicates that a significant systematic difference from the timing lights was observed for device A in the 6-m straight condition and device B in the 6-m curved condition.

Mean Differences and 95% Confidence Intervals Obtained From Analysis of Bland-Altman Plotsa

Table 2:

Mean Differences and 95% Confidence Intervals Obtained From Analysis of Bland-Altman Plots

Discussion

The purpose of this study was to assess the consistency and level of agreement with a criterion measure of 2 accelerometers on the straight and curved portions of an indoor track over long and short distances. It was hypothesized that 1) wristband computer systems would be consistent and demonstrate good agreement with timing lights in measuring running speed indoors for both short and long distances and 2) the level of agreement would be influenced by the curved portions of the track.

In this study, device A was found to be less consistent and less accurate in both the 6-m straight and 6-m curved conditions than device B. However, accuracy relative to the criterion measure of the photoelectric timing lights over longer distances was more comparable. Contrary to our expectation, device A was less consistent and less accurate in the short straight condition, compared with the short curved condition. However, as expected, device B was equally consistent but only slightly less accurate in the curved condition relative to the straight condition.

Device B had been previously validated during treadmill running for speeds ranging from 3.33 to 5.0 m/s for 30-second intervals.8 However, treadmill running differs from overground running due to the altered stride characteristics and reduced variability of lower extremity motions, especially at the ankle, during treadmill running.10 Nonetheless, similar to previous work, device B was found to be consistent and demonstrated good level of agreement with photoelectric timing lights during overground running at recreational running paces. To the authors’ knowledge, device A has not been studied previously. In general, device A was less consistent and less accurate, especially in the short distance conditions.

Device B tended to overestimate running speed (<0.01 to 0.08 m/s), whereas device A tended to underestimate running speed (–0.02 to −0.25 m/s) over short distances. For both devices, differences between the device and the timing lights generally fell within the mean difference ± 1.96 SD, which is within the limits of agreement. The clinical importance of these differences will need to be determined by the researchers and the research question. If gait speed differences over a short distance of 0.25 m/s are a concern, the 2 devices cannot be used interchangeably and device B is therefore recommended.

In comparison with shorter distances, device A performed better over longer distances. The observed bias was small (–0.03 m/s), but limits of agreement were broad. Given the limits of agreement shown in the Figure, a person running at 3.7 m/s could actually be running between 3.28 and 4.06 m/s. The performance of device B over 200 m was comparable with 6 m and in general demonstrated more narrow limits of agreement when compared with device A. Using the limits of agreement from the Figure, a person running at 3.7 m/s could actually be running between 3.54 and 3.81 m/s. Whether these differences are important depends on the analytical goals of the researchers or individuals using the device. A commonly reported running pace for running injury–related studies is 3.7 m/s ± 5%, equating to a velocity range of 3.53 to 3.89 m/s. Based on this, device B may be more suitable for research purposes.

Overground running often involves the use of both straight and curved paths, whereas treadmill running does not account for curved running conditions where there is an increase in foot contact time.13 Greater contact time is related to the need to generate large braking forces when running the turns.14 These large changes in ground force likely require a coordinated biomechanical response of the lower extremity to restore runner stability. Therefore, it is possible that gait characteristics may be more variable when running on curved paths, affecting the overall consistency and accuracy of accelerometry-based measurements of speed. In comparison with the straight condition, the Bland-Altman plots suggest that device B displayed a systematically greater bias in the curved condition, which is consistent with our hypothesis. It is possible that device B was more sensitive to the greater contact times associated with running the curves previously reported in the literature. For unknown reasons, device A was more accurate in the curved condition relative to the straight condition, suggesting that speed detection was less sensitive to a suspected change in contact times. It is important to note that magnitude of running speed has the potential to influence the accuracy of devices on the curved portion of a track. However, the Bland-Altman plots from this study do not reveal a consistent relationship that would support the existence of a relationship between measurement accuracy and running speed. The running speeds assessed in this study tended to be clustered around 3.5 m/s. It is possible that a greater influence of track curvature on measurement accuracy would be observed at faster running speeds.

Differences between the devices and criterion measure may be explained by technology used in these devices. Although neither device details the formulae used in its calculations of running speed, the accuracy of device B may be attributed to its horizontal axis.8 Hausswirth et al8 described the S3 footpod as a uniaxial accelerometer with an anterior–posterior axis. It seems reasonable to suggest, given our findings, that the weighting of the anterior–posterior output when analyzing foot mounted accelerometer data should be considered when assessing overground running.

In the current study, the same researcher (D.H.) placed the footpod on all participants. Nonetheless, as Wang et al15 argued, accelerometer placement may be susceptible to artifact from normal movement. Given that a greater level of foot motion is utilized during running, waist-mounted accelerometers may be an alternative to consider in the future. Various types of triaxial waist-mounted accelerometers have been successfully used to assess running biomechanics away from laboratory setting.16–19 However, a study of the accuracy of waist-mounted triaxial accelerometers used to assess overall running speed is not yet available.

Implications for Clinical Practice

Despite the observed bias in speed measurement for both devices, our results suggest that accuracy of speed measures on curves may be device dependent, with device A being slightly less accurate. Device B was found to be more consistent and demonstrated better levels of agreement under all conditions (straight, curved, and lap). Device A demonstrated less bias over longer distances (200 m), but showed greater error for shorter (6 m) distances. Thus, device B may be more suitable when accuracy is needed, especially over short distances. The use of commercially available accelerometer-based running computers on indoor running tracks appears to be a viable low-cost and convenient option to assess running speed. Use of such devices may open up research opportunities associated with the study of running in a more ecologically valid environment.

References

  1. Butler RJ, Hamill J, Davis I. Effect of footwear on high and low arched runners’ mechanics during a prolonged run. Gait Posture. 2007;26(2):219–225. doi:10.1016/j.gaitpost.2006.09.015 [CrossRef]
  2. Milner CE, Hamill J, Davis I. Are knee mechanics during early stance related to tibial stress fracture in runners?Clin Biomech. 2007;22(6):697–703. doi:10.1016/j.clinbiomech.2007.03.003 [CrossRef]
  3. Voloshin A, Wosk J. An in vivo study of low back pain and shock absorption in the human locomotor system. J Biomech. 1982;15(1):21–27. doi:10.1016/0021-9290(82)90031-8 [CrossRef]
  4. Voloshin AS, Mizrahi J, Verbitsky O, Isakov E. Dynamic loading on the human musculoskeletal system—effect of fatigue. Clin Biomech. 1998;13(7):515–520. doi:10.1016/S0268-0033(98)00030-8 [CrossRef]
  5. Meardon SA, Hamill J, Derrick TR. Running injury and stride time variability over a prolonged run. Gait Posture. 2011;33(1):36–40. doi:10.1016/j.gaitpost.2010.09.020 [CrossRef]
  6. Mercer JA, Vance J, Hreljac A, Hamill J. Relationship between shock attenuation and stride length during running at different velocities. Eur J Appl Phys. 2002;87(4–5):403–408. doi:10.1007/s00421-002-0646-9 [CrossRef]
  7. Jordan K, Challis JH, Newell KM. Speed influences on the scaling behavior of gait cycle fluctuations during treadmill running. Human Movement Sci. 2007;26(1):87–102. doi:10.1016/j.humov.2006.10.001 [CrossRef]
  8. Hausswirth C, Le Meur Y, Couturier A, Bernard T, Brisswalter J. Accuracy and repeatability of the Polar RS800sd to evaluate stride rate and running speed. Int J Sports Med. 2009;30(5):354–359. doi:10.1055/s-0028-1105936 [CrossRef]
  9. Yang S, Mohr C, Li Q. Ambulatory running speed estimation using an inertial sensor. Gait Posture. 2011;34(4):462–466. doi:10.1016/j.gaitpost.2011.06.019 [CrossRef]
  10. Dingwell JB, Cusumano JP, Cavanagh PR, Sternad D. Local dynamic stability versus kinematic variability of continuous overground and treadmill walking. J Biomech Eng. 2001;123(1):27–32. doi:10.1115/1.1336798 [CrossRef]
  11. Bland JM, Altman DG. Comparing methods of measurement: why plotting difference against standard method is misleading. Lancet. 1995;346(8982):1085–1087. doi:10.1016/S0140-6736(95)91748-9 [CrossRef]
  12. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1(8476):307–310. doi:10.1016/S0140-6736(86)90837-8 [CrossRef]
  13. Greene PR. Running on flat turns: experiments, theory, and applications. J Biomech Eng. 1985;107(2):96–103. doi:10.1115/1.3138542 [CrossRef]
  14. Jindrich DL, Qiao M. Maneuvers during legged locomotion. Chaos. 2009;19(2):026105. doi:10.1063/1.3143031 [CrossRef]
  15. Wang N, Redmond SJ, Ambikairajah E, Celler BG, Lovell NH. Can triaxial accelerometry accurately recognize inclined walking terrains?IEEE Trans Biomed Eng. 2010;57(10):2506–2516. doi:10.1109/TBME.2010.2049357 [CrossRef]
  16. Kawabata M, Goto K, Fukusaki C, et al. Acceleration patterns in the lower and upper trunk during running. J Sports Sci. 2013;31(16):1841–1853. doi:10.1080/02640414.2013.805884 [CrossRef]
  17. Wixted AJ, Billing DC, James DA. Validation of trunk mounted inertial sensors for analysing running biomechanics under field conditions, using synchronously collected foot contact data. Sports Eng. 2010;12(4):207–212. doi:10.1007/s12283-010-0043-2 [CrossRef]
  18. Lafortune MA. Three-dimensional acceleration of the tibia during walking and running. J Biomech. 1991;24(10):877–879, 881–886. doi:10.1016/0021-9290(91)90166-K [CrossRef]
  19. Bergamini E, Picerno P, Pillet H, Natta F, Thoreux P, Camomilla V. Estimation of temporal parameters during sprint running using a trunk-mounted inertial measurement unit. J Biomech. 2012;45(6):1123–1126. doi:10.1016/j.jbiomech.2011.12.020 [CrossRef]

Intraclass Correlation Values for Reliability and Validity, With Lower-Bound and Upper-Bound 95% Confidence Intervals

CONSISTENCY WITHIN DEVICE
6-m STRAIGHT6-m CURVELAP

Timing lights0.98 (0.96–0.99)0.99 (0.99–1.00)0.99 (0.99–1.00)
Device A0.84 (0.63–0.96)0.87 (0.70–0.96)0.99 (0.98–1.00)
Device B0.99 (0.97–1.00)1.00 (0.99–1.00)0.99 (0.98–1.00)
LEVEL OF AGREEMENT WITH PHOTOELECTRIC TIMING LIGHTS
6-m STRAIGHT6-m CURVELAP

Device A0.87 (0.27–0.97)a0.94 (0.81–0.98)b0.96 (0.86–0.98)c
Device B0.99 (0.98–1.00)d0.98 (0.94–1.00)e0.99 (0.98–1.00)f

Mean Differences and 95% Confidence Intervals Obtained From Analysis of Bland-Altman Plotsa

DIFFERENCE FROM PHOTOELECTRIC TIMING LIGHTS
6-m STRAIGHT6-m CURVELAP
Device A−0.25 (−0.43 to −0.08)a−0.11 (−0.30 to 0.08)−0.03 (−0.15 to 0.10)
Device B<0.01 (–0.04 to 0.05)0.08 (<0.01 to 0.16)a−0.02 (–0.07 to 0.02)

10.3928/19425864-20140306-02

Sign up to receive

Journal E-contents