Athletic Training and Sports Health Care

Original Research 

Minimal Detectable Change Scores and Reliability of the Balance Error Scoring System in Student-Athletes With Acute Concussion

Casey D. Carlson, MS, ATC; Jody L. Langdon, PhD; Barry A. Munkasy, PhD; Kelsey M. Evans, MS; Thomas A. Buckley, EdD, ATC

Abstract

Purpose:

To investigate the inter-rater and intra-rater reliability and calculate the Balance Error Scoring System (BESS) minimal detectable change among certified athletic trainers in a population with acute concussion.

Methods:

Sixteen certified athletic trainers scored BESS videos of student-athletes with acute concussion on two occasions 1 week apart. The inter-rater and intra-rater reliability values and minimal detectable change scores were calculated for the two viewings.

Results:

The inter-rater reliability was 0.75 and the intra-rater reliability was 0.86. The inter-rater and intra-rater minimal detectable change scores were 11.3 and 8.6 errors, respectively.

Conclusions:

The reliability values were moderate to good, but the minimal detectable change scores were higher than those previously reported for healthy participants. Importantly, the minimal detectable change scores in the current study exceed the normal change reported for BESS performance (4 to 6 errors) after concussion.

[Athletic Training & Sports Health Care. 2020;12(2):67–73.]

Abstract

Purpose:

To investigate the inter-rater and intra-rater reliability and calculate the Balance Error Scoring System (BESS) minimal detectable change among certified athletic trainers in a population with acute concussion.

Methods:

Sixteen certified athletic trainers scored BESS videos of student-athletes with acute concussion on two occasions 1 week apart. The inter-rater and intra-rater reliability values and minimal detectable change scores were calculated for the two viewings.

Results:

The inter-rater reliability was 0.75 and the intra-rater reliability was 0.86. The inter-rater and intra-rater minimal detectable change scores were 11.3 and 8.6 errors, respectively.

Conclusions:

The reliability values were moderate to good, but the minimal detectable change scores were higher than those previously reported for healthy participants. Importantly, the minimal detectable change scores in the current study exceed the normal change reported for BESS performance (4 to 6 errors) after concussion.

[Athletic Training & Sports Health Care. 2020;12(2):67–73.]

Concussions are associated with diverse neurological presentations in deficits in cognition, balance, autonomics, oculomotor function, psychological changes, and increased self-reported symptomology.1–7 When addressing the range of neurological deficits during the past two decades, the International Conference on Concussion in Sport Position Statement has consistently recommended a multifaceted assessment battery.1,8 A baseline examination should occur prior to the competitive athletic season, and the individual with concussion should achieve preinjury scores before initiating a physically exertional progression culminating in return to play.1 Impaired motor or postural control is a common sign of concussion, with most individuals self-reporting balance impairments or dizziness after acute concussion.1,9,10

The most commonly used and recommended postural control assessment after concussion is the Balance Error Scoring System (BESS), likely due to its low cost and clinical feasibility, despite its low sensitivity (0.34 to 0.44) after acute injury.1,11–17 The BESS includes three stances (double limb, single limb, and tandem) on two surfaces (firm and foam) and is scored by the clinician counting the number of errors (ie, deviations from the test position) during each stance.1,18 The BESS has multiple administration- and measurement-related limitations, including the practice effect whereby healthy participants will improve their performance by 1 to 2 errors with repeat administration.19–23 The BESS can also be negatively influenced by the test environment, participant fatigue, and lower extremity injury.21–25 The reliability of individual BESS stances and total score ranged from poor (0.41) to high (0.98).18,19 Unlike objectively measured neurological assessments, the BESS errors are subjectively identified (eg, hip moving more than 30° or staying out of the test position for 5 seconds), which raises additional measurement concerns.18

Typically, collegiate student-athletes commit 4 to 6 more errors immediately after concussion than at baseline, which reduces to 2 to 3 errors 24 hours after injury.11,12 Given the described measurement limitations of the BESS, the statistical and clinical meaningfulness of this change must be considered. The minimal detectable change score is the smallest amount of change that likely reflects true change, as opposed to measurement error (ie, variability in clinician scoring) inherent in the score.26,27 The minimal detectable change can assist clinicians with identifying if the change in BESS score reflects a real and reliable change.26 In terms of BESS testing after a suspected concussion, the minimal detectable change would reflect the variability in the clinician's scoring of the BESS test but not the injured patient's actual performance.

The minimal detectable change is important to clinicians because it assesses the patient's reliability and responsiveness to change. This helps identify how real and reliable the change is, and determine if the assessment itself (ie, BESS) can detect the change.26 In healthy student-athletes, the BESS minimal detectable change ranged from 7.3 (intra-rater) to 9.4 (inter-rater) errors, which exceeded the expected change of 4 to 6 errors after concussion. Thus, an increase of 7 errors after concussion may reflect scorer variability but not an actual change in task performance.11,12,27 In this sense, the minimal detectable change refers to the scorer variability or how much the clinician's score varies when evaluating the identical performance twice. Both intra-rater and inter-rater minimal detectable change outcomes are important because it is commonplace for different athletic trainers to evaluate assessments at baseline and after injury.14,15 Understanding the BESS minimal detectable change values may help clinicians differentiate between true changes in test performance and measurement errors, which should improve the clinical care of athletes with suspected concussions.

The BESS is the most commonly used concussion balance assessment despite test administration and scoring limitations; however, the measurement error associated with clinician scoring individuals after acute injury has not been established. Therefore, the purpose of this study was to use certified athletic trainers to investigate BESS reliability and minimal detectable change in athletes with acute concussion. We hypothesized that the reliability would be adequate (interclass correlation coefficients [ICCs] > 0.60),28 and that the minimal detectable change values for student-athletes with acute concussion would exceed the values for healthy individuals.

Methods

Participants

The participants were 16 clinically practicing athletic trainers (13 women and 3 men with 1 to 10 years of clinical practice) who worked in intercollegiate athletic settings across diverse collision, contact, and non-contact sports. The participants reported receiving formal education on the BESS as undergraduate/graduate students or from professional continuing education courses. All participants graduated from accredited athletic training education programs (ie, Commission on Accreditation of Athletic Training Education/Commission on Accreditation of Allied Health Educational Programs). The participants self-reported performing 8.6 ± 9.9 (range: 1 to 30) BESS tests of individuals after acute concussion in the previous year, which exceeded the testing experience of the previous BESS minimal detectable change publication.27

The individuals with acute concussion were not the actual participants in the study, but they were National Collegiate Athletic Association (NCAA) Division I student-athletes enrolled in a separate parent research study. The student-athletes (5 men, 5 women; age = 19.7 ± 1.4 years; height = 171.7 ± 11.1 cm; weight = 74.7 ± 18.7 kg; 1.4 ± 1.0 prior concussions, 0% loss of consciousness rate, 20% post-traumatic amnesia rate) participated in football (n = 5), cheerleading (n = 4), and women's soccer (n = 1). Initially, concussions were identified by an athletic trainer and a licensed physician confirmed the diagnosis based on guidelines from the 4th Consensus Statement on Concussion in Sport (CIS) because the current study was conducted prior to the release of the 5th edition.29 All participants (certified athletic trainers and student-athletes with acute concussion) provided written informed consent to participate in the current study, which was approved by Georgia Southern University's institutional review board.

Procedures

Consistent with current best practices, student-athletes with acute concussion performed baseline BESS tests prior to athletic participation and were reassessed within 24 hours after concussion in the same quiet and private location.1,13 All six stances of the BESS were performed and scored according to standard Sport Concussion Assessment Tool-3 guidelines.29 All student-athletes with acute concussion completed the assessment without difficulty. The student-athletes' BESS tests were recorded by two video cameras (HF M31; Canon, Tokyo, Japan) that were placed orthogonally 5 meters from the participants in the frontal and sagittal planes. The edited videos showed 20 seconds of each stance with audio/visual indications of the initiation and termination of each trial.

Similar to the previous protocol, athletic trainers scored the 10 videos on two occasions 1 week apart in a quiet laboratory setting, seated approximately 1.5 meters from the 42-inch television, and connected to a laptop with an HDMI cable.27 Because the purpose of this study was to evaluate minimal detectable change after acute concussion, only the videos after concussion were evaluated by the participants, which occurred on two separate occasions (time 1 [T1] and time 2 [T2]) 1 week apart at a similar time of day. Prior to each scoring session, participants watched a BESS scoring procedure video that reviewed scoring guidelines based on the 4th Consensus Statement on CIS.29 Each trial began with the participant recording the identification number of the student-athlete with acute concussion and then proceeded to score each BESS stance. At the conclusion of a completed BESS trial, the video was paused and the score sheet was collected and stored by a member of the research team. A new score sheet was used for each BESS trial, and the participants did not have access to their prior scores at any time. The second scoring session (T2) followed identical procedures, except the 10 videos were displayed in a different order from the first session (T1) with different identification numbers for the student-athletes with acute concussion.27

Statistical Analysis

The BESS total score, the sum of the six stances (total number of observed errors committed during the tasks), was recorded for each test session after concussion and, consistent with clinical practice, was the dependent variable of interest. Inter-rater and intra-rater reliability for the BESS total score was calculated via ICCs for the videos of individuals with acute concussion because the BESS minimal detectable change of the healthy participants had already been established.19,27,30–33 The minimal detectable change was calculated as equal to the standard error of measurement × √2 × 1.96, using the standard error of measurement between T1 and T2. The standard error of measurement is the standard error of the mean and 1.96 reflects the 95% confidence interval, which means that 95% of stable participants would demonstrate a random variation less than this amount when tested on two separate occasions.26,27 The comparison of BESS scores between T1 and T2 was assessed with the Wilcoxon ranked-sum test, due to the assumption of normality being violated, and the alpha level set to 0.05. Although not compared as a component of this study, the treating athletic trainers score is presented as descriptive data only.

Results

All participants (the clinical athletic trainers) scored all 10 trials on both T1 and T2. There were no outliers (> 3 standard deviations from the mean),34 so all 160 trials were included in the analysis. The inter-rater (0.75) and intra-rater (0.86) reliabilities were moderate to good. The inter-rater and intra-rater minimal detectable change values were 11.3 and 8.6 errors, respectively (Figure 1).

Balance Error Scoring System (BESS) change scores relative to minimal detectable change (MDC). The video scores (mean ± standard deviation of time 1 [T1] and time 2 [T2]) of the 10 reviewed videos of individuals with concussion. The y-axis depicts the change in the number of errors in BESS between baseline (T1) and after acute concussion (T2) relative to the intra-rater and inter-rater minimal detectable change values. A “0” represents no change in errors between baseline and after concussion, and a positive number represents increased errors (worse performance). The video-based score failed to exceed the minimal detectable change in 8 of 10 inter-rater and 7 of 10 intra-rater assessments.

Figure 1.

Balance Error Scoring System (BESS) change scores relative to minimal detectable change (MDC). The video scores (mean ± standard deviation of time 1 [T1] and time 2 [T2]) of the 10 reviewed videos of individuals with concussion. The y-axis depicts the change in the number of errors in BESS between baseline (T1) and after acute concussion (T2) relative to the intra-rater and inter-rater minimal detectable change values. A “0” represents no change in errors between baseline and after concussion, and a positive number represents increased errors (worse performance). The video-based score failed to exceed the minimal detectable change in 8 of 10 inter-rater and 7 of 10 intra-rater assessments.

There were no significant differences in the participants' video-based total BESS scores between T1 and T2: 17.7 ± 8.2 and 17.4 ± 8.9 errors, respectively (P = .460) (Figure 2). The absolute value of the mean BESS change score was 3.2 ± 3.3, and the overall range was −14 to 14 change in scores. A negative score reflected fewer errors on the T2 scoring.

Ranked individual score differences by magnitude of change for the 10 videos across the 16 participants. The score was calculated by time 2 (T2) minus time 1 (T1), such that a negative number represents a lower score on T2. Thirty-one of 160 (19.4%) Balance Error Scoring System tests were scored identically between T1 and T2.

Figure 2.

Ranked individual score differences by magnitude of change for the 10 videos across the 16 participants. The score was calculated by time 2 (T2) minus time 1 (T1), such that a negative number represents a lower score on T2. Thirty-one of 160 (19.4%) Balance Error Scoring System tests were scored identically between T1 and T2.

The clinical BESS scores from the certified athletic trainers (scored visually without video review as per standard CIS protocol) found a 4.9 error increase between the two test time points (pre-season/healthy: 12.7 ± 5.2 errors and after acute concussion: 17.6 ± 8.7 errors).

Discussion

The purpose of this study was to assess the minimal detectable change values of the certified athletic trainers who evaluated student-athletes with acute concussion to establish and compare the range of changes associated with athletic trainer measurement error and true change in performance.1,11,13–17,19–25 As expected and consistent with previous findings, there were moderate to good ICC values (0.75 to 086) for the BESS test when scoring student-athletes with acute concussion.28,32,33 However, the primary results in the current study were that the minimal detectable change substantially exceeded the change in the actual observed clinical scoring (intra-rater: 8.6 errors and inter-rater: 11.3 errors), the expected change in BESS reported in the literature (4 to 6 errors), and the reported minimal detectable change scores of healthy athletes (7.3 to 9.4).11,27 More specifically, this means that the same clinician scoring the same BESS test after acute concussion on two separate occasions had a minimal detectable change of 8.6 errors, whereas if two different clinicians scored the same BESS test, the minimal detectable change was 11.3 errors. Considering the established limitations of the BESS and minimal detectable change values at baseline and after acute concussion exceeded the expected change after concussion, these results are additional challenges of using the BESS to manage concussion.

As expected, the intra-rater scoring was consistently better than inter-rater scoring, which is the most commonly reported management pattern for NCAA Division I but not Division II, Division III, or certified athletic trainers.14,15,27 Based on the video scoring, the BESS score was 28 (range: −14 to +14, with a negative number reflecting a “better” score at T2) across all participants, despite all of the participants having received formal education on administering the BESS, routinely performed the BESS, and watched an educational video prior to each scoring session (Figure 2). When scored by the treating clinicians “live,” there was a substantial increase in the number of errors committed by student-athletes with acute concussion (4.9 ± 6.7 errors). This change is within the expected amount of change (4 to 6 error increase) for individuals after acute concussion, but well below the minimal detectable change values identified in the current study.11,12 Therefore, it is unknown in this sample if the increased/decreased BESS score truly reflected impaired performance by the injured student-athletes or simply a reflection of scorer variability.

When viewed in conjunction with the reported minimal detectable change scores of healthy student-athletes, the range of scores becomes clinically problematic.27 For example, when clinically scored (eg, one athletic trainer scoring the BESS visually), the mean was 12.7 errors at baseline and 17.6 errors after injury, which was a worsening of 4.9 errors but within the expected change.11,12 If considering the “better” intra-rater minimal detectable change, the actual range of scores would be 5 to 20 at baseline27 and 9 to 26 after acute concussion, thus making it unlikely that the difference can be confidently attributed to an actual change in performance rather than measurement error (Figure 3). In the current study, the scores of three student-athletes with acute concussion exceeded the intra-rater minimal detectable change, whereas only two scores exceeded inter-rater minimal detectable change (Figure 1). Thus, health care providers should rely on the overall multifaceted battery to support their clinical judgment in the assessment of patients with suspected acute concussion without overreliance on a single test.1

The minimal detectable change potential overlap of the Balance Error Scoring System (BESS) scores at baseline and after acute concussion. Baseline data were derived from Finnoff et al.,27 and the data after acute concussion were from the results of the current study.

Figure 3.

The minimal detectable change potential overlap of the Balance Error Scoring System (BESS) scores at baseline and after acute concussion. Baseline data were derived from Finnoff et al.,27 and the data after acute concussion were from the results of the current study.

The Sport Concussion Assessment Tool-5 provides three options for balance testing, including the BESS, the modified BESS (mBESS) (which only includes the three firm surfaces), and the tandem gait. Given the overall limitations of the BESS, these other options should be explored.1 Research on the mBESS has been limited, but it has shown moderate reliability, adequate sensitivity (0.71), and a typical increase of 1 to 2 errors in patients with acute concussion.35 Although the sensitivity of the mBESS (0.71) is higher than the BESS (0.16) 24 hours after concussion, the clinical meaningfulness of 1 to 2 errors on the mBESS and the scoring accuracy must be considered.11,25

Additionally, the tandem gait task has received attention and has shown high sensitivity and specificity after acute concussion.17,36–40 These emerging clinical tests appear to have better measurement properties than the BESS, with the tandem gait task having the most potential given the relative scoring objectivity.36,37 Subjectively evaluated assessments potentially expose clinicians to “cognitive diagnostic errors,” which are erroneous verifications of diagnostic hypotheses and occur when clinicians overweight clinical findings that agree with their diagnostic hypothesis.41 The subjective nature of BESS errors (eg, hip moving more than 30° or staying out of the test position for 5 seconds) potentially increases the likelihood of cognitive diagnostic errors. Although the clinical applicability is low, instrumented measures of postural control (eg, gait-related assessments) consistently demonstrate impairments beyond apparent clinical recovery.42–44 The development of clinically feasible and instrumented cost-effective measures of postural control would likely improve the diagnostic sensitivity and management of concussion.30,45

Limitations

This study intentionally used more participants/scores and fewer patients with acute concussion, which is methodologically more appropriate for reliability measures. However, study limitations must be considered.46 Because there were 320 total observations (20 per participant) in the current study, participant fatigue from scoring 10 BESS tests per session is a potential limitation; however, exploratory analysis indicated no differences in scores between early (1 to 3) and later (7 to 10) tests. Although all of the participants were certified athletic trainers who regularly assessed student-athletes after concussion, their education background, experience, and familiarity with the BESS inherently varied. Future studies could investigate the reliability of athletic trainers with varying experience. The scorers in the current study evaluated videos of individuals diagnosed as having concussion, and the aforementioned cognitive diagnostic errors could have influenced their assessment of the student-athletes' performances.

Implications for Clinical Practice

Currently, the BESS is the most recommended and clinically used concussion balance assessment; however, the clinical utility of the BESS is reduced by numerous limitations affecting both the patient performing the test battery and the clinician scoring the performance, which likely underlies the test's low sensitivity. Despite moderate to good reliability, the main finding of this study was that there was a high minimal detectable change score that substantially exceeded the expected change after concussion (intra-rater: 8.6 errors and inter-rater: 11.3 errors). Furthermore, when viewed with the minimal detectable change scores of healthy individuals, the scorer variability creates a range of variability scores that are problematic for the clinical use of the BESS. Thus, additional research is needed to develop clinically feasible balance assessments of individuals after concussion with stronger measurement properties.

References

  1. McCrory P, Meeuwisse W, Dvorak J, et al. Consensus Statement on Concussion in Sport: the 5th International Conference on Concussion in Sport held in Berlin, October 2016. Br J Sports Med. 2017;51:838–847.
  2. Sussman ES, Ho AL, Pendharkar AV, Ghajar J. Clinical evaluation of concussion: the evolving role of oculomotor assessments. Neurosurg Focus. 2016;40:E7. doi:10.3171/2016.1.FOCUS15610 [CrossRef]
  3. Buckley TA, Oldham JR, Caccese JB. Postural control deficits identify lingering post-concussion neurological deficits. J Sport Health Sci. 2016;5:61–69. doi:10.1016/j.jshs.2016.01.007 [CrossRef]
  4. Turner S, Langdon J, Shaver G, et al. Comparison of psychological response between concussion and musculoskeletal injury in collegiate athletes. Sport Exerc Perform Psychol. 2017;6:277–288. doi:10.1037/spy0000099 [CrossRef]
  5. Dobson JL, Yarbrough MB, Perez J, et al. Sport-related concussion induces transient cardiovascular autonomic dysfunction. Am J Physiol Regul Integr Comp Physiol. 2017;312:575–584. doi:10.1152/ajpregu.00499.2016 [CrossRef]
  6. Garcia GP, Lavieri MS, Jiang R, et al. A data-driven approach to unlikely, possible, probable, and definite acute concussion assessment. J Neurotrauma. 2019;36:1571–1583. doi:10.1089/neu.2018.6098 [CrossRef]
  7. Weber ML, Lynall RC, Hoffman NL, et al. ; CARE Consortium Investigators. Health-related quality of life following concussion in collegiate student-athletes with and without concussion history. Ann Biomed Eng. 2019;47:2136–2146. doi:10.1007/s10439-018-02151-7 [CrossRef]
  8. Garcia GP, Broglio SP, Lavieri MS, et al. Quantifying the value of multidimensional assessment models for acute concussion: an analysis of data from the NCAA-DoD care consortium. Sports Med. 2018;48:1739–1749. doi:10.1007/s40279-018-0880-x [CrossRef]
  9. Oldham JR, Munkasy BA, Evans KM, et al. Altered dynamic postural control during gait termination following concussion. Gait Posture. 2016;49:437–442. doi:10.1016/j.gaitpost.2016.07.327 [CrossRef]
  10. Buckley TA, Oldham JR, Munkasy BA, Evans KM. Decreased anticipatory postural adjustments during gait initiation acutely post-concussion. Arch Phys Med Rehabil. 2017;98:1962–1968. doi:10.1016/j.apmr.2017.05.002 [CrossRef]
  11. McCrea M, Barr WB, Guskiewicz K, et al. Standard regression-based methods for measuring recovery after sport-related concussion. J Int Neuropsychol Soc. 2005;11:58–69. doi:10.1017/S1355617705050083 [CrossRef]
  12. McCrea M, Guskiewicz KM, Marshall SW, et al. Acute effects and recovery time following concussion in collegiate football players: the NCAA Concussion Study. JAMA. 2003;290:2556–2563. doi:10.1001/jama.290.19.2556 [CrossRef]
  13. Broglio SP, McCrea M, McAllister T, et al. A national study on the effects of concussion in collegiate athletes and US Military Service Academy Members: the NCAA-DoD Concussion Assessment, Research and Education (CARE) consortium structure and methods. Sports Med. 2017;47:1437–1451. doi:10.1007/s40279-017-0707-1 [CrossRef]
  14. Kelly KC, Jordan EM, Joyner AB, et al. National Collegiate Athletic Association Division I athletic trainers' concussion-management practice patterns. J Athl Train. 2014;49:665–673. doi:10.4085/1062-6050-49.3.25 [CrossRef]
  15. Buckley TA, Burdette G, Kelly K. Concussion-management practice patterns of National Collegiate Athletic Association Division II and III athletic trainers: how the other half lives. J Athl Train. 2015;50:879–888. doi:10.4085/1062-6050-50.7.04 [CrossRef]
  16. Buckley T, Baugh C, Meehan W, DiFabio M. Concussion management plan compliance: a study of NCAA power 5 schools. Ortho J Sports Med. 2017;5:1–7.
  17. Oldham JR, DiFabio MS, Kaminski TW, et al. Efficacy of tandem gait to identify impaired postural control following concussion. Med Sci Sports Exerc. 2018;50:1162–1168. doi:10.1249/MSS.0000000000001540 [CrossRef]
  18. Riemann BL, Guskiewicz KM, Shields EW. Relationship between clinical and forceplate measures of postural stability. J Sport Rehabil. 1999;8:71–82. doi:10.1123/jsr.8.2.71 [CrossRef]
  19. Broglio SP, Katz BP, Zhao S, et al. Test-retest reliability and interpretation of common concussion assessment tools: findings from the NCAA-DoD CARE consortium. Sports Med. 2018;48:1255–1268. doi:10.1007/s40279-017-0813-0 [CrossRef]
  20. Valovich TC, Perrin DH, Gansneder BM. Repeat administration elicits a practice effect with the balance error scoring system but not with the standardized assessment of concussion in high school athletes. J Athl Train. 2003;38:51–56.
  21. Burk JM, Munkasy BA, Joyner AB, Buckley TA. Balance Error Scoring System performance changes after a competitive athletic season. Clin J Sport Med. 2013;23:312–317. doi:10.1097/JSM.0b013e318285633f [CrossRef]
  22. Rahn C, Munkasy BA, Joyner AB, Buckley TA. Sideline Performance of the Balance Error Scoring System during a live sporting event. Clin J Sport Med. 2015;25:248–253. doi:10.1097/JSM.0000000000000141 [CrossRef]
  23. Fox ZG, Mihalik JP, Blackburn JT, Battaglini CL, Guskiewicz KM. Return of postural control to baseline after anaerobic and aerobic exercise protocols. J Athl Train. 2008;43:456–463.. doi:10.4085/1062-6050-43.5.456 [CrossRef]
  24. Docherty CL, McLeod TCV, Shultz SJ. Postural control deficits in participants with functional ankle instability as measured by the balance error scoring system. Clin J Sport Med. 2006;16:203–208. doi:10.1097/00042752-200605000-00003 [CrossRef]
  25. Buckley TA, Munkasy BA, Clouse BP. Sensitivity and specificity of the modified Balance Error Scoring System in concussed student-athletes. Clin J Sport Med. 2017;28:174–176.
  26. Lin KC, Hsieh YW, et al. Minimal Detectable change and clinically important difference of the Wolf Motor Function Test in stroke patients. Neurorehabil Neural Repair. 2009;23:429–434. doi:10.1177/1545968308331144 [CrossRef]
  27. Finnoff JT, Peterson VJ, Hollman JH, Smith J. Intrarater and inter-rater reliability of the Balance Error Scoring System (BESS). PM R. 2009;1:50–54. doi:10.1016/j.pmrj.2008.06.002 [CrossRef]
  28. Portney L, Watkins M. Foundations of Clinical Research: Applications to Practice, 2nd ed. Upper Saddle River, NJ: Prentice Hall; 2000.
  29. McCrory P, Meeuwisse WH, Aubry M, et al. Consensus Statement on Concussion in Sport: the 4th International Conference on Concussion In Sport, Zurich, November 2012. J Athl Train. 2013;48:554–575. doi:10.4085/1062-6050-48.4.05 [CrossRef]
  30. Caccese JB, Buckley TA, Kaminski TW. Sway area and velocity correlated with MobileMat Balance Error Scoring System (BESS) Scores. J Applied Biomech. 2016;32:329–334. doi:10.1123/jab.2015-0273 [CrossRef]
  31. Riemann BL, Guskiewicz KM. Effects of mild head injury on postural stability as measured through clinical balance testing. J Athl Train. 2000;35:19–25.
  32. Chin EY, Nelson LD, Barr WB, McCrory P, McCrea MA. Reliability and validity of the Sport Concussion Assessment Tool-3 (SCAT3) in high school and collegiate athletes. Am J Sports Med. 2016;44:2276–2285. doi:10.1177/0363546516648141 [CrossRef]
  33. Stuart EA, Rodenberg RE, Dotson ML, et al. Reliability measures of the balance error scoring system as administered by certified athletic trainers. Curr Res: Concussion. 2015;2:22–24.
  34. Mendenhall W, Beaver R, Beaver B. Introduction to Probability and Statistics, 10 ed. Pacific Grove, CA: Brooks/Cole Publishing Company; 1999.
  35. Luoto TM, Silverberg ND, Kataja A, et al. Sport Concussion Assessment Tool 2 in a civilian trauma sample with mild traumatic brain injury. J Neurotrauma. 2014;31:728–738. doi:10.1089/neu.2013.3174 [CrossRef]
  36. Howell DR, Osternig LR, Chou LS. Single-task and dual-task tandem gait test performance after concussion. J Sci Med Sport. 2017;20:622–626. doi:10.1016/j.jsams.2016.11.020 [CrossRef]
  37. Oldham JR, DiFabio MS, Kaminski TW, et al. Normative tandem gait in collegiate athletes implications for clinical concussion assessment. Sports Health. 2017;9:305–311. doi:10.1177/1941738116680999 [CrossRef]
  38. Howell DR, Oldham JR, Meehan WP, et al. Dual task tandem gait and average walking speed in healthy collegiate athletes. Clin J Sport Med. 2019;29:238–244. doi:10.1097/JSM.0000000000000509 [CrossRef]
  39. Howell DR, Oldham JR, DiFabio M, et al. Single-task and dual-task gait among collegiate athletes of different sport classifications: implications for concussion management. J Appl Biomech. 2017;33:24–31. doi:10.1123/jab.2015-0323 [CrossRef]
  40. Howell DR, Berkstresser B, Wang F, et al. Self-reported sleep duration affects tandem gait, but not steady-state gait outcomes among healthy collegiate athletes. Gait Posture. 2018;62:291–296. doi:10.1016/j.gaitpost.2018.03.038 [CrossRef]
  41. van den Berge K, Mamede S. Cognitive diagnostic error in internal medicine. Euro J Intern Med. 2013;24:525–529. doi:10.1016/j.ejim.2013.03.006 [CrossRef]
  42. Gao J, Hu J, Buckley T, et al. Shannon and Renyi Entropies to classify effects of mild traumatic brain injury on postural sway. PLoS One. 2011;6:e24446. doi:10.1371/journal.pone.0024446 [CrossRef]
  43. Buckley TA, Munkasy BA, Tapia-Lovler TG, Wikstrom EA. Altered gait termination strategies following a concussion. Gait Posture. 2013;38:549–551. doi:10.1016/j.gaitpost.2013.02.008 [CrossRef]
  44. Howell DR, Osternig LR, Christie AD, Chou LS. Return to physical activity timing and dual-task gait stability are associated 2 months following concussion. J Head Trauma Rehabil. 2016;31:262–268. doi:10.1097/HTR.0000000000000176 [CrossRef]
  45. Howell D, Osternig L, Chou LS. Monitoring recovery of gait balance control following concussion using an accelerometer. J Biomech. 2015;48:3364–3368. doi:10.1016/j.jbiomech.2015.06.014 [CrossRef]
  46. Gwet KL. Handbook of Inter-Rater Reliability: The Definitive Guide to Measuring the Extent of Agreement Among Multiple Raters, 3rd ed. Gaithersburg, MD: Advanced Analytics, LLC.; 2012.
Authors

From Houston Dynamo, Houston, Texas (CDC); the School of Health and Kinesiology, Georgia Southern University, Statesboro, Georgia (JLL, BAM); the Brody School of Medicine, East Carolina University, Greenville, North Carolina (KME); and the University of Delaware, Newark, Delaware (TAB).

Supported in part by NIH/NINDS Grant No. 1R15NS070744–01A1.

The authors have no financial or proprietary interest in the materials presented herein.

Correspondence: Thomas A. Buckley, EdD, ATC, University of Delaware, 349 Tower at STAR Campus, 100 Discovery Blvd., Newark, DE 19716. E-mail: TBuckley@UDel.edu

Received: June 14, 2018
Accepted: January 07, 2019
Posted Online: May 08, 2019

10.3928/19425864-20190401-02

Sign up to receive

Journal E-contents