Athletic Training and Sports Health Care

Original Research 

The Effect of the Environment on SCAT2 Scores

Deanna H. Smith, MSEd; Thomas G. Bowman, PhD; Debbie A. Bradney, DPE

Abstract

Current concussion management guidelines emphasize the use of baseline testing to properly assess the presence of deficits following injury. However, there are no specific guidelines for the environment in which these tests are administered. The purpose of this study was to determine if testing environment has an effect on Sport Concussion Assessment Tool 2 (SCAT2) scores in healthy collegiate club male lacrosse and female soccer athletes. Participants completed the SCAT2 test in two environments, a controlled classroom and an uncontrolled sideline, in two separate testing sessions. Total SCAT2 scores decreased by a mean of 7.64 ± 1.38 points (P < .001) for combined genders in the uncontrolled sideline environment compared to the controlled clinical environment. The results suggest that baseline SCAT2 testing should be administered in an environment in which testing after injury will most likely be conducted to produce the most valid scores. [Athletic Training & Sports Health Care. 2016;8(5):193–201.]

Abstract

Current concussion management guidelines emphasize the use of baseline testing to properly assess the presence of deficits following injury. However, there are no specific guidelines for the environment in which these tests are administered. The purpose of this study was to determine if testing environment has an effect on Sport Concussion Assessment Tool 2 (SCAT2) scores in healthy collegiate club male lacrosse and female soccer athletes. Participants completed the SCAT2 test in two environments, a controlled classroom and an uncontrolled sideline, in two separate testing sessions. Total SCAT2 scores decreased by a mean of 7.64 ± 1.38 points (P < .001) for combined genders in the uncontrolled sideline environment compared to the controlled clinical environment. The results suggest that baseline SCAT2 testing should be administered in an environment in which testing after injury will most likely be conducted to produce the most valid scores. [Athletic Training & Sports Health Care. 2016;8(5):193–201.]

The current recommended guidelines for concussion management include the implementation of preseason baseline testing to provide an individualistic method of post-injury assessment and recovery tracking in athletics.1,2 These standardized tests are a vital aspect of the multifaceted approach used by health care providers when making return to play decisions. The Sport Concussion Assessment Tool 2 (SCAT2) offers a standardized, multifaceted injury evaluation that can be completed as a baseline test and then used as an assessment tool following injury. The tool is widely recommended for implementation by athletic trainers due to its ease of use and trouble-free portability.2 The SCAT2 is composed of several well-known, preexisting concussion assessment tools, such as the Standardized Assessment of Concussion (SAC), a modified version of the Balance Error Scoring System (BESS), the Glasgow Coma Scale, and the Maddocks sideline assessment score. These elements are combined, along with a symptom scale, to more accurately diagnose and identify the incidence of a concussion from the time of injury.3 The validity of the SCAT2 has been previously examined in the literature. Researchers found that the SCAT2 has 96% sensitivity and 83% specificity in identifying concussion when a baseline is present.4 Therefore, when used correctly, the SCAT2 may be a useful tool in the assessment of concussion.

Although research exists demonstrating the clinical relevance of several standardized concussion assessment tools,4–6 there are still gaps in the literature regarding the validity of such tests conducted in a controlled clinical environment compared to an uncontrolled sideline environment. The effect of the environment on both cognition and postural control has been previously examined.7–9 Researchers found that the environment poses little effect on cognitive tests9; however, environmental factors may negatively affect postural control in athletes.10 As previously stated, the SCAT2 includes the SAC and a modified BESS, both of which have been previously validated in the literature.5,6 Although these findings are pertinent to this study, it is important to note that these validated components account for only 35 of a total 100 points on the SCAT2. No studies to date have examined the effect of test environment on the SCAT2 as a whole.

There are currently no guidelines for test environment in which athletes should be administered a baseline SCAT2. The lack of specific guidelines on test environment creates a potential problem because an inconsistent environment between the baseline test and the test conducted after a suspected concussion, most often the sideline, may yield results based on environmental differences and not head trauma. Therefore, the purpose of this study was to examine the effect of the test environment on SCAT2 scores. We hypothesized that SCAT2 scores taken during a sideline environment test administration would be lower than those taken in a clinically controlled environment.

Methods

We used an experimental, randomized, repeated measures design to complete this study. The independent variable was test environment (controlled vs uncontrolled) and the dependent variables were the SCAT2 scores.

Participants

Eighteen healthy male club lacrosse players (age: 20.39 ± 1.46 years, height: 180.17 ± 6.92 cm, weight: 81.89 ± 8.92 kg) and 15 healthy female club soccer players (age: 19.41 ± 1.24 years, height: 158.53 ± 6.81 cm, weight: 58.85 ± 7.53 kg) volunteered to participate in our study. The researchers randomly assigned half of the participants from each sport to the uncontrolled sideline group first and half to the controlled clinical classroom group. All participants completed the same SCAT2 test protocol in each environment during two sessions separated by 8.05 ± 1.63 days. The researchers excluded participants if they had any prior history of head injury in the past 12 months, any lower extremity injury reported in the past 2 months that may affect balance, and any prior history of attention deficit disorder or attention deficit hyperactivity disorder. These exclusions were determined by inquiries from the lead investigator prior to signing the informed consent form. All participants read and signed an informed consent form approved by the institutional review board of the host institution and completed a physical activity readiness questionnaire (PAR-Q)11 to ensure they had the physical ability to complete the study.

Instruments

The SCAT2 is a standardized method of assessing athletes for the presence of a concussion, with preseason baseline testing recommended as a helpful interpreter of post-injury test scores.2 The tool was first introduced at the 3rd International Conference on Concussion in Sport in Zurich, Switzerland, in November, 2008.2 It is a written test comprising seven components: a subjective symptom score graded on a 0 to 6 severity, a physical signs score, the Glasgow coma scale, a Maddocks score, a balance examination, a coordination examination, and a cognitive assessment. Each component has a scoring system, which is then totaled out of 100 points at the completion of the testing, or 105 points including the Maddox score.

Each component of the SCAT2 was specifically selected for inclusion based on previous studies examining the validity and reliability of such tests on diagnosing deficits following head injury.2 The symptom score comprises a 22-item post-concussion scale using a 7-point Likert rating. The symptom score is a commonly used subjective post-concussion assessment tool, and is typically used to make both return to learn and return to play decisions.12,13 Previous studies have demonstrated the reliability and sensitivity of self-reported symptoms in determining presence and severity as described by the patient,12,14–17 especially when combined with other concussion assessment tools as a multifaceted concussion assessment approach. Furthermore, a symptom assessment may be used by clinicians to track self-reported recovery from concussion because an athlete's ability to perform both sports-specific and academic tasks without a recurrence of symptoms may be a sign of that individual's recovery.

The SAC was initially developed in 1997 by McCrea et al. as an objective measure of changes in cognitive function following concussion.18 The SAC comprises four main assessment components: orientation, immediate memory, concentration, and delayed recall, all of which can be easily examined on the sideline. Several studies have previously validated the SAC as a sensitive form of sport-related concussion assessment.6–8

The balance portion of the SCAT2 uses a modified version of the BESS. This modified version does not include the three unstable conditions on a foam surface. The modified BESS is scored by counting the number of errors an athlete receives within a 20-second period for double-leg, single-leg, and tandem stances. A maximum of 30 errors can be obtained; errors include taking a step, stumbling, opening the eyes, and removing the hands from the hips. An increase in errors from an athlete's baseline to post-injury score may indicate a decline in postural control. The BESS has been previously demonstrated as a valid form of balance assessment in the literature19–22; however, it has been shown to have varying low to high reliability depending on the type of reliability assessed.22,23

Procedures

The institutional review board of the host institution approved our study prior to the collection of data. We initially e-mailed all members on the men's club lacrosse and the women's club soccer team rosters to request volunteers for the study. We chose members of club teams because these individuals are not required to complete baseline SCAT2 testing prior to participation in the season and are therefore unfamiliar with the test, limiting the possibility of a practice effect. No participants had taken the baseline test previously.

Once we secured a group of volunteers for each gender, we set up meeting appointment times by e-mail for each participant. Each participant agreed to report on two separate occasions between 4:00 pm and 6:00 pm in the front lobby of the gymnasium at the host institution. We instructed participants to report any head or lower extremity injuries that occurred between testing sessions; we discarded the results of those who had sustained such injuries in an effort to reduce extraneous variables that may alter SCAT2 scores. On the first occasion, each participant filled out a PAR-Q,11 read through and signed the informed consent agreement, and then read a script detailing participation in the study.

The lead investigator administered all of the SCAT2 tests throughout the study. We counterbalanced the design by randomly assigning half of the participants to one group (controlled classroom environment) and half to another group (uncontrolled sideline environment) by flipping a coin to reduce scoring differences due to a learning effect. We took the participants chosen to take the test in the controlled setting first into one of the empty classrooms in the gymnasium building to administer the SCAT2. Participants chosen to take the test on the sideline first were instructed to attend the scheduled club practice for the day on the practice field. We pulled participants out of practice to complete the SCAT2 test on the sideline, the location in which an athletic trainer might conduct an immediate assessment of an athlete who sustained a potential concussion during an athletic event.

For each data collection session, the lead investigator gave each participant a number to correspond with his or her test scores to maintain confidentiality. This number was written at the top of the participant's SCAT2 test packet. The researcher then started the SCAT2 test with the “What is the SCAT2” heading to give the participant more of an idea of why and how the SCAT2 is used prior to administering the actual testing portion. The researcher then began the test chronologically, starting with the symptom evaluation portion. The rest of the components of the test were read to the participant in the exact same order with the exact wording used on the SCAT2. The participant received a certain score for each component of the test, and these scores were totaled at the end of the testing.

We recorded scores for each component and a total SCAT2 score for each participant. Due to the fact that participants were healthy for the duration of the study, full points were earned for the Glasgow Coma Scale, physical signs, coordination, and Maddox scores. The Maddox score was then excluded from the overall score because it is not included in the SCAT2 total score. We asked each participant to report again approximately 1 week from the test day. We used the approximate 1-week period to decrease the likelihood of a participant obtaining a better score due to a learning effect.

When the participants reported for their second session, we repeated the administration of the SCAT2 in the environment the participants did not complete in their first visit. During the cognitive component of the SCAT2, we asked the participant to remember a set of five words and repeat a string of numbers in reverse order. There are a total of four sets of words and number lists, with a main list and three alternative lists. The only change from the first SCAT2 test to the second was the word and number list we asked the participants to remember. During the second data collection session, we used the set of words and the set of numbers from the second list. The reasoning for this procedure was to decrease the likelihood of memorization or a learning effect from the first SCAT2 administration to the second. With the completion of the test, the researcher recorded the participant's individual component scores and total SCAT2 score. We marked each participant's two completed SCAT2 forms with his or her corresponding number and the environment in which we conducted each SCAT2 test.

Statistical Analysis

We analyzed the data using IBM SPSS software (version 19.0; IBM Inc., Somers, NY) to determine the environmental differences in SCAT2 individual component scores and overall scores for each participant. We did not include the Maddox score in our data analysis to keep the SCAT2 total score set at 100. We evaluated the data for differences due to test environment by completing a multivariate analysis of variance. We also completed follow-up analyses of variance (ANOVAs) for each of the SCAT2 components. Alpha levels of statistical significance were set at P ≤ .05 a priori.

Results

We found statistically significant group mean differences between testing environments (multivariate F7,58 = 11.098, P < .001). Our results also indicated a strong association between the testing environment and the combined dependent variables (η2 = 0.573). In follow-up ANOVAs, we found statistical differences for total scores (F1,64 = 25.040, P < .001, ω2 = 0.267), symptom scores (F1,64 = 10.492, P = .002, ω2 = 0.123), the modified BESS scores (F1,64 = 36.767, P < .001, ω2 = 0.351), immediate memory scores (F1,64 = 5.036, P = .028, ω2 = 0.036), and concentration scores (F1,64 = 7.230, P = .009, ω2 = 0.086) based on the testing environment. Components that we found to not be statistically significant were the delayed recall scores (F1,64 = 0.125, P = .725, 1-β = 0.064, ω2 = 0.032) and orientation scores (F1,64 = 3.154, P = .417, 1-β= .080, ω2 = 0.013). The physical signs scores, Glasgow Coma Scale, and coordination scores were identical for the participants regardless of the testing environment because participants in both groups obtained perfect scores. Table 1 shows the mean, standard deviation, and 95% confidence interval of the combined gender data for each environment. Figure 1 shows the differences in mean component and total SCAT2 scores.


Combined Gender Data by Environment

Table 1:

Combined Gender Data by Environment


Mean Sport Concussion Assessment Tool 2 (SCAT2) component and total scores for both sexes. *Physical sign score, Glasgow Coma Scale, coordination, and Maddocks scores are not included because there was no change between environments. **Indicates statistical significance between environments (P ≤ .05 a priori).

Figure 1.

Mean Sport Concussion Assessment Tool 2 (SCAT2) component and total scores for both sexes. *Physical sign score, Glasgow Coma Scale, coordination, and Maddocks scores are not included because there was no change between environments. **Indicates statistical significance between environments (P ≤ .05 a priori).

Males

We found statistically significant group mean differences between testing environments for the male participants (multivariate F7,28 = 6.759, P < .001). Our results also indicated a strong association between the testing environment and the combined dependent variables (η2 = 0.628). In follow-up ANOVAs, we found statistical differences for total scores (F1,34 = 16.756, P < .001, ω2 = 0.312), the modified BESS scores (F1,34 = 17.118, P < .001, ω2 = 0.311), the symptom scores (F1,34 = 6.725, P = .014, ω2 = 0.137), and the concentration scores (F1,34 = 4.501, P = .041, ω2 = 0.089) based on the testing environment. Components that we found to not be statistically significant were the delayed recall scores (F1,34 = 3.859, P = .058, 1-β = 0.481, ω2 = 0.074), immediate memory scores (F1,34 = .860, P = .360, 1-β = 0.153, ω2 = 0.003), and orientation scores (F1,34 = .168, P = .684, 1-β = 0.071, ω2 = 0.024). The physical signs scores, Glasgow Coma Scale, and coordination scores were identical for the participants regardless of testing environment because participants in both groups obtained perfect scores. Figure 2 shows the differences in mean component and total SCAT2 scores.


Mean Sport Concussion Assessment Tool 2 (SCAT2) component and total scores in males. *Physical sign score, Glasgow Coma Scale, coordination, and Maddocks scores are not included because there was no change between environments. **Indicates statistical significance between environments (P ≤ .05 a priori).

Figure 2.

Mean Sport Concussion Assessment Tool 2 (SCAT2) component and total scores in males. *Physical sign score, Glasgow Coma Scale, coordination, and Maddocks scores are not included because there was no change between environments. **Indicates statistical significance between environments (P ≤ .05 a priori).

Females

We found statistically significant group mean differences between testing environments for females (multivariate F6,23 = 8.306, P < .001). Our results also indicated a strong association between the testing environment and the combined dependent variables (η2 = 0.684). In follow-up ANOVAs, we found statistical differences for total scores (F1,28 = 37.799, P < .001, ω2 = 0.551), symptom scores (F1,28 = 7.814, P = .009, ω2 = 0.185), the modified BESS scores (F1,28 = 23.805, P < .001, ω2 = 0.432), and the immediate memory scores (F1,28 = 5.502, P = .026, ω2 = 0.131) based on the testing environment. Components that we found were not statistically significant were the orientation scores (F1,28 = 2.514, P = .071, 1-β = 0.294, ω2 = 0.037), concentration scores (F1,28 = 2.560, P = .121, 1-β = 0.339, ω2 = 0.051), and delayed recall scores (F1,28 = 0.180, P = .675, 1-β = 0.069, ω2 = 0.003). The physical signs scores, Glasgow Coma Scale, and coordination scores were also identical for the female participants regardless of testing environment because participants in both groups obtained perfect scores. Figure 3 shows the differences in mean component and total SCAT2 scores.


Mean Sport Concussion Assessment Tool 2 (SCAT2) component and total scores in females. *Physical sign score, Glasgow Coma Scale, coordination, and Maddocks scores are not included because there was no change between environments. **Indicates statistical significance between environments (P ≤ .05 a priori).

Figure 3.

Mean Sport Concussion Assessment Tool 2 (SCAT2) component and total scores in females. *Physical sign score, Glasgow Coma Scale, coordination, and Maddocks scores are not included because there was no change between environments. **Indicates statistical significance between environments (P ≤ .05 a priori).

Discussion

The results of this study suggest that test environment could potentially lead to false-positive SCAT2 scores. On average, we found that overall SCAT2 scores decreased (worsened) by a mean of 7.64 ± 1.38 points for combined genders when tests were administered in the uncontrolled sideline environment. Our finding that an uncontrolled environment decreases total SCAT2 scores is clinically significant because the SCAT2 is currently one of the nationally recommended concussion assessment tools for preseason baseline and post-injury assessment.2 Our main finding demonstrates that conducting baseline and post-injury assessment in a similar environment may make testing more accurate. Furthermore, the baseline test in a more controlled environment has the potential to make the SCAT2 more conservative, which we believe would be advantageous compared to a false-negative situation where patients are allowed to continue athletic participation when they have a concussion.

Recent studies have been conducted to determine normative values for the SCAT2 for both collegiate and adolescent populations.24,25 These studies are important because the existence of normative values for the SCAT2 can provide a good marker for clinicians when baseline testing is lacking. Although the current gold standard is to conduct baseline testing, normative values may reflect similar sensitivity for concussion diagnosis.26 Zimmer et al.25 found that a normative value for total SCAT2 scores in a collegiate population is 90.83 ± 5.60 for males and 91.65 ± 5.58 for females. We found lower scores in our study for the controlled environment for males (87.17 ± 4.77) and higher scores for the controlled environment for females (95.00 ± 2.24). These scores were meant to most closely resemble a baseline test for our participants. The disparity in our findings may be attributed to differences in study design and procedures, with the largest disparity being a much smaller sample size in our study. Based on normative data studies23,24 and our own findings, we recommend that baseline testing be conducted when feasible.

Balance

The results of our study indicated that the greatest differences between environments existed within the balance component of the SCAT2. The balance portion of the SCAT2 is a modified version of the BESS, which clinicians originally created as a sideline assessment tool for postural control immediately following injury.27 To our knowledge, although the validity and reliability of the BESS has been previously examined, the modified BESS (only the firm conditions) has not. Riemann et al.27 reported that an average error rate of 2.3, or an overall total estimated score of 26.7 out of a maximum of 30, is considered normal for the firm stance in a college-aged population. For male participants, we found a mean BESS score of 2.00 ± 1.14 errors for the controlled environment and 7.67 ± 5.63 errors for the uncontrolled environment. Similarly, for female participants, we found a mean BESS score of 1.93 ± 1.49 errors for the controlled environment and 6.53 ± 3.33 errors for the uncontrolled environment. Due to such significant differences (an average of approximately 4.5 and 5 points) from the controlled environment values, our findings suggest that an uncontrolled sideline environment could adversely affect an athlete's balance.

Onate et al.10 also conducted a study comparing the effects of the environment on the BESS scores of collegiate level baseball players. Investigators found that an uncontrolled sideline environment may negatively affect BESS scores when compared with a controlled clinical environment.10 The investigators in this study10 used the sideline of a baseball field as the uncontrolled environment and a locker room as the controlled clinical environment. Researchers10 examined both firm and foam conditions and found that the single-leg foam surface condition yielded the most significantly decreased scores between environments. As previously stated, the modified BESS that is completed as part of the SCAT2 only includes firm surface conditions; therefore, we caution the interpretation of these previous findings10 with our own. Based on this knowledge and our own findings, we recommend that test environment be considered when assessing balance during baseline and post-injury testing.

In a similar study, Rahn et al.28 found that testing environment may lead to deterioration in BESS scores. Investigators in this study examined the influence of live sporting environments (football stadium and basketball arena) on BESS performance with a female population, and found that a live sporting event led to an increase (worsening) of scores compared to a control group in a quiet environment. Similar to our own conclusions, Rahn et al.28 also recommended that clinicians consider the environment in which the BESS is performed, and attempt to match baseline testing environment with the post-injury testing environment.

To fully understand the impact an uncontrolled environment may have on postural control, it is important to discuss the dual-task paradigm. A dual-task paradigm is a procedure that requires an individual to perform two tasks simultaneously. During the sideline concussion assessment, a concussed athlete may experience a reduction in attentional resources, making postural control more difficult.29 Multiple studies have been published examining the role that the dual-task procedure plays in the control of balance during gait.30–32 In a controlled environment, the sole focus of our participants was balance; however, in the uncontrolled environment, external stimuli may have affected their focus. We hypothesize that the dual-task procedure negatively affected the BESS scores obtained by our participants. This furthers our conclusion that baseline testing should be conducted in an environment most similar to where post-injury testing will be conducted. Future studies should further examine the dual-task paradigm to better understand its effect on sideline concussion assessment.

We acknowledge that a potential limitation of this study is a lack of control for fatigue in participants during the uncontrolled environment test sessions. The effect of fatigue on postural control has been previously examined in the literature.33,34 Researchers found that after completing a functional exercise protocol that simulates exertion during athletic activity, participants had balance deficits.34 Researchers also found that these deficits can be expected to return to pretest levels after 20 minutes of inactivity.33 Based on these findings, the deficits in our participants' balance may have been attributed to not only the environment, but also fatigue. The SCAT2 does attempt to account for fatigue by assessing many aspects of cognition before balance because the SAC is not as affected by fatigue8; however, we recommend that clinicians still take fatigue into account when assessing balance on the sideline as part of the SCAT2.

Symptoms

The differences we found in symptom scores between environments can most likely be attributed to each participant's activity level for that specific day. Participants in the uncontrolled sideline environment tended to report higher symptom scale levels for fatigue or low energy, drowsiness, and headache compared with the controlled environment. We attributed this to the fact that we pulled participants out of practice to perform the SCAT2 after each individual had already been exercising for a period of time. As previously mentioned, the SCAT2 may be administered post-injury on the sideline during either a practice or a game. In either of these scenarios, an athlete may report certain symptoms because of other extraneous variables besides an obvious head trauma. For example, dehydration could cause symptoms in an athlete that could potentially negatively affect concussion testing. The effect of dehydration on concussion assessment has been previously examined in the literature.35,36 Researchers found that when compared to their euhydrated state, dehydrated athletes experienced higher symptom severity scores for headache, feeling slowed down, and feeling in a fog.35 Although we did not incorporate hydration measures into our procedures, we recognize that this factor could have influenced our symptom findings. We recommend that hydration status be considered during concussion assessment, and that clinicians examine deviations from baseline in symptoms by considering whether any described symptoms get worse with physical or mental activity. Future studies examining the effect of different levels of exercise on certain symptoms of a concussion that are listed on the SCAT2 would be a vital contribution to research surrounding the validity of the SCAT2 as a whole.

Cognition

We also found several differences in SAC component scores when comparing the two environments. The SAC is composed of orientation, immediate memory, concentration, and delayed recall scores. We found concentration in males to have the only statistically significant impairment in score when tested in the uncontrolled environment. Female data yielded different results, with a statistically significant impairment in score for immediate memory when tested in the uncontrolled environment. Currently, there is no conclusive evidence to suggest why gender differences existed in our study; therefore, studies investigating the reasons behind the significant decrease in certain components of the SAC between environments for specific genders should be conducted.

We also found that the female SAC scores had an average drop of a little over 1 point between environments (controlled = 27.13 ± 2.32, uncontrolled = 25.85 ± 1.598). However, our male findings did not have this drop in points between environments (controlled = 25.71 ± 5.61, uncontrolled = 25.76 ± 5.16). Literature on the SAC states that a drop of 1 point or more from preseason baseline score on the SAC is 95% sensitive and 76% specific in correctly classifying injured and uninjured participants.8 Therefore, we found that the environment could potentially cause a false-positive result while assessing cognition for females but not males. Previously reported normative values for the SAC are 26.97 ± 2.05 for males and 27.63 ± 1.87 for females.25 Although consistent with our female controlled scores, these findings are inconsistent with our male controlled scores. Due to an inconsistency in scores for males, we recommend caution when interpreting our results. We believe the gender differences we found in this study provide a gateway for future research that should further examine how the environment affects male and female cognitive function during concussion assessment.

It is important to note that overall SAC scores were not statistically lower between environments. This suggests that the individual SAC component findings may not be clinically significant by themselves, much like the SAC is only one facet of the SCAT2. In our study, the small drop in points we found in the uncontrolled environment for both genders combined (controlled = 26.36 ± 2.16, uncontrolled = 25.82 ± 1.13) shows that the testing environment may not have had as much of an impact on cognitive function compared to postural stability or symptoms. These findings are similar to a study that investigated the differences between a clinical and sideline environment when evaluating cognitive test performance using only the SAC.9 The researchers concluded that clinicians could administer cognitive tests in an uncontrolled environment to a certain male athletic population and expect to obtain results similar to those in a controlled environment.9 Although the overall SAC score did not significantly change between environments, we found statistical significance in the concentration portion of the SAC for males and the immediate memory portion for females between the uncontrolled and controlled environments. Therefore, we still recommend that athletic trainers take the environment into consideration when conducting cognitive assessment because the SAC is only one component of the SCAT2.

Limitations and Future Directions

It is important to note that since the completion of this study, the SCAT3 has been released. An updated version of the SCAT2, the SCAT3 contains an additional tandem gait assessment during the balance component.2 Furthermore, the SCAT3 does not contain a composite score. No additional significant updates were made to the symptom, cognitive, or original balance portions. Future research should investigate the effect of the environment on the newly updated SCAT3.

Another limitation to the study is that our population was a small convenience sample of healthy, college-aged male and female club sport athletes, limiting generalizability. Further research should investigate the effects of the testing environment on other populations such as high school, adolescent, and a more varied sample of collegiate athletes. Also, uncontrolled environments are deemed such for a reason—they are unpredictable. We made no attempt to control for noises, distractions, or interference while testing in this environment with the reasoning that this would be most reflective of an actual practice or game. Future research could measure and examine these factors and also incorporate the use of video cameras to ensure that the environment was reasonably consistent for each testing session.

Implications for Clinical Practice

Clinicians should consider the effect of environmental factors when administering the SCAT2 in different settings. Although we found that some components of the SCAT2 yielded similar results between environments, an uncontrolled sideline environment adversely affected overall SCAT2 scores. Therefore, we recommend that clinicians conduct baseline SCAT2 testing in an environment similar to post-injury SCAT2 testing to decrease the likelihood of producing a false-positive test score. However, administering the baseline test in a more controlled environment has the potential to make the SCAT2 more conservative, which we believe would be advantageous compared to a false-negative situation where athletes are allowed to continue athletic participation when they have a concussion.

References

  1. Guskiewicz KM, Bruce SL, Cantu RC, et al. National Athletic Trainers' Association position statement: management of sport-related concussion. J Athl Train. 2004;39:280–297.
  2. McCrory P, Meeuwisse W, Johnston K, et al. Consensus statement on concussion in sport: the 3rd International Conference on Concussion in Sport held in Zurich, November 2008. Am J Sport Med. 2009;21:36–45.
  3. Valovich McLeod TC. The value of various assessment techniques in detecting the effects of concussion on cognition, symptoms, and postural control. J Athl Train. 2009;44:663–665. doi:10.4085/1062-6050-44.6.663 [CrossRef]
  4. Putukian M, Echemendia R, Dettwiler-Danspeckgruber A, et al. Prospective clinical assessment using Sideline Concussion Assessment Tool-2 testing in the evaluation of sport-related concussion in college athletes. Clin J Sport Med. 2015;25:36–42. doi:10.1097/JSM.0000000000000102 [CrossRef]
  5. Barlow M, Schlabach D, Peiffer J, Cook C. Differences in change scores and the predictive validity of three commonly used measures following concussion in the middle school and high school aged population. Int J Sports Phys Ther. 2011;6:150–157.
  6. McCrea M, Barr WB, Guskiewicz K, et al. Standard regression-based methods for measuring recovery after sport-related concussion. J Int Neuropsychol Soc. 2005;11:58–69. doi:10.1017/S1355617705050083 [CrossRef]
  7. Barr WB, McCrea M. Sensitivity and specificity of standardized neurocognitive testing immediately following sports concussion. J Int Neuropsychol Soc. 2001;7:693–702. doi:10.1017/S1355617701766052 [CrossRef]
  8. McCrea M. Standardized mental status testing on the sideline after sport-related concussion. J Athl Train. 2001;36:274–279.
  9. Onate JA, Guskiewicz KM, Riemann BL, Prentice WE. A comparison of sideline versus clinical cognitive test performance in collegiate athletes. J Athl Train. 2000;35:155–160.
  10. Onate JA, Beck BC, Van Lunen BL. On-field testing environment and balance error scoring system performance during preseason screening of healthy collegiate baseball players. J Athl Train. 2007;42:446–451.
  11. Thomas S, Reading J, Shephard RJ. Revision of the physical activity readiness questionnaire (PAR-Q). Can J Sport Sci. 1992;17:338–345.
  12. Alla S, Sullivan SJ, Hale L, McCrory P. Self-report scales/checklists for the measurement of concussion symptoms: a systematic review. Br J Sports Med. 2009;43(suppl 1):i3–i12. doi:10.1136/bjsm.2009.058339 [CrossRef]
  13. Notebaert AJ, Guskiewicz KM. Current trends in athletic training practice for concussion assessment and management. J Athl Train. 2005;40:320–325.
  14. Broglio SP, Macciocchi SN, Ferrara MS. Sensitivity of the concussion assessment battery. Neurosurgery. 2007;60:1050–1057. doi:10.1227/01.NEU.0000255479.90999.C0 [CrossRef]
  15. Lovell MR, Iverson GL, Collins MW, et al. Measurement of symptoms following sports-related concussion: reliability and normative data for the post-concussion scale. Appl Neuropsychol. 2006;13:166–174. doi:10.1207/s15324826an1303_4 [CrossRef]
  16. Mailer BJ, Valovich-McLeod TC, Bay RC. Healthy youth are reliable in reporting symptoms on a graded symptom scale. J Sport Rehabil. 2008;17:11–20. doi:10.1123/jsr.17.1.11 [CrossRef]
  17. Piland SG, Motl RW, Ferrara MS, Peterson CL. Evidence for the factorial and construct validity of a self-report concussion symptoms scale. J Athl Train. 2003;38:104–112.
  18. McCrea M, Kelly JP, Randolph C, et al. Standardized assessment of concussion (SAC): on-site mental status evaluation of the athlete. J Head Trauma Rehabil. 1998;13:27–35. doi:10.1097/00001199-199804000-00005 [CrossRef]
  19. Guskiewicz KM. Postural stability assessment following concussion: one piece of the puzzle. Clin J Sport Med. 2001;11:182–189. doi:10.1097/00042752-200107000-00009 [CrossRef]
  20. Guskiewicz KM, Ross SE, Marshall SW. Postural stability and neuropsychological deficits after concussion in collegiate athletes. J Athl Train. 2001;36:263–273.
  21. Riemann BL, Guskiewicz KM. Effects of mild head injury on postural stability as measured through clinical balance testing. J Athl Train. 2000;35:19–25.
  22. Bell DR, Guskiewicz KM, Clark MA, Padua DA. Systematic review of the balance error scoring system. Sports Health. 2011;3:287–295. doi:10.1177/1941738111403122 [CrossRef]
  23. Finnoff JT, Peterson VJ, Hollman JH, Smith J. Intrarater and inter-rater reliability of the Balance Error Scoring System (BESS). PM R. 2009;1:50–54. doi:10.1016/j.pmrj.2008.06.002 [CrossRef]
  24. Jinguji TM, Bompadre V, Harmon KG, et al. Sport Concussion Assessment Tool-2: baseline values for high school athletes. Br J Sports Med. 2012;46:365–370. doi:10.1136/bjsports-2011-090526 [CrossRef]
  25. Zimmer A, Marcinak J, Hibyan S, Webbe F. Normative values of major SCAT2 and SCAT3 components for a college athlete population. Appl Neurophysiol Adult. 2015;22:132–134. doi:10.1080/23279095.2013.867265 [CrossRef]
  26. Schmidt JD, Register-Mihalik JK, Mihalik JP, Kerr ZY, Guskiewicz KM. Identifying impairments after concussion: normative data versus individualized baselines. Med Sci Sports Exerc. 2012;44:1621–1628. doi:10.1249/MSS.0b013e318258a9fb [CrossRef]
  27. Riemann BL, Guskiewicz KM, Shields EW. Relationship between clinical and forceplate measures of postural stability. J Sport Rehabil. 1999;8:71–82. doi:10.1123/jsr.8.2.71 [CrossRef]
  28. Rahn C, Munkasy BA, Barry Joyner A, Buckley TA. Sideline performance of the balance error scoring system during a live sporting event. Clin J Sport Med. 2015;25:248–253. doi:10.1097/JSM.0000000000000141 [CrossRef]
  29. Howell DR, Osternig LR, Chou LS. Dual-task effect on gait balance control in adolescents with concussion. Arch Phys Med Rehabil. 2013;94:1513–1520. doi:10.1016/j.apmr.2013.04.015 [CrossRef]
  30. Parker TM, Osternig LR, Lee HJ, Donkelaar P, Chou LS. The effect of divided attention on gait stability following concussion. Clin Biomech (Bristol, Avon). 2005;20:389–395. doi:10.1016/j.clinbiomech.2004.12.004 [CrossRef]
  31. Redfern MS, Jennings JR, Martin C, Furman JM. Attention influences sensory integration for postural control in older adults. Gait Posture. 2001;14:211–216. doi:10.1016/S0966-6362(01)00144-8 [CrossRef]
  32. Woollacott M, Shumway-Cook A. Attention and the control of posture and gait: a review of an emerging area of research. Gait Posture. 2002;16:1–14. doi:10.1016/S0966-6362(01)00156-4 [CrossRef]
  33. Susco TM, Valovich McLeod TC, Gansneder BM, Shultz SJ. Balance recovers within 20 minutes after exertion as measured by the balance error scoring system. J Athl Train. 2004;39:241–246.
  34. Wilkins JC, Valovich McLeod TC, Perrin DH, Gansneder BM. Performance on the balance error scoring system decreases after fatigue. J Athl Train. 2004;39:156–161.
  35. Patel AV, Mihalik JP, Notebaert AJ, Guskiewicz KM, Prentice WE. Neuropsychological performance, postural stability, and symptoms after dehydration. J Athl Train. 2007;42:66–75.
  36. Weber AF, Mihalik JP, Register-Mihalik JK, Mays S, Prentice WE, Guskiewicz KM. Dehydration and performance on clinical concussion measures in collegiate wrestlers. J Athl Train. 2013;48:153–160. doi:10.4085/1062-6050-48.1.07 [CrossRef]

Combined Gender Data by Environment

DEPENDENT VARIABLEMEANSD95% CONFIDENCE INTERVAL

LOWERUPPER
Uncontrolled environment
  Symptom18.643.5517.3719.89
  Physical signs2.000.002.002.00
  Glasgow15.000.0015.0015.00
  Balance22.854.6921.1824.51
  Coordination1.000.001.001.00
  Orientation4.840.334.724.98
  Immediate memory3.631.063.003.72
  Concentration14.181.0113.6214.54
  Delay recall3.421.222.993.86
  Maddocks5.000.005.005.00
  Total83.096.8580.9485.25
Controlled environment
  Symptom21.062.4220.2121.92
  Physical signs2.000.002.002.00
  Glasgow15.000.0015.0015.00
  Balance28.031.4227.5328.54
  Coordination1.000.001.001.00
  Orientation4.890.334.764.99
  Immediate memory3.931.053.564.32
  Concentration14.720.5714.5414.93
  Delay recall2.811.532.763.61
  Maddocks5.000.005.005.00
  Total90.735.4788.5792.88
Authors

From St. Albans School, Washington, DC (DHS); and the School of Health Sciences and Human Performance, Lynchburg College, Lynchburg, Virginia (TGB, DAB).

The authors have no financial or proprietary interest in the materials presented herein.

Correspondence: Deanna H. Smith, MSEd, St. Albans School, 3001 Wisconsin Ave., Washington, DC 20015. E-mail: deannahelens@yahoo.com

Received: January 01, 2015
Accepted: January 18, 2016
Posted Online: May 16, 2016

10.3928/19425864-20160510-01

Sign up to receive

Journal E-contents