In the city of Greenville, North Carolina, the occurrence of sports-related concussions has been highly visible in the media, especially since the tragic death of a local high school football player from second-impact syndrome on September 4, 2008. In response to this tragedy, supervisors of Greenville schools, Pitt County School District, created a sports medicine team, which included a sports medicine director; certified athletic trainers at each high school; and an educational program that instructed students, coaches, and parents on the signs and symptoms of concussion, including neurological sequelae of concussion. Greenville schools also implemented baseline and postconcussive computerized neuropsychological testing (CNT) for high-risk student athletes. The school system’s decision to address the inadequacies of the care of student athletes was warranted and significantly improved the management of sports-related concussions. However, a recent review1 raised doubt about the utility of baseline CNT in modifying the risk of second-impact syndrome. Therefore, the role of CNT in the management of sports-related concussions has come into question.
According to the Zurich Consensus Statement,2 a concussion is defined as “a complex pathophysiological process affecting the brain, induced by traumatic biomechanical direct or indirect forces with rapid onset of short-lived symptoms and impairments, largely functional, with spontaneous resolution (in most cases) and normal neuroimaging studies.” The National Collegiate Athletic Association Concussion Study3 showed that on average, most athletes who sustain a concussion had their symptoms and cognitive impairment resolve within 7 days and balance deficits resolve within 5 days. The Consensus Statement2 also reenforced concussion as being a clinical diagnosis with 5 domains: symptoms, physical signs, behavioral, cognitive, and sleep disturbances, with CNT evaluating the cognitive domain. Computerized neuropsychological testing, at best, has an 82% sensitivity and 89% specificity,4 which does not make it a useful diagnostic tool for concussion.
The role of CNT, therefore, is in the management of concussions and in making return-to-play decisions. The Consensus Statement2 and the National Athletic Trainers’ Association (NATA)5 recommended that athletes should be asymptomatic with activities of daily living before return-to-play protocol implementation. Computerized neuropsychological testing should be proven as sensitive in detecting neurocognitive impairments after self-reported postconcussive symptoms have resolved. As with any neuropsychological test, the CNT should demonstrate adequate data to assess its test–retest reliability, validity, sensitivity, specificity, change score, and clinical utility.6 Evaluation of CNT should also be compared with the gold standard paper-and-pencil neuropsychological test. This article briefly reviews 4 commercially available CNTs: ImPACT™, CNS Vital Signs®, CogSport®, and HeadMinder™ CRI.
Test–retest reliability establishes the extent to which scores on the battery of the tests remain stable over time (ie, from baseline to postconcussion testing). Ideally, reliability should be close to 0.9; however, a value above 0.6 is the minimal acceptable test–retest reliability for clinical decision making, and the paper-and-pencil neuropsychological test reliability ranges from 0.8 to 0.4.7
The test–retest reliability of ImPACT over a 2-year period ranged from 0.74 to 0.43, with a processing speed, reaction time, and verbal memory being the most reliable composites.8 The CNS Vital Signs test–retest reliability over 62 days ranged from 0.88 to 0.65, with psychomotor speed, coding, and finger tapping being the most reliable domains.9 The test–retest reliability of CogSport over a 1-week period ranged from 0.82 to 0.31,10 with response speed in each domain being the most reliable measurement. The test–test reliability of HeadMinder CRI over a 2-week period was 0.82 to 0.68,11 with processing speed and simple reaction time being the most reliable indices.
Sensitivity and Specificity
Sensitivity and specificity demonstrate that a test can differentiate between patients with concussions and normal controls. Of the 4 CNTs evaluated, only ImPACT reports data on sensitivity and specificity for concussion in athletes. Schatz et al4 tested 72 high school student athletes within 72 hours of sustaining a concussion compared with 66 non-concussed athletes, demonstrating a sensitivity of 81.9% and specificity of 89.4%.
Validity assures that the test measures what it is purported to measure, which means that the CNTs in question are compared to paper-and-pencil tests, and both measure concussion-related neurocognitive deficits. A peer-reviewed article12 on ImPACT reports the validity of processing speed and reaction time composite only as compared with paper-and-pencil tests. A peer-reviewed article9 on CNS Vital Signs reports moderate correlation and validity when compared with traditional paper-and-pencil tests. The peer-reviewed article10 on CogSport reports positive correlation with 2 independent measures of processing speed only. The peer-reviewed article11 on HeadMinder CRI also reports expected concurrent validity with seven standardized tests.
Change score assesses a test’s stability over time and degree of variability from one test to another. This is a form of test–retest reliability that is composed of a combination of true variance, real changes in cognition over time, error variance, or changes in the test score attributable to flaws in the test itself. For example, if an athlete with concussion, currently asymptomatic, repeats CNT and scores 5 points below his or her baseline in a particular domain, is this change statistically significantly signifying cognitive deficits or just an inherent error in the test itself? The Reliable Change Index (RCI) is helpful in determining this change; an index score >1.96 demonstrates a statistically significant change, meaning the difference in the 2 scores is approximately twice the standard error of the test.13 Two peer-reviewed articles8,14 on ImPACT studied RCI and reliable change with consistent and reliable results. One study evaluated the change from baseline compared with the athlete’s in-season CNT; the other study had a 2-year interval in testing. Both CNS Vital Signs and CogSport have no published data on RCI or reliable change data. The peer-reviewed, well-designed study of HeadMinder reports reliable change-scores; however, the testing period was over the short interval of 2 weeks.11
Different aspects of the CNT will affect whether it can be applicable clinically. For example, if a test takes hours to complete and is cumbersome to administer, it would not be useful in concussion management. According to NATA’s position statement, in an effort to reduce practice effects, self-reported symptom resolution should be used as an indicator to begin neurocognitive testing. Therefore, CNTs should be proven sensitive in detecting neurocognitive deficits after self-reported concussive symptoms have resolved.
Currently, only ImPACT has assessed neurocognitive performance of athletes with concussions when they are asymptomatic. The study evaluated 21 NCAA college athletes, for which ImPACT was administered at baseline, when they were symptomatic (within 72 hours of injury) and when they were asymptomatic. During symptomatic assessment, 81% of athletes had deficits in at least one ImPACT variable, whereas 38% of asymptomatic concussed athletes continued to shows deficits of at least one ImPACT variable.15
Computerized neuropsychological testing provides valuable objective data about when to return to play, especially because other objective data, such as balance assessment, will often return to baseline prior to cognitive function.3
Baseline and Postconcussive Neuropsychological Testing
Another clinical question that needs to be answered is the frequency of baseline CNT readministration, timing, and frequency of postconcussion CNT evaluations. Currently, only ImPACT has long-term test–retest reliability of baseline cognitive assessment over a 2-year period; however, these data were collected in college athletes.4 A recent study16 of asymptomatic high school athletes using paper-and-pencil neuropsychological testing reported test scores to improve as the athletes matured and recommended yearly baseline CNTs.
No data currently are available to determine the best timing and frequency of postconcussion CNT evaluation. Using self-reported symptom resolution as an indicator to begin neurocognitive assessment can lead to an abnormal test, especially when 38% of asymptomatic postconcussion athletes have deficits in CNT. This can lead to repeating CNT multiple times, which could increase a learned effect and cost to the patient and the health care system.
Understanding the strengths and limitations of CNT allows for better interpretation of the results and management of athletes with concussions. Much of the published literature is on ImPACT and has addressed each of the 5 criteria for an adequate CNT. However, ImPACT’s test–retest reliability is the weakest of all the CNTs, with many of its domains below the 0.6 level considered significant for clinical decision making. CNS Vital Signs has excellent test–retest reliability and validity, but it lacks evidence in sensitivity and specificity, change score, and clinical utility. Both CogSport and HeadMinder CRI have good test–retest reliability and validity, but both lack evidence in sensitivity and specificity and clinical utility; however, HeadMinder CRI does have a reliable change score.
This article is not meant to promote one CNT system over another, but it is meant to present the most current data on these CNTs and how to reliably use them in the management of sports-related concussions. Just as the diagnosis of a concussion has 5 domains, so should the return-to-play decision, with CNT providing objective data in the assessment of the athlete.
If baseline CNT is used, it subsequently should be done on a yearly basis in high school athletes and every 2 years for college athletes depending on the CNT in use. When an athlete becomes concussed, he or she should be evaluated with a Sport Concussion Assessment Tool 2 (SCAT2) and a comprehensive physical examination. If a baseline CNT has not been obtained or if baseline CNT is invalid, one should be obtained for comparison purposes when the athlete becomes asymptomatic. When a concussed athlete is asymptomatic with activity of daily living, he or she should progress through the Zurich Consensus Statement’s2 graduated return-to-play protocol, and before starting full-contact practice a CNT should be administered to obtain objective data to guide the health care provider regarding return-to-play decisions. This allows for minimal testing and allows physical exercise to elicit symptoms prior to CNT administration.
- Randolph C. Baseline neuropsychological testing in managing sport-related concussion: does it modify risk?Curr Sports Med Rep. 2011;10(1):21–26.
- McCrory P, Meeuwisse W, Johnston K, et al. Consensus statement on concussion in sport—the Third International Conference on Concussion in Sport held in Zurich, November 2008. Clin J Sport Med. 2009;19(3):185–200. doi:10.1097/JSM.0b013e3181a501db [CrossRef]
- McCrea M, Guskiewicz KM, Marshall S, et al. Acute effects and recovery time following concussion in collegiate football players: the NCAA concussion study. JAMA. 2003;290(19):2556–2563. doi:10.1001/jama.290.19.2556 [CrossRef]
- Schatz P, Pardini JE, Lovell MR, Collins MW, Podell K. Sensitivity and specificity of the ImPACT test battery for concussion in athletes. Arch Clin Neuropsychol. 2006;21(1):91–99. doi:10.1016/j.acn.2005.08.001 [CrossRef]
- Guskiewicz KM, Bruce SL, Cantu RC, et al. National Athletic Trainers’ Association position statement: management of sport-related concussion. J Athl Train. 2004;39(3):280–297.
- Randolph C, McCrea M, Barr WB. Is neuropsychological testing useful in the management of sport-related concussion?J Athl Train. 2005;40(3):139–154.
- Nunnally JC, Bernstein IH. Psychometric Theory. 3rd ed. New York, NY: McGraw-Hill; 1994.
- Schatz P. Long-term test-retest reliability of baseline cognitive assessments using ImPACT. Am J Sports Med. 2010;38(1):47–53. doi:10.1177/0363546509343805 [CrossRef]
- Gualtieri TC, Johnson LG. Reliability and validity of a computerized neurocognitive test battery, CNS Vital Signs. Arch Clin Neuropsychol. 2006;21(7):623–643. doi:10.1016/j.acn.2006.05.007 [CrossRef]
- Collie A., Maruff P, Makdissi M, McCrory P, McStephen M, Darby D. CogSport: reliability and correlation with conventional cognitive tests used in postconcussion medical evaluations. Clin J Sports Med. 2003;13(3):28–32. doi:10.1097/00042752-200301000-00006 [CrossRef]
- Erlanger DM, Feldman DJ, Kuther K, et al. Development and validation of a web-based neuropsychological test protocol for sports-related return-to-play decision-making. Arch Clin Neuropsychol. 2003;18(3):293–316.
- Iverson GL, Lovell MR, Collins MW. Validity of ImPACT for measuring processing speed following sports-related concussion. J Clin Exper Neuropsychol. 2005;27(6):683–689. doi:10.1081/13803390490918435 [CrossRef]
- Collie A, Maruff P, Makdissi M, McStephen M, Darby DG, McCrory P. Statistical procedures for determining the extent of cognitive change following concussion. Br J Sports Med. 2004;38(3):273–278. doi:10.1136/bjsm.2003.000293 [CrossRef]
- Iverson GL, Lovell MR, Collins MW. Interpreting change on ImPACT following sport concussion. Clin Neuropsychol. 2003;17(4):460–467. doi:10.1076/clin.17.4.460.27934 [CrossRef]
- Broglio SP, Macciocchi SN, Ferrara MS. Neurocognitive performance of concussed athletes when symptom free. J Athl Train. 2007;42(4):504–508.
- Hunt TH, Ferrera MS. Age-related differences in neuropsychological testing among high school athletes. J Athl Train. 2009;44(4):405–409. doi:10.4085/1062-6050-44.4.405 [CrossRef]