Feature

Computerized Neurocognitive Testing in the Medical Evaluation of Sports Concussion

David G. Darby, MBBS, PhD, FRACP; Christina L. Master, MD; Matthew F. Grady, MD

  • Pediatric Annals
  • September 2012 - Volume 41 · Issue 9: 371, 374-376
  • DOI: 10.3928/00904481-20120827-11
Rights and Permissions

In some individuals, return to normal function after a concussion occurs quickly and is easy to assess. In these individuals, neuropsychological testing is not necessary and adds little additional information to the clinical exam. In more difficult circumstances, such as a timely return to a contact or collision sport, or prolonged severe symptoms, additional information may be helpful to the clinician. In these circumstances, the addition of computerized neurocognitive testing offers practical advantages for detection of subtle or profound cognitive impairment, especially if compared with a valid baseline. However, as with any tool, there are also important limitations to its proper use and interpretation. Since there are insufficient neuropsychologists to provide regular monitoring of every concussion, pediatricians need to be aware of the utility, advantages and limitations of computerized neurocognitive testing. It is but one additional component of the management armamentarium that can aid their clinical management of the evolution and resolution of their patient’s impairment from concussion.

The medical evaluation of concussion involves a comprehensive assessment of brain function, including the vestibulo-ocular system, balance, mood, and cognitive function. Traditional clinical exam techniques to evaluate cognitive function are limited. Sideline assessment tools such as the Sport Concussion Assessment Tool 2 (SCAT2) have been designed for sideline evaluation of concussion but have limitations beyond the acute concussion period. The development of in-office computerized neuropsychological testing provides clinicians with an additional tool to assess the cognitive effects of head injuries.

Formal neuropsychological testing has been the gold standard in documenting deficits in cognitive function and is useful in detecting deficits following sports-related concussions.1 This type of testing measures cognitive impairment on a single occasion and measures deficits in different cognitive domains (eg, attention, memory, language, visuo-spatial, and executive function) to inform diagnosis and rehabilitation decisions. It is particularly helpful in documenting deficits that are fixed or stable. Such formal testing is not intended to measure transient neurocognitive changes over time such as those seen in concussion.2

Formal neuropsychological testing is optimally performed by expert neuropsychologists, who supplement clinical observational skills with standardized psychometric techniques.3 However, traditional neuropsychological “pencil and paper testing” is both time-consuming and labor-intensive, and not suitable for high frequency testing over days or weeks.4 Concussion-related cognitive impairment typically evolves and resolves over days and weeks and is often subtle, so testing is only practical and clinically useful if it can be done quickly and repeated frequently.5

Computerized neurocognitive testing, which is easily administered and takes only about 12 to 30 minutes to complete, has become a popular tool to evaluate transient neurologic deficits; however, it should not be considered equivalent to a full neuropsychological evaluation. Commercial products are used routinely at the college and professional level. There has also been a recent increase in testing at the high school level, where most sports concussions occur. Although commercial testing is available for adolescents, testing for pediatric patients (ages 5–11 years) requires different testing techniques which are currently under development.

Different approaches to testing are also available, with some programs adopting nontraditional approaches that emphasize the measurement of reaction time and change over time as global indicators of the magnitude of impairment, whereas other programs utilize domain-specific impairments in verbal and visual memory in addition to reaction time and visual motor speed.6,7 The reliability of these computerized tests has been debated with regard to both theoretical “psychometric” issues as well as practical ones.2,6,8–12

As long as the principles underpinning such tests are understood, they can be applied rationally to complement, but not supplant, the clinical evaluation of concussion. In general, the aim of such testing is to reduce sources of error that can in turn increase the variability of test performance and decrease the ability to detect true or subtle neurological change on repeated testing.3,13–15

Principles of Cognitive Testing

Cognitive evaluation, including both traditional “pencil and paper” and computerized testing, can be affected by a number of environmental factors and individual states, including level of arousal, mood, physical or mental fatigue, patient effort and even bladder volume.2,16,17 Scores may be lower than expected due to these suboptimal environmental or patient conditions, and not reflect actual cognitive impairments.

During formal testing, experienced neuropsychologists may observe these factors and may make allowances in the interpretation of results, but this accommodation is not possible with self-administered computerized testing. In addition, although group testing is possible with computerized testing, as evidenced by some large college programs that have used self-administered computer laboratory testing effectively,18,19 the advantage of mass testing (ie, an entire sports team testing in one computer lab) may be compromised by higher rates of suboptimal performance in certain environments.20

Evaluation Environments

Testing environments should facilitate optimal concentration and minimize distractions, particularly from communication devices and physical interaction with others. Noise-canceling headphones may be useful. Certain environments, such as the sidelines of sporting fields where there are multiple distractions as well as acute physical fatigue, may make optimal performance unlikely.

Practice Effect

Traditional neuropsychological testing is not intended to follow transient cognitive changes, so a “practice effect” is of concern when multiple tests are given over a short period of time. Practice effects occur when an individual “learns” how to take the test and improvements in test performance are not due to improvements in cognitive function, but rather reflect improvements in test-taking.

The testing design must aim to minimize these effects on repeated testing since they introduce unwanted complexities into the interpretation of results. The magnitude of practice effect can vary in complex ways relating to the type of test, inter-test interval and number of tests performed,4 as well as to the population being assessed.13 The magnitude of practice effect in healthy controls, including children, may not correlate with the effect seen in those with cognitive impairment.21

Even after concussion, practice effect may occur in some tests but not others.13 Interpretation of a test that is assumed to have practice effects in healthy controls with repeated administration is doubly problematic after a concussion since a test that is “back to baseline” could still reflect abnormality due to a “failure to improve” as expected. These are important factors to account for when making return-to-play decisions, although they can be minimized by optimal test design.5,22

Interpretation of Results

Another important issue is the interpretation of tests in individuals at the extremes of the testing ranges. Athletes who are of superior testing ability will perform well above the mean. Even when they are impaired, their scores may appear normal (at or above the mean) after a concussion. Because most tests do not have normative data for repeat testing, interpreting results from testing done over short time intervals of 1 or 2 days may be difficult.4,5 Furthermore, these tests are not generally designed to be given repeatedly without suffering practice and ceiling effects.

For repeated administration, measurement of the variability in neurologically stable individuals is required so that statistical measures of change can be applied.23 In addition, intra-individual variability is generally much smaller than inter-individual or population-based variability. If a clinician is attempting to determine whether cognitive impairment has occurred in an individual following a concussion, comparison to the individual’s own baseline is likely to be more informative than a decline that remains within a population-based range that might be interpreted as still “within normal limits.”

However, all tests show some variability when repeated in the same individual. This variability can be measured and used to determine the range of normal variability based on a statistical threshold at which scores beyond the threshold are unlikely to have occurred by chance. In addition, normal variability in self-reported “asymptomatic” athletes may also lead to “abnormal” tests on single occasions.24,25

Baseline Testing

Baseline testing is useful if comparisons to the individual’s own abilities are to be made following injury. However, baseline testing must be optimal, to accurately reflect the athlete’s pre-injury performance.17 Methods to assess whether or not an athlete has performed a best effort have been described,19 but this remains an area requiring further research.

In competitive sports, there is the additional problem of misguided athletes who believe that under-performing at baseline will allow their impaired post-injury performances to go undetected (“faking bad” or “sand-bagging”).17 Criteria for detecting suboptimal performances can involve many different techniques including seeking incomplete, abnormally slow, or inaccurate performances, including at or worse than chance; or harder tasks performed faster than easier ones.19,26

It is important that the benefits of baseline testing are communicated and coupled with the positive outcome of facilitating safe return to play after injury. Testing systems that incorporate integrity checks and require them to be passed for a “valid” baseline at the time of testing are available.19

Repeated Baseline Testing

Another issue is the frequency of repeated baseline testing. The advantages of repeating a baseline at least annually include resetting the baseline to account for maturational improvements in children without which there might be an underestimate of the severity of concussion-related impairment.27 Annual testing also allows the monitoring of cumulative impairment or new unsuspected impairment.

Further research is required in children since some age ranges are associated with rapid cognitive improvements as seen in cross-sectional studies,27–29 suggesting that multiple tests within a year might be more accurate for comparisons at these ages.27 A possible future approach may include longitudinal trajectories for children with a comparison of performance with established baseline percentiles.

Appropriate Cognitive Assessments

A further concern is the specific tasks and abilities that should be assessed after concussion.30–37 The overarching aim of testing should be to inform clinical decision-making, with the choice of technique based upon the clinician’s questions at that time. Assessments within minutes or hours of an injury are primarily concerned with clinical features (symptoms and signs) as assessed using standardized batteries such as the Standardized Assessment of Concussion (SAC)31 or more recent SCAT2,38,39 which incorporate elementary cognitive testing. These are simple attentional, registration, or memory tests for gross impairment.

Computerized tests at this stage may not add much information if symptoms are disclosed and physical examination deficits detected. Results of computerized testing are typically more abnormal in symptomatic concussed athletes vs. asymptomatic concussed athletes36 and may be useful in demonstrating the relative severity of deficits.

Several days after a concussion, decisions about residual impairment are relevant to the clinician’s decision regarding return to activity.38 Tests that can be repeated frequently (even daily) have advantages in tracking the time-course of the athlete’s recovery. Symptom resolution may occur before or after resolution of cognitive test deficits, so both must be evaluated when making return-to-play decisions.

The clinician should note that deteriorations on cognitive tests may not necessarily be due to neurologic impairment after concussion, since changes can occur with physical fatigue, poor effort, mood and psychological disorders, alcohol and drug usage, systemic and neurological conditions, so test results must always be considered within the broader clinical context.

Conclusion

Pediatricians will encounter more children with concussion in their practices as states implement return-to-play legislation and public awareness of concussion and its possible consequences increase. Pediatricians have the advantage of providing a medical home in which to manage the myriad clinical features of concussions and may include ordering other investigations, repeat clinical follow-up, specialist referrals, or rehabilitation therapies.

In practical terms, there are not sufficient sports physicians or neurologists to manage every concussion, so pediatricians need to be familiar with the return-to-school and return-to-play issues for student-athletes. As part of their assessment, evaluation and documentation of return-to-normal cognitive function is an important component of a comprehensive evaluation.

References

  1. Echemendia RJ, Putukian M, Mackin RS, Julian L, Shoss N. Neuropsychological test performance prior to and following sports-related mild traumatic brain injury. Clin J Sport Med. 2001;11:23–31. doi:10.1097/00042752-200101000-00005 [CrossRef]
  2. Broglio SP, Ferrara MS, Macciocchi SN, Baumgartner TA, Elliott R. Test-retest reliability of computerized concussion assessment programs. J Athl Train. 2007;42:509–514.
  3. Collie A, Maruff P. Computerised neuropsychological testing. Br J Sports Med. 2003;37:2–3. doi:10.1136/bjsm.37.1.2 [CrossRef]
  4. Bartels C, Wegrzyn M, Wiedl A, Ackermann V, Ehrenreich H. Practice effects in healthy adults: A longitudinal study on frequent repetitive cognitive testing. BMC Neuroscience. 2010;11:118. doi:10.1186/1471-2202-11-118 [CrossRef]
  5. Falleti MG, Maruff P, Collie A, Darby DG. Practice effects associated with the repeated assessment of cognitive function using the CogState battery at 10-minute, one week and one month test-retest intervals. J Clin Exp Neuropsychol. 2006;28:1095–1112. doi:10.1080/13803390500205718 [CrossRef]
  6. Collie A, Maruff P, Makdissi M, McCrory P, McStephen M, Darby DG. CogSport: reliability and correlation with conventional cognitive tests used in post-concussion medical examinations. Clin J Sport Med. 2003;13:28–32. doi:10.1097/00042752-200301000-00006 [CrossRef]
  7. Eckner JT, Kutcher JS, Richardson JK. Between-seasons test-retest reliability of clinically measured reaction time in national collegiate athletic association division I athletes. J Athl Train. 2011;46:409–414.
  8. Randolph C, McCrea M, Barr WB. Is neuropsychological testing useful in the management of sport-related concussion?J Athl Train. 2005;40:139–152.
  9. Iverson GL, Lovell MR, Collins MW. Interpreting change on ImPACT following sport concussion. Clin Neuropsychol. 2003;17:460–467. doi:10.1076/clin.17.4.460.27934 [CrossRef]
  10. Grindel SH, Lovell MR, Collins MW. The assessment of sport-related concussion: the evidence behind neuropsychological testing and management. Clin J Sport Med. 2001;11:134–143. doi:10.1097/00042752-200107000-00003 [CrossRef]
  11. Iverson GL, Lovell MR, Collins MW. Validity of ImPACT for measuring processing speed following sports-related concussion. J Clin Exp Neuropsychol. 2005;27:683–689. doi:10.1081/13803390490918435 [CrossRef]
  12. Halstead ME, Walter KD. American Academy of Pediatrics. Clinical report--sport-related concussion in children and adolescents. Pediatrics. 2010;126:597–615. doi:10.1542/peds.2010-2005 [CrossRef]
  13. Makdissi M, Collie A, Maruff P, et al. Computerised cognitive assessment of concussed Australian Rules footballers. Br J Sports Med. 2001;35:354–360. doi:10.1136/bjsm.35.5.354 [CrossRef]
  14. Makdissi M, Darby D, Maruff P, et al. Natural history of concussion in sport: markers of severity and implications for management. Am J Sports Med. 2010;38:464–471. doi:10.1177/0363546509349491 [CrossRef]
  15. Collie A, Darekar A, Weissgerber G, et al. Cognitive testing in early-phase clinical trials: development of a rapid computerized test battery and application in a simulated Phase I study. Contemp Clin Trials. 2007;28:391–400. doi:10.1016/j.cct.2006.10.010 [CrossRef]
  16. Lewis MS, Snyder PJ, Pietrzak RH, et al. The effect of acute increase in urge to void on cognitive function in healthy adults. Neurourol Urodyn. 2011;30:183–187. doi:10.1002/nau.20963 [CrossRef]
  17. Hunt TN, Ferrara MS, Miller LS, Macciocchi S. The effect of effort on baseline neuropsychological test scores in high school football athletes. Arch Clin Neuropsychol. 2007;22:615–621. doi:10.1016/j.acn.2007.04.005 [CrossRef]
  18. Moriarity J, Collie A, Olson D, et al. A prospective controlled study of cognitive function during an amateur boxing tournament. Neurology. 2004;62:1497–1502. doi:10.1212/WNL.62.9.1497 [CrossRef]
  19. Moriarity JM, Pietrzak RH, Kutcher JS, et al. Unrecognised ringside concussive injury in amateur boxers. Br J Sports Med. 2012 [ePub ahead of print]. doi:10.1136/bjsports-2011-090893 [CrossRef] . doi:10.1136/bjsports-2011-090893 [CrossRef]
  20. Moser RS, Schatz P, Neidzwski K, Ott SD. Group versus individual administration affects baseline neurocognitive test performance. Am J Sports Med. 2011; 39(11):2325–2330. doi:10.1177/0363546511417114 [CrossRef]
  21. Darby D, Maruff P, Collie A, McStephen M. Mild cognitive impairment can be detected by multiple assessments in a single day. Neurology. 2002;59:1042–1046. doi:10.1212/WNL.59.7.1042 [CrossRef]
  22. Maruff P, Collie A, Falleti MG, et al. Quantifying the cognitive impairment associated with concussion: using blood alcohol concentration as a reference point. Br J Sports Med. 2001;35:374.
  23. Collie A, Maruff P, Makdissi M, McStephen M, Darby DG, McCrory P. Statistical procedures for determining the extent of cognitive change following concussion. Br J Sports Med. 2004;38:273–278. doi:10.1136/bjsm.2003.000293 [CrossRef]
  24. Van Kampen DA, Lovell MR, Pardini JE, Collins MW, Fu FH. The “value added” of neurocognitive testing after sports-related concussion. Am J Sports Med. 2006;34:1630–1635. doi:10.1177/0363546506288677 [CrossRef]
  25. Broglio SP, Macciocchi SN, Ferrara MS. Neurocognitive performance of concussed athletes when symptom free. J Athl Train. 2007;42:504–508.
  26. Fredrickson J, Maruff P, Woodward M, et al. Evaluation of the usability of a brief computerized cognitive screening test in older people for epidemiological studies. Neuroepidemiology. 2010;34:65–75. doi:10.1159/000264823 [CrossRef]
  27. McCrory P, Collie A, Anderson V, Davis G. Can we manage sport related concussion in children the same as in adults?Br J Sports Med. 2004;38:516–519. doi:10.1136/bjsm.2004.014811 [CrossRef]
  28. Betts J, McKay J, Maruff P, Anderson V. The development of sustained attention in children: the effect of age and task load. Child Neuropsychol. 2006;12:205–221. doi:10.1080/09297040500488522 [CrossRef]
  29. Kirkwood MW, Randolph C, Yeates KO. Returning pediatric athletes to play after concussion: the evidence (or lack thereof) behind baseline neuropsychological testing. Acta Paediatr. 2009;98:1409–1411. doi:10.1111/j.1651-2227.2009.01448.x [CrossRef]
  30. Brown CN, Guskiewicz KM, Bleiberg J. Athlete characteristics and outcome scores for computerized neuropsychological assessment: a preliminary analysis. J Athl Train. 2007;42:515–523.
  31. McCrea M, Kelly JP, Kluge J, Ackley B, Randolph C. Standardized assessment of concussion in football players. Neurology. 1997;48:586–588. doi:10.1212/WNL.48.3.586 [CrossRef]
  32. McCrea M, Prichep L, Powell MR, Chabot R, Barr WB. Acute effects and recovery after sport-related concussion: a neurocognitive and quantitative brain electrical activity study. J Head Trauma Rehabil. 2010;25:283–292. doi:10.1097/HTR.0b013e3181e67923 [CrossRef]
  33. Barr WB. Neuropsychological testing of high school athletes. Preliminary norms and test-retest indices. Arch Clin Neuropsychol. 2003;18:91–101.
  34. Segalowitz SJ, Mahaney P, Santesso DL, MacGregor L, Dywan J, Willer B. Retest reliability in adolescents of a computerized neuropsychological battery used to assess recovery from concussion. NeuroRehabilitation. 2007;22:243–251.
  35. Collie A, Darby D, Maruff P. Computerised cognitive assessment of athletes with sports related head injury. Br J Sports Med. 2001;35:297–302. doi:10.1136/bjsm.35.5.297 [CrossRef]
  36. Collie A, Makdissi M, Maruff P, et al. Post-concussion cognitive function in symptomatic and asymptomatic athletes. Paper presented at: Second International Conference on Concussion in Sport. ; November 2004. ; Prague. .
  37. Collie A, Makdissi M, Maruff P, Bennell K, McCrory P. Cognition in the days following concussion: comparison of symptomatic versus asymptomatic athletes. J Neurol Neurosurg Psychiatry. 2006;77:241–245. doi:10.1136/jnnp.2005.073155 [CrossRef]
  38. McCrory P, Meeuwisse W, Johnston K, et al. Consensus statement on Concussion in Sport 3rd International Conference on Concussion in Sport held in Zurich, November 2008. Clin J Sport Med. 2009;19:185–200. doi:10.1097/JSM.0b013e3181a501db [CrossRef]
  39. McCrory P. Sport concussion assessment tool 2. Scand J Med Sci Sports. 2009;19:452. doi:10.1111/j.1600-0838.2009.00978.x [CrossRef]
AUTHORS

David G. Darby, MBBS, PhD, FRACP, is a Behavioral Neurologist at The Florey Neuroscience Institutes and Mental Health, University of Melbourne, Parkville VIC, Australia. Christina L. Master, MD, is the Associate Program Director, Primary Care Sports Medicine Fellowship, The Children’s Hospital of Philadelphia; and Associate Professor of Clinical Pediatrics, Departments of Surgery and Pediatrics, Division of Pediatric Orthopedics and Sports Medicine, Perelman School of Medicine at the University of Pennsylvania. Matthew F. Grady, MD, is the Director, Primary Care Sports Medicine Fellowship, The Children’s Hospital of Philadelphia; and Assistant Professor of Clinical Pediatrics, Departments of Surgery and Pediatrics, Division of Pediatric Orthopedics and Sports Medicine, Perelman School of Medicine at the University of Pennsylvania.

Address correspondence to: David G. Darby, MBBS, PhD, FRACP, The Florey Institute of Neuroscience and Mental Health, Level 3, Alan Gilbert Building, 161 Barry Street, University of Melbourne, Parkville VIC 3010, Australia; email: ddarby@unimelb.edu.au.

Disclosure: Dr. Darby is a founder, shareholder, and now consultant to CogState Ltd, which commercializes the Axon Sports computerized testing products. Drs. Master and Grady have no relevant financial relationships to disclose.

doi: 10.3928/00904481-20120827-11

Healio is intended for health care provider use and all comments will be posted at the discretion of the editors. We reserve the right not to post any comments with unsolicited information about medical devices or other products. At no time will Healio be used for medical advice to patients.

[X]