Athletic Training and Sports Health Care

Original Research 

Reliability and Validity of a New Questionnaire Created to Establish the Presence of Functional Ankle Instability: The IdFAI

Matthew Donahue, PhD, ATC; Janet Simon, MS, ATC; Carrie L. Docherty, PhD, ATC, FNATA

Abstract

The purpose of this investigation was to examine the consistency (reliability) and accuracy (validity) of a new ankle instability questionnaire—the Identification of Functional Ankle Instability (IdFAI). One hundred ten participants were asked to complete the IdFAI on 2 occasions and the Lower Extremity Functional Screen (LEFS) on 1 occasion. Test–retest reliability was evaluated by intraclass correlation coefficient (ICC2,1). Convergent validity was evaluated by comparing the IdFAI with the LEFS using Spearman’s rho (ρ). The dependent variables were the scores on the IdFAI and the LEFS. Test–retest reliability ranged from 0.81 to 0.94 for the questionnaire factors and was 0.92 for the overall questionnaire. Results of validity testing identified a statistically significant correlation between the IdFAI and LEFS (ρ = −0.38, P < .01). The IdFAI is a simple, valid, reliable questionnaire that can be used to categorize an individual’s FAI status.

Dr Donahue is from the Department of Health Promotion and Human Performance, Weber State University, Ogden, Utah; Ms Simon and Dr Docherty are from the Department of Kinesiology, Indiana University, Bloomington, Indiana.

The authors have no financial or proprietary interest in the materials presented herein.

Address correspondence to Carrie L. Docherty, PhD, ATC, FNATA, Department of Kinesiology, Indiana University, 1025 E. 7th Street, Bloomington, IN 47404; e-mail: cdochert@indiana.edu.

Received: February 07, 2012
Accepted: June 12, 2012
Posted Online: December 12, 2012

Abstract

The purpose of this investigation was to examine the consistency (reliability) and accuracy (validity) of a new ankle instability questionnaire—the Identification of Functional Ankle Instability (IdFAI). One hundred ten participants were asked to complete the IdFAI on 2 occasions and the Lower Extremity Functional Screen (LEFS) on 1 occasion. Test–retest reliability was evaluated by intraclass correlation coefficient (ICC2,1). Convergent validity was evaluated by comparing the IdFAI with the LEFS using Spearman’s rho (ρ). The dependent variables were the scores on the IdFAI and the LEFS. Test–retest reliability ranged from 0.81 to 0.94 for the questionnaire factors and was 0.92 for the overall questionnaire. Results of validity testing identified a statistically significant correlation between the IdFAI and LEFS (ρ = −0.38, P < .01). The IdFAI is a simple, valid, reliable questionnaire that can be used to categorize an individual’s FAI status.

Dr Donahue is from the Department of Health Promotion and Human Performance, Weber State University, Ogden, Utah; Ms Simon and Dr Docherty are from the Department of Kinesiology, Indiana University, Bloomington, Indiana.

The authors have no financial or proprietary interest in the materials presented herein.

Address correspondence to Carrie L. Docherty, PhD, ATC, FNATA, Department of Kinesiology, Indiana University, 1025 E. 7th Street, Bloomington, IN 47404; e-mail: cdochert@indiana.edu.

Received: February 07, 2012
Accepted: June 12, 2012
Posted Online: December 12, 2012

Functional ankle instability (FAI) has been widely investigated, yet there remains to be no benchmark measure or universally accepted definition of this pathology. Although numerous studies1–7 have documented the lasting deficits that exist following an initial ankle sprain, it remains difficult to identify a homogeneous subset of the population who can be classified as having FAI. The current standard of practice by many researchers is to use a self-report questionnaire to determine an individual’s incidents of instability. Several self-reported questionnaires have been published in the past decade.8–15 These include the Ankle Instability Instrument (AII), Ankle Joint Functional Assessment Tool, Chronic Ankle Instability Scale, Cumberland Ankle Instability Tool (CAIT), Foot and Ankle Ability Measure (FAAM), Foot and Ankle Instability Questionnaire, and Foot and Ankle Outcome Score. An investigation of these widely used questionnaires found that no singular instrument could accurately predict whether an individual meets a minimally accepted criteria for FAI.16 In addition, although many of these questionnaires have established test–retest reliability and are frequently reported in the literature, only limited validity testing has been conducted.

The Identification of Functional Ankle Instability (IdFAI) is a newly published questionnaire that was specifically designed to detect whether individuals meet a minimum criteria necessary for inclusion in an FAI population.17 The IdFAI is intended to give both researchers and clinicians a simple and effective tool to determine an individual’s ankle stability status. The IdFAI is based on 2 previous FAI instruments: the CAIT and the AII. The underlying concept of the IdFAI is to consolidate the elements of each instrument and combine them in a manner that results in a simple and concise means to identify individuals with FAI. One of the main elements included in the IdFAI, which is not in any other questionnaire, is a specific definition of giving way. This definition was provided to ensure that all individuals understood the term and answered questions based on the same definition. The definition included in the questionnaire is: “‘Giving way’ is described as a temporary uncontrollable sensation of instability or rolling over of one’s ankle.” This definition was based on the work originally conducted by Freeman,18,19 as well as on current literature.20

The IdFAI was tested, and the final items appearing on the questionnaire are a result of an exploratory factor analysis.17 Items on the questionnaire can be grouped into 3 factors—factor 1 focuses on the history of ankle instability, factor 2 centers around information related to the initial ankle sprain, and factor 3 contains information about instability during activities of daily living (ADL). Following the development of the IdFAI, the next step was to test the consistency (reliability) and accuracy (validity) of this questionnaire. Therefore, the purpose of this investigation was to establish the reliability and validity of the IdFAI questionnaire.

Methods

Participants

College-aged students were recruited from classroom settings at a large Midwestern university. One hundred ten participants volunteered to participate in the study. Only information about the dominant limb was used when completing the questionnaires. Limb dominance was established by asking participants “Which leg would you prefer to kick a ball with?” All participants gave informed consent, and the university institutional review board approved this study.

Procedures

Participants were asked to complete the IdFAI on 2 separate occasions 14 days apart. Participants were also asked to complete the Lower Extremity Functional Scale (LEFS) during the first testing session. The IdFAI is a 10-question form that focuses specifically on questions related to FAI. The questionnaire and the specific score rubric is shown in Figure 1.17 A total score of 10 or lower indicates that the participant is unlikely to have FAI, whereas a total score of 11 or higher indicates that a participant is likely to have FAI.17

The Identification of Functional Ankle Instability (IdFAI) questionnaire, including scoring rubric.Reprinted with permission from Simon J, Donahue M, Docherty C. Development of the Identification of Functional Ankle Instability (IdFAI). Foot & Ankle International. 33(9):755–763, © 2012 by the American Orthopaedic Foot and Ankle Society, Inc.

Figure 1. The Identification of Functional Ankle Instability (IdFAI) questionnaire, including scoring rubric.Reprinted with permission from Simon J, Donahue M, Docherty C. Development of the Identification of Functional Ankle Instability (IdFAI). Foot & Ankle International. 33(9):755–763, © 2012 by the American Orthopaedic Foot and Ankle Society, Inc.

To test validity, we used the same methods as Hiller et al11 and compared the IdFAI score with a lower limb reference standard—the LEFS. The LEFS is a widely used questionnaire that measures overall lower extremity function, as well as a wide range of lower extremity disabilities. The LEFS has a maximum score of 80, signifying no disability, and a lower score indicates greater disability.21 Because no benchmark for FAI exists, the LEFS is meant to evaluate convergent validity. This type of validity is tested by evaluating whether a measurement is similar to (converges on) another measurement that has already been validated.

All data collection was done in the classroom setting. An investigator (M.D., J.S., or C.L.D.) was present during all testing sessions and made certain that no outside distractions occurred during the testing period. The investigator was also present to answer any participant questions, but additional information was rarely required. Individuals were allowed as much time as necessary to complete the survey, but participants usually finished in approximately 10 minutes.

Statistical Analysis

Test–retest reliability was evaluated using intraclass correlation coefficients (ICC2,1) for each item, each factor, and the total score on the IdFAI between test days 1 and 2. Cronbach’s alpha was calculated to estimate internal consistency of the items. Validity was evaluated by using Spearman’s rho (ρ) to compare the IdFAI and LEFS.

Results

Fifty-four men (49%) and 56 women (51%) were included in the study. The average age was 19.80±1.40 years. The majority of the individuals (n = 98, 89.1%) had right limb dominance, and the remaining individuals had left limb dominance (n = 12, 10.9%). Seventy-nine (71.8%) individuals had a history of an ankle sprain and 31 (28.2%) did not have a history of ankle injury. Of the previously injured participants, 30 (37.9%) had sprained the right ankle only, 10 (12.7%) had sprained the left ankle only, and 39 (49.4%) had an ankle sprain on both ankles.

Reliability

Reliability of the individual factors was 0.81 (standard error of measurement [SEM] = 2.21) for history factor (factor 1), 0.94 (SEM = 1.06) for initial ankle sprain factor (factor 2), 0.83 (SEM = 1.06) for instability during ADL factor (factor 3), and 0.92 (SEM = 2.76) for the overall questionnaire. Individual question reliability is presented in Table 1. The Cronbach’s alpha coefficients were 0.89, 0.97, and 0.91 for the 3 factors respectively, and 0.96 for the overall instrument.

Reliability Statistics of the Identification of Functional Ankle Instability (IdFAI)

Table 1: Reliability Statistics of the Identification of Functional Ankle Instability (IdFAI)

Between the 2 test occasions, there was exact agreement for 26 (23.6%) participants, and an additional 64 (58.2%) participants differed by only 1 or 2 points. An additional 13 (11.8%) participants differed by 3 points, 6 (5.5%) by 4 points, and 1 (0.91%) by 10 points. A Bland-Altman plot indicates that reliability of the IdFAI score did not change systematically (Figure 2).

Bland-Altman plot showing the average of the 2 Identification of Functional Ankle Instability (IdFAI) test occasions and the difference between the 2 tests (n = 110). The solid line represents the mean difference between the 2 test occasions, and the dashed line represents 1 standard deviation of the difference between the 2 test occasions.

Figure 2. Bland-Altman plot showing the average of the 2 Identification of Functional Ankle Instability (IdFAI) test occasions and the difference between the 2 tests (n = 110). The solid line represents the mean difference between the 2 test occasions, and the dashed line represents 1 standard deviation of the difference between the 2 test occasions.

Validity

A statistically significant correlation was noted between the LEFS and the IdFAI (ρ = −0.38, P < .01) (Table 2). The correlations between the LEFS and history and initial ankle sprain factors on the IdFAI were weak (ρ = −0.29, P < .01; and ρ = −0.24, P < .01, respectively). However, a moderate correlation was identified between the LEFS and instability during ADL factor on the IdFAI, with ρ = −0.41 (P < .01).

Correlation Between the IdFAI and the LEFS

Table 2: Correlation Between the IdFAI and the LEFS

Discussion

In 2002, the Scientific Advisory Committee of Medical Outcomes Trust22 published a statement on the creation and review criteria for assessing health status and quality-of-life instruments. This statement highlighted the importance of 8 principal foci for these instruments, 2 of which indicated that any new instrument should establish reliability and validity.22 Without a reliable and valid tool to consistently identify individuals with similar symptoms, researchers and clinicians may inappropriately classify individuals with FAI. To date, of the numerous self-report questionnaires that have been published to identify individuals with FAI, only 4 have reported both reliability and validity information.11,13,15,23

Reliability

In our study, the IdFAI clearly demonstrated overall excellent test–retest reliability (ICC2,1= 0.92) in a sample of 110 independent limbs. The history of ankle instability factor achieved good reliability (ICC2,1 = 0.81), the initial ankle sprain factor achieved excellent reliability (ICC2,1 = 0.94), and the instability during ADL factor achieved good reliability (ICC2,1 = 0.83). High reliability scores indicate that the IdFAI will be a valuable tool in both the clinical and research settings, providing confidence to researchers and clinicians that individuals will consistently answer the questions in a similar manner.

The IdFAI has similar test–retest reliability when compared with other published instruments. For example, the AII has good reliability (ICC2,1 = 0.70 to 0.89) of 8 in a sample of 101 independent limbs, and the CAIT has excellent reliability (ICC2,1 = 0.96) of 11 in an 18-participant sample (36 limbs).11 In addition, the reliability of the Foot and Ankle Outcome Score subscales ranged from ICC2,1 = 0.85 to 0.9615 in a 38-participant sample, the Foot and Ankle Disability Index subscales ranged from ICC2,1 = 0.82 to 0.9913 in a 50-participant sample, and FAAM subcales ranged from ICC2,1 = 0.87 to 0.8912 in a 79-participant sample.

Clinically, we believe it is also important to determine whether the IdFAI will classify participants as either having FAI or not having FAI in a similar manner on different days. To evaluate this, we used the previously reported cut-off score of 11.17 On the first day of testing, 57 participants were identified as having FAI, and on the second day, 59 participants were identified. Nine individuals changed status (FAI or no FAI) between the 2 test days, and their scores varied between 1 and 6 points. Ideally, any questionnaire would identify the same number of individuals in the same sample, but there are a number of reasons for the slight disparity in the number of identifications made by the IdFAI on day 1 versus day 2. Between the 2 test days, participants may have experienced new or more frequent symptoms. Participants may also have participated in higher-risk activities in the time between administrations, which might have created a more vulnerable environment for the participant to experience a more recent or severe episode of instability. It is also plausible that previous exposure to the questionnaire may have heightened some individuals’ awareness to potential symptoms described in the questionnaire; thus, they were more accurately reporting these symptoms. However, the difference in classifications was minimal (n = 2) between the 2 test days and can easily be explained due to the nature of FAI symptoms.

Validity

Performing validity testing on any self-report instrument used to identify individuals with FAI is difficult without a benchmark measure. Similar to the current study, the developers of the CAIT compared their instrument with the LEFS because it was designed to measure function across a wide range of lower-limb disabilities.11 Hiller et al11 found a moderate correlation between the CAIT and the LEFS (ρ = 0.50). Other studies have used an array of comparisons to evaluate the validity of these self-report questionnaires. When testing for the Foot and Ankle Outcome Score, some authors compared it with the Karlsson score and identified moderate correlations (ρ = 0.58 to 0.67).15 Researchers investigating the FAAM compared it with the Global Rating of Function.12 They also found a range of relationships from a low of 0.23 to a high of 0.79 (Kendall τ rank correlation).23 The main reason for the range in correlations was sample tested. When including participants with ankle instability only, the validity values were lower. Finally, when evaluating the Chronic Ankle Instability Scale, researchers compared the Impairments subscale to talar tilt values and compared the Disabilities sub-scale to performance on a functional hop test and the perceived difficulty during the test. Correlations with the Impairments subscale were low (−0.05 to −0.07), but correlations for the Disability subscales were moderate, ranging from −0.38 to −0.49.9 Due to the variety of measures used in these previous studies, it is difficult to compare and contrast these results; however, the current study establishes a moderate correlation between the IdFAI and the LEFS (ρ = −0.38), which seems to be in the range of other published questionnaires.11,12 The correlation was negative because a low score on the LEFS indicates a high degree of lower extremity dysfunction, whereas a high score on the IdFAI is indicative of a high degree of ankle instability; this difference in scoring resulted in an inverse relationship.

One potential reason for the moderate correlation was due to the nature of the LEFS. The LEFS is not an ankle-specific instrument and focuses mainly on disability related to the lower limb during ADL. We found that a number of individuals who were identified as having FAI received the maximum score on the LEFS, indicating normal function. Specifically, 20 (44.5%) of 45 individuals who were identified as having FAI by the IdFAI scored the maximum of 80 on the LEFS. This indicates that the complaints or deficiencies reported by individuals with FAI do not create a change in the LEFS score. Other tools, such as the SF-3624 or a Global Rating Scale of Function,12 warrant further comparison in future research.

Future Research

Several areas of this study can be advanced by future research. Previous research16 has shown that the use of the CAIT and the AII in combination can identify individuals with FAI. Therefore, the next logical step is to compare the IdFAI to the combined CAIT/AII. The results of this comparison would suggest whether the use of the new instrument (IdFAI) would be comparable to or better than the combined usage of existing instruments.

Researchers should also investigate whether the IdFAI has the ability to detect change over time. For example, participants can take the IdFAI before and after a rehabilitation protocol to determine whether changes have occurred. One limitation of the current investigation is the restricted age of the participant sample; future research should expand the inclusive ages to ensure that the IdFAI achieves similar results across all age groups.

Implications for Clinical Practice

The IdFAI is specifically designed to detect individuals with FAI in both clinical and research settings. It is a concise and valid instrument, with strong day-to-day reliability. We believe the IdFAI should continue to be used in further research. Specifically, until a benchmark measure or universally accepted definition for FAI is reported in the literature, the IdFAI can be used to help ensure that individuals meet a minimum set of criteria for FAI.

References

  1. Arnold BL, de la Motte S, Linens S, Ross SE. Ankle instability is associated with balance impairments: a meta-analysis. Med Sci Sports Exerc. 2009;41(5):1048–1062. doi:10.1249/MSS.0b013e318192d044 [CrossRef]
  2. Brown CN, Mynark R. Balance deficits in recreational athletes with chronic ankle instability. J Athl Train. 2007;42(3):367–373.
  3. Arnold B, Docherty C. Low-load eversion force sense, self-reported ankle instability, and frequency of giving way. J Athl Train. 2006;41(3):233–238.
  4. Hertel J. Functional anatomy, pathomechanics, and pathophysiology of lateral ankle instability. J Athl Train. 2002;37(4):363–375.
  5. Hopkins J, Brown T, Christensen L, Palmieri-Smith R. Deficits in peroneal latency and electromechanical delay in patients with functional ankle instability. J Orthop Res. 2009;27(12):1541–1546. doi:10.1002/jor.20934 [CrossRef]
  6. Hertel J. Functional instability following lateral ankle sprain. Sports Med. 2000;29(5):361–371. doi:10.2165/00007256-200029050-00005 [CrossRef]
  7. Hertel J. Sensorimotor deficits with ankle sprains and chronic ankle instability. Clin Sports Med. 2008;27(3):353–370. doi:10.1016/j.csm.2008.03.006 [CrossRef]
  8. Docherty C, Gansneder B, Arnold BL, Hurwitz S. Development and reliability of the Ankle Instability Instrument. J Athl Train. 2006;41(4):154–158.
  9. Eechaute C, Vaes P, Duquet W. The Chronic Ankle Instability Scale: clinimetric properties of a multidimensional, patient-assessed instrument. Phys Ther Sport. 2008;9(2):57–66. doi:10.1016/j.ptsp.2008.02.001 [CrossRef]
  10. Eechaute C, Vaes P, Van Aerschot L, Asman S, Duquet W. The clinimetric qualities of patient-assessed instruments for measuring chronic ankle instability: a systematic review. BMC Muscloskelet Disord.2007;8:6. doi:10.1186/1471-2474-8-6 [CrossRef]
  11. Hiller C, Refshauge K, Bundy A, Herbert R, Killbreath S. The Cumberland Ankle Instability Tool: a report of validity and reliability testing. Arch Phys Med Rehabil. 2006;87(9):1234–1241. doi:10.1016/j.apmr.2006.05.022 [CrossRef]
  12. Martin RL, Irrgang JJ, Burdett R, Conti S, Van Swearingen JM. Evidence of validity for the Foot and Ankle Ability Measure (FAAM). Foot Ankle Int. 2005;26(11):968–983.
  13. Hale S, Hertel J. Reliability and sensitivity of the Foot and Ankle Disability Index in subjects with chronic ankle instability. J Athl Train. 2005;40(1):35–40.
  14. Hubbard T, Kaminski T. Kinesthesia is not affected by functional ankle instability status. J Athl Train. 2002;37(4):481–486.
  15. Roos E, Brandsson M, Karlsson J. Validation of the Foot and Ankle Outcome Score. Foot Ankle Int. 2001;22(10):788–794.
  16. Donahue M, Simon J, Docherty C. Critical review of self-reported functional ankle instability measures. Foot Ankle Int. 2011;32(12):1140–1146. doi:10.3113/FAI.2011.1140 [CrossRef]
  17. Simon J, Donahue M, Docherty C. Development of the Identification of Functional Ankle Instability (IdFAI). Foot Ankle Int. 2012;33(9):755–763. doi:10.3113/FAI.2012.0755 [CrossRef]
  18. Freeman M. Instability of the foot after injuries to the lateral ligament of the ankle. J Bone Joint Surg Br. 1965;47b(4):669–677.
  19. Freeman M, Dean M, Hanham I. The etiology and prevention of functional instability of the foot. J Bone Joint Surg Br. 1965;47(4):678–685.
  20. Delahunt E, Coughlan GF, Caulfied B, Nightingale E, Chung-Wei CL, Hiller C. Inclusion criteria when investigating insufficiencies in chronic ankle instability. Med Sci Sport Exerc. 2010;42(11):2106–2121. doi:10.1249/MSS.0b013e3181de7a8a [CrossRef]
  21. Binkley JM SP, Lott SA, Riddle DL. The Lower Extremity Functional Scale (LEFS): scale development, measurement properties, and clinical application. North American Orthopaedic Rehabilitation Research Network. Phys Ther. 1999;79(4):371–383.
  22. Scientific Advisory Committee of Medical Outcomes Trust. Assessing health status and quality of life-instruments: attributes and review criteria. Qual Life Res. 2002;11(3):193–205.
  23. Carcia C, Martin RL, Drouin J. Validity of the Foot and Ankle Ability Measure in athletes with chronic ankle instability. J Athl Train. 2008;43(2):179–183. doi:10.4085/1062-6050-43.2.179 [CrossRef]
  24. Jenkinson C, Coulter A, Wright L. Short form 36 (SF36) health survey questionnaire: normative data for adults of working age. BMJ. 1993;306(6890):1437–1440. doi:10.1136/bmj.306.6890.1437 [CrossRef]

Reliability Statistics of the Identification of Functional Ankle Instability (IdFAI)

ITEMaICC2,1SEMCRONBACH’S ALPHA
Factor 1: History of ankle instability0.812.210.89
  Q10: Unstable during sport or recreation0.760.470.86
  Q5: Last time of giving way0.691.090.82
  Q6: How often giving way occurs0.700.460.82
  Q7: Roll over—can you stop it?0.820.610.90
Factor 2: Initial ankle sprain0.941.060.97
  Q1: No. of ankle sprains0.950.420.98
  Q2: Time since last sprain0.820.510.90
  Q4: Crutches—how long?0.820.110.90
  Q3: Health care provider; grade?0.840.280.92
Factor 3: Instability during ADL0.831.010.91
  Q8: After ankle sprain, return to normal0.770.890.87
  Q9: During ADL, unstable0.710.300.83
Overall IdFAI questionnaire0.922.760.96

Correlation Between the IdFAI and the LEFS

IdFAILEFS
Factor 1−0.29
Factor 2−0.24
Factor 3−0.41
Score−0.38

10.3928/19425864-20121212-02

Sign up to receive

Journal E-contents