Journal of Nursing Education

Major Article 

Assessing the Construct Congruence of the RN Comprehensive Predictor and NCLEX-RN Test Plan

Xin Liu, PhD; Christine Mills, PhD

Abstract

Background:

The 2013 NCLEX-RN® test plans specified an underlying general entry-level nursing ability encompassing all the tested specific categories and subcategories, as indicated by a second-order factor structure. This study attempted to verify this hierarchical factor structure in real data.

Method:

Data from the RN Comprehensive Predictor® 2013 assessment, developed by Assessment Technologies Institute using the 2013 NCLEX-RN test plans, were used to test the entry-level nursing ability second-order factor structure by a confirmatory factor analysis technique.

Results:

The results indicated a good fit of the proposed factor model to the RN Comprehensive Predictor 2013 assessment data.

Conclusion:

The verified second-order factor structure provides evidence for the existence of the general entry-level nursing ability, which has important theoretical implications to the development and testing of a nursing instrument or theory in nursing science research. In addition, the verified congruent factor structure provides strong evidence for the construct validity of the RN Comprehensive Predictor 2013 assessment. [J Nurs Educ. 2017;56(7):412–419.]

Abstract

Background:

The 2013 NCLEX-RN® test plans specified an underlying general entry-level nursing ability encompassing all the tested specific categories and subcategories, as indicated by a second-order factor structure. This study attempted to verify this hierarchical factor structure in real data.

Method:

Data from the RN Comprehensive Predictor® 2013 assessment, developed by Assessment Technologies Institute using the 2013 NCLEX-RN test plans, were used to test the entry-level nursing ability second-order factor structure by a confirmatory factor analysis technique.

Results:

The results indicated a good fit of the proposed factor model to the RN Comprehensive Predictor 2013 assessment data.

Conclusion:

The verified second-order factor structure provides evidence for the existence of the general entry-level nursing ability, which has important theoretical implications to the development and testing of a nursing instrument or theory in nursing science research. In addition, the verified congruent factor structure provides strong evidence for the construct validity of the RN Comprehensive Predictor 2013 assessment. [J Nurs Educ. 2017;56(7):412–419.]

Nursing careers have long been in high demand throughout the country. According to the Bureau of Labor Statistics (2015), the total number of job openings for RNs is expected to grow 16% from 2014 to 2024, much faster than the average for all occupations. In addition to the state member board requirements, nursing graduates need to pass the NCLEX-RN® to practice as an RN in the United States, Canada, and U.S. territories. The NCLEX-RN is developed and owned by the National Council of State Boards of Nursing, Inc. (NCSBN). The NCLEX-RN is a licensure examination that is used by the jurisdictional boards of nursing in the United States and U.S. territories to assess the minimal competence needed to perform safely and effectively as a newly licensed, entry-level RN.

The aim of this study is to confirm the second-order hierarchical factor structure using a set of real data from an examination that was built using the NCLEX-RN test plan. First, the authors will describe the foundation supporting a general entry-level nursing ability as measured by the 2013 NCLEX-RN test plan creation process. Second, the authors will describe how this test plan was used to construct the RN Comprehensive Predictor® 2013 assessment. It is the data from the RN Comprehensive Predictor 2013 assessment that are used to test the hypothesized second-order factor structure presumed to exist for the general entry-level nursing ability.

The purpose of the RN Comprehensive Predictor examination is to provide nursing students with a probability of passing the NCLEX-RN licensure examination. The RN Comprehensive predictor is built using the NCLEX test plan to ensure, to the extent possible, that the content specifications measure the construct of a general entry-level nursing ability that is closely aligned with that of the NCLEX-RN. Because the NCSBN implements a series of rigorous steps to define the NCLEX test plan, as detailed below, the test plan is the best reference to build the RN comprehensive predictor.

The NCLEX-RN test plan undergoes a series of rigorous development processes to select the content and behaviors that are essential to the practice of nursing in meeting the client needs in maintaining, promoting, and restoring health. The NCSBN first conducts a practice analysis that gathers the data on the nursing care activities from the newly licensed RNs. Twelve thousand newly licensed RNs were surveyed about the frequency and importance of performing 141 nursing care activities. Then data related to these current nursing practices were analyzed regarding the effects of meeting client needs and the fundamentals of nursing. Based on the analysis results of these practice surveys, the test plan document was developed from the entry-level nursing practice deemed essential by survey respondents. Finally, the test plan was reviewed by the experts of the Examination Committee, the NCSBN content staff, and the NCSBN's Member Boards and was finally approved by the NCSBN's Delegate Assembly. The 2013 NCLEX-RN test plans (NCSBN, 2012b) are directly based on the results of a nationwide practice analysis conducted by the NCSBN (2012a) in 2011. Of note is that the NCSBN (2016) has just released the 2016 NCLEX-RN test plans this year. Due to insufficient test data available for the new test plan, the focus of the current study was on the 2013 NCLEX-RN test plans. With the 2016 update of the NCLEX RN test plan, minimal modifications were made to specific nursing tasks in the outline, and only two tasks were added. Consequently, the authors of the current study hypothesize that the general factor structure for a general entry-level nurse ability remains largely consistent across the two test outlines.

The 2013 NCLEX-RN test plans are defined in the framework of client needs, including four major client needs categories: (a) safe and effective care environments (SEC), (b) health promotion and maintenance (HPM), (c) psychosocial integrity (PSI), and (d) physiological integrity (PHI). The first category, SEC, is further divided into two subcategories: management of care (MC) and safety and infection control (SI). Similarly, the fourth category, PHI, is further divided into four subcategories: basic care and comfort (BC), pharmacological and parenteral therapies (PP), reduction of risk potential (RR), and physiological adaptation (PA). The second and third categories, HPM and PSI, are not composed of any subcategories. The percentage of how many test questions should be assigned to each client needs category or subcategory is determined by the findings of the practice analysis and expert judgments (Table 1).

Test Specifications for the 2013 NCLEX-RN and the RN Comprehensive Predictor 2013 Assessment

Table 1:

Test Specifications for the 2013 NCLEX-RN and the RN Comprehensive Predictor 2013 Assessment

The structure of the 2013 NCLEX-RN test plans define the content areas required for entry-level nursing competency. This content structure presumes a hierarchical factorial structure. First, the test plan requires an examinee to answer a sufficient number of questions covering each category, and a single total score integrated over all the categories is used to determine the pass or fail decision. The information that a single total score provides is valid only when it is legitimate to conceive of the object being measured as a single entity that can be scored on a single unidimensional scale (Hambleton, Swaminathan, Cook, Eignor, & Gifford, 1978; Hattie, 1985). This indicates that these categories are positively correlated to reflect the influence of an underlying general factor that entered into performance on all kinds of entry-level nursing tasks. The general factor reflected in the test plan indicates entry-level nursing competency that is required of an RN to practice in the United States, Canada, and U.S. territories. The general-factor structure of the NCLEX-RN is in alignment with the fact that in nursing science, theoretical concepts are often multidimensional; however, there exists a coherent association among the different facets. For example, the scale of self-care management of heart failure can be defined using correlated concepts of monitoring signs, implementing treatment, and evaluating treatment (Barbaranelli, Lee, Vellone, & Riegel, 2014); adherence to health regimens may be defined using concepts such as medical care, responsibility, care planning, and implementing care (Kyngas, 1999). Therefore, a general factor that the authors call entry-level nursing ability in the current article is assumed to hold across the four main categories.

With the general entry-level nursing ability at the top, more specific categories are conceptualized at successive lower layers. The division of the NCLEX-RN categories into subcategories indicate a sequential ordering of the nursing concepts in the measurement of entry-level nursing competency. In particular, a two-layer hierarchical pattern is conceptualized with the general entry-level nursing ability at the top and the more specific categories at successive lower layer of the sequence. The two-layer hierarchical pattern is known as a second-order factor solution (Chen, West, & Sousa, 2006; Gorsuch, 1983; Gustafsson & Balke, 1993; Mulaik & Quartetti, 1997).

The top-layer factor is called the second-order factor (i.e., the general factor), and the more specific categories are called first-order factors. In the model, correlations among first-order factors are explained by the second-order factor. Likewise, each first-order factor explains the correlation among the nursing tasks loading on each factor, respectively. The general factor (i.e., the second-order factor) at the apex in the NCLEX-RN factor models explains the correlations of all four categories and is measured by the total scores over all four categories as an indicator for a minimum competency required for licensing an entry-level RN for nursing practice.

The hierarchical model has long been explored in depth in the factor-analytic research on the measurement of the mental or human traits. Spearman (1904) observed that a general factor does emerge from analysis of mental ability tests, and his findings have been confirmed by many leading psychologists in the years since (Carroll, 1993; Jensen, 1998; Johnson, Bouchard, Krueger, McGue, & Gottesman, 2004; Kamphaus, Winsor, Rowe, & Kim, 2005). This evidence puts a general human cognitive ability factor at the apex in the model, with more specific aptitudes arrayed at successively lower levels. The so-called group factors, such as verbal ability, mathematical reasoning, spatial visualization, and memory, are just below the general factor, and below these are skills that are more dependent on knowledge or experience, such as the principles and practices of a particular job or profession.

In contrast to the abundance of the hierarchical factorial research on the mental cognitive assessment is the scarcity of that on the assessment of nursing care concepts and theories. As far as the authors of the current study know, few analyses have been conducted on exploring or confirming the existence of a general factor as an outcome of factor analysis of any nursing concept or theory. As for the NCLEX-RN scale, a previous factorial analysis study was conducted by O'Neill and Reynolds (2006) that reveals the existence of a single dimension underlying the NCLEX-RN construct through a principal component analysis of residuals estimated from Winsteps. This has been the only factorial study of the NCLEX-RN published so far that probed into the factor structure of this high-stakes examination. However, the hierarchical factor structure, as clearly prescribed in the national practice analysis and summarized in the NCLEX-RN test plan, has not yet been verified in any real test data in any form that the authors of the current study could find.

As stated earlier, the aim of this study is to confirm the hypothesized second-order hierarchical factor structure using a set of real data from an examination that was built from the NCLEX-RN test plan. The data used in this study are from the fourth version of Assessment Technologies Institute (ATI) RN Comprehensive Predictor assessment. The rationale behind the application of this data set is described below.

The test specifications of RN Comprehensive Predictor assessment are directly based on those of the NCLEX-RN test plan and measure constructs that are essential to entry-level registered nursing practice (Table 1). The items on the RN Comprehensive Predictor 2013 assessment were written by internal and external RN educators. The RN educators held master's of science in nursing or PhD degrees and had teaching or practical experience in the specific content areas (e.g. adult medical–surgical or mental health). To ensure that all test items met the requirements as outlined by the NCLEX-RN test plan, a series of test development procedures were followed.

Prior to developing new items, an item writing workshop was held for external RN educators to provide training about interpreting and using NCLEX-RN test blueprints. Internal RN educators must also complete additional training, which involves multiple iterations of feedback or revisions for newly written items. As each item was written, item writers were asked to follow the guidelines of alignment to the NCLEX-RN test plan, including writing items to specific NCLEX-RN tasks, presenting the same item format, aligning item cognitive levels with NCLEX-RN classifications (i.e., Bloom's Taxonomy), and assembling a test with the similar number of items for each category and subcategory. Finally, all newly written items underwent a subsequent rigorous review and validation process. The RN educators were asked to verify the item's alignment with the NCLEX-RN task assignment, outcome categories, priority setting, and nursing process. Item content was validated using approved references and supported by the indication of applicable NCLEX-RN content. This entire test development process contributes important evidence that pertains to the alignment of the RN Comprehensive Predictor with the NCLEX-RN test plan.

The resultant findings will produce substantial theoretical contributions to the development and measurement of nursing concepts and theories. The verification of the second-order factor model with a general factor at the apex and more specific categories below will be the first exploratory study of the hierarchical factorial structure of examination data based on the general entry-level nursing construct. The resultant findings will provide evidence as to what degree the long-formulated general factor theories may hold true for the NCLEX-RN construct. In addition, the implications on the verification of the hierarchical structure will definitely enhance the theoretical framework for the development of nursing theories in general.

The results will provide evidence for the construct validity of RN Comprehensive Predictor 2013 assessment by verifying the congruence between the actual structure of the examination and the specified factor structure of the test plan. Any test that is built to model the specified construct of the test plan should mirror the same factor structures. The invariant factor structure is an important piece of evidence for construct validity of a scale because its scores represent what it is supposed to mirror (Byrne, Shavelson, & Muthen, 1989). Meanwhile, the knowledge on how the NCLEX-RN client care needs categories and subcategories relate to each other and how they relate to the general entry-level nursing ability will have practical implications on the appropriate use of the RN Comprehensive Predictor 2013 assessment. Knowledge of those relationships helps achieve meaningful interpretations of the total score and sub-scale scores, which in turn helps faculty members better align their curricula to help students form tailored study plans.

Method

Two broad categories of factor analysis exist: exploratory (EFA) and confirmatory (CFA) (Gorsuch, 1983; Stevens, 1996; Watson & Thompson, 2006). A major difference between these two techniques is whether an a priori model exists where the indicator–factor correspondence has been explicitly specified by the measurement theory. When such an a priori model does not exist, the EFA will be appropriate. In EFA, the user assumes each variable could be related to each latent factor. As such, the data are explored to obtain information about the number of factors to represent the data and which variable is related to which latent factor. In CFA, the user assumes a predefined priori model. As such, the CFA conducts a statistical hypothesis test on the proposed theory that models the number of factors and the relationship between variables and factors, based on a priori theory or the results of EFA.

In this study, the a priori model that is defined by the NCLEX-RN test plan and used to build the RN Comprehensive Predictor assessment represents a hierarchical relationship between the categories and subcategories and the general entry-level nursing factor. The goal is to evaluate the degree to which this factor structure can be confirmed in the RN Comprehensive Predictor assessment data. Therefore, the CFA is applied as the quantitative analysis technique to confirm or reject the measurement theory. As pointed out in the earlier sections, the model as defined in the 2013 NCLEX-RN test plan follows a second-order factor model. The following section will elaborate on the model in more detail (Table 2) and visually present it in Figure 1.

NCLEX-RN 2013 Factor Model Specifications

Table 2:

NCLEX-RN 2013 Factor Model Specifications

NCLEX-RN 2013 factor models. Error variance represented by circles labeled with r1 to r10 was specified to account for the fact that the factors will not perfectly predict the manifest variables. Nursing ability = entry-level nursing ability; SEC = safe and effective care environments; MC = management of care; SI = safety and infection control; HPM = health promotion and maintenance; PSI = psychosocial integrity; PHI = physiological integrity; BC = basic care and comfort; PP = pharmacological and parenteral therapies; RR = reduction of risk potential; PA = physiological adaptation.

Figure 1.

NCLEX-RN 2013 factor models. Error variance represented by circles labeled with r1 to r10 was specified to account for the fact that the factors will not perfectly predict the manifest variables. Nursing ability = entry-level nursing ability; SEC = safe and effective care environments; MC = management of care; SI = safety and infection control; HPM = health promotion and maintenance; PSI = psychosocial integrity; PHI = physiological integrity; BC = basic care and comfort; PP = pharmacological and parenteral therapies; RR = reduction of risk potential; PA = physiological adaptation.

In CFA, observed variables are called manifest variables, which are represented by rectangles, and latent variables are called factors, which are represented by ovals. Arrows show the direction of causal relationship. In the second-order factor model, eight manifest variables exist, which are the RN Comprehensive Predictor assessment scores on eight of the categories and subcategories: MC, SI, HPM, PSI, BC, PP, RR, and PA. Two first-order latent factors exist: SEC and PHI. The SEC factor was regressed on the two manifest variables of MC and SI (i.e., the SEC explains the relationships among MC and SI). The PHI factor was regressed on the four manifest variables of BC, PP, RR, and PA. The second-order latent factor refers to the general entry-level nursing ability, which explains the correlations among the two first-order factors (i.e., SEC and PHI) and the two manifest variables (i.e., HPM and PSI). As clearly hypothesized in the test plan, each manifest variable loads on only one factor, which implies no cross-loadings. Error variance represented by circles refers to the amount of variance not explained by its corresponding factor. Error variance was specified to be random for each manifest variable and each first-order factor, respectively. The random error variance assumes that no correlational relationship exists between the manifest variables (or the first-order factor) after accounting for the influence from the first-order factors (or the second-order factors). This assumption is made due to two reasons: (a) no such relationship is implied in either the test plan or the previous dimensionality analysis, and (b) due to the lack of method covariance, the specifications of correlated errors are unnecessary because the data of the variables are collected from a single assessment modality (i.e., RN Comprehensive Predictor assessment) rather than the different assessment modalities (e.g., self-report, behavioral observations, or interview ratings, known as the multitrait–multimethod approach). For model identification purposes, the second-order factor variance was fixed to one, as were the first-order factor variances.

Confirmatory factor analysis is a powerful multivariate analytic tool. Despite the many advantages of CFA, like any statistical modeling procedure, the accuracy of results is vulnerable to violation of certain statistical assumptions underlying the model of estimation. Among the most important assumptions are multivariate normally distributed data, adequate sample size, linear relationship between observed variables, and a properly specified model (Child, 2006; Comrey & Lee, 1992; Curran, 1994; Field, 2009; Gorsuch, 1983). In terms of the specification of an a priori model, the long-term histories of the NCLEX-RN test plan and its robust development provide evidence that the model is appropriately specified. The other assumptions will be tested before conducting the CFA with the data.

A sample of 39,113 students who took their first attempt at the RN Comprehensive Predictor 2013 assessment was included in this study. The sample included the test data from almost all the candidates who registered and took the test during the 2013 to 2014 school year. These students came from 824 institutions in 58 states and U.S. territories. The majority (i.e., over 90%) of these students were between ages 18 and 45 years. The demographic information about candidates' gender, ethnicity, and primary language were self-reported and collected during the test registration stage (Table 3). Candidates were able to choose not to reveal such personal information. As Table 3 shows, approximately 17% to 23% of the candidates did not report their gender, ethnicity, or primary language.

Sample Characteristics for RN Comprehensive Predictor 2013 Assessment Data

Table 3:

Sample Characteristics for RN Comprehensive Predictor 2013 Assessment Data

Results

Before interpreting the CFA results, the authors reviewed the results for the test of the statistical assumptions underlying the CFA model. Prior to testing whether the observed variables collectively are normally distributed (i.e., multivariate normality), the univariate normality of each observed variable is tested and supported by the most popular graphical method of normal quantile–quantile plot (Stevens, 1996). For multivariate normality test, the procedure of the Mahalanobis distance is used (Field, 2009). The authors have eight observed variables; therefore, the degree of freedom is eight. For alpha = .001, the authors found a critical chi-square = 26.13. The highest Mahalanobis distance is 25.9; therefore, multivariate outliers do not exist. This indicates that the property of multivariate normal distribution is met in the data set. As for the sample size requirement, the sample of 39,113 students was deemed sufficient. To test whether the combinations of variables would produce linear factors, the authors examined the correlation matrix of the observed variables. All the pairs of the correlations ranged from 0.30 to 0.50. These moderate yet distinct correlations indicate the factorability of the correlation matrix, as Tabachnick and Fidell (2007) explained, “If no correlation exceeds .30, use of FA is questionable because there is probably nothing to factor analyze” (p. 614). Finally, no missing data exist for each observed variable in the data.

The SAS CALIS procedure (SAS Institute, 2013) was used to conduct the CFA of the RN Comprehensive Predictor assessment data. The results of the goodness-of-fit model analysis are presented in Table 4. The goodness-of-fit analysis tests the hypothesis that the RN Comprehensive Predictor 2013 assessment fits the proposed second-order factor model. A goodness-of-fit measure of the chi-square statistic is usually used to evaluate model fit. However, the chi-square statistic is sensitive to large sample sizes; therefore, additional fit measures are used to evaluate the fit of the model (Byrne, 2001; Schumacker & Lomax, 2004; Thompson, 2000). Five additional model-fit statistics were used: adjusted goodness-of-fit index (Jöreskog & Sörbom, 1996), root mean squared error of approximation (Steiger, 1990; Steiger & Lind, 1980), the standardized root mean square residual (Hu & Bentler, 1999), the comparative fit index (Bentler, 1990), and the non-normed fit index (i.e., the Tucker–Lewis Index) (Tucker & Lewis, 1973). Good model fit is observed if the adjusted goodness-of-fit index values exceed .95 or higher (Thompson, 2000); the comparative fit index equals .95 or higher (Hu & Bentler, 1999); the standardized root mean square residual values are less than .05, with adequate fit associated with the root mean squared error of approximation values less than .06 (Byrne, 1998; Hu & Bentler, 1999); and when the non-normed fit index values are .95 or higher (Hu & Bentler, 1999). As shown in Table 4, all the fit indices fell in the desired ranges, which clearly indicates an adequate fit of the factor model to the RN Comprehensive Predictor assessment data.

Model Fit Statistics for NCLEX-RN 2013 Factor Models

Table 4:

Model Fit Statistics for NCLEX-RN 2013 Factor Models

The factor loadings are the regression coefficients (i.e., slopes) for predicting the indicators from the latent factor. In general, the higher the factor loading the better, and typically, standardized loadings below .30 are not interpreted. Usually, loadings >.71 = excellent, >.63 = very good, >.55 = good, >.45 = fair, and >.32 = poor (Tabachnick & Fidell, 2007). The variance accounted for by the latent factor is equal to the square of the factor loadings (Child, 2006). Note that a loading of .71 squared is equal to 50% accounted variance, whereas .32 squared is 10% accounted variance.

The standardized factor loadings and error variance are shown in Figure 2. Note that all the factor loadings were greater than 0.5 and significant at the .001 level. The strong loadings indicate that the percentage of variance explained in the observed variables by that latent factor is high. The factor loadings on the first-order factor SEC varied from .69 for MC and to .51 for SI. The factor loadings on the other first-order factor, PHI, varied from .56 for BC, .67 for PP, .69 for RR, and .66 for physiological adaptation. The factor loadings on the second-order factor entry-level nursing ability varied from .53 for HPM, .52 for PSI, .95 for SEC, and .96 for PHI. The standardized values of error variance in Figure 2 report similar results: higher factor loading corresponds to smaller error variance. In summary, the highest factor loading (and the lowest error variance) was for PHI on nursing ability, indicating that 92% of the variance in PHI was explained by the general entry-level nursing ability. The lowest factor loading (and the highest error variance) was for SI on the SEC, in which 26% of the response variability in subcategory SI was accounted for by the SEC factor.

Standardized factor loadings for NCLEX-RN® 2013 factor model. Nursing Ability = entry-level nursing ability; SEC = safe and effective care environment; MC = management of care; SI = safety and infection control; HPM = health promotion and maintenance; PSI = psychosocial integrity; PHI = physiological integrity; BC = basic care and comfort; PP = pharmacological and parenteral therapies; RR = reduction of risk potential; PA = physiological adaptation.

Figure 2.

Standardized factor loadings for NCLEX-RN® 2013 factor model. Nursing Ability = entry-level nursing ability; SEC = safe and effective care environment; MC = management of care; SI = safety and infection control; HPM = health promotion and maintenance; PSI = psychosocial integrity; PHI = physiological integrity; BC = basic care and comfort; PP = pharmacological and parenteral therapies; RR = reduction of risk potential; PA = physiological adaptation.

Discussion

The hierarchical nature of many nursing care concepts is closely related to the goal of the nursing practice profession, which is to serve human needs and meet client needs for preventing, maintaining, promoting, and restoring health (NCSBN, 2016). As noted in Maslow's hierarchy of needs (Maslow, 1943, 1954), human needs are arranged as physiological, safety, love, esteem, and self-actualization, in order of importance. The five-stage model can be classified into lower order (i.e., physiological, safety, and love) and higher order (i.e., esteem and self-actualization) needs. It should be noted that the hierarchical classifications may vary across cultures, peacetime, wartime, and by gender and age groups (McLeod, 2007). Maslow's (1970a, 1970b) hierarchy of needs has been expanded to include more refined needs. Human needs are interrelated and some needs depend on others.

The hierarchical nature of nursing care concepts has been hypothesized by many nursing researchers in some nursing care studies. Several researchers have gone further and explored a hierarchical structure of the nursing concepts through factorial analysis techniques. For example, a three-layered hierarchical model (with an overall quality factor at the top level and the subordinate factors at the lower level) have been verified on the Observable Indicators of Nursing Home Care Quality Instrument (Rantz et al., 2006).

The current study is the first known attempt in the nursing theory research that statistically confirmed the hierarchical factor structure of a general entry-level nursing ability construct as specified in the NCLEX-RN test plan through the use of assessment data that share the same content test specifications. The verification of the second-order factor model in this study has not just echoed the previous unidimensional findings of the NCLEX-RN scale (O'Neill & Reynolds, 2006), but it has significantly enhanced the theoretical framework of the nursing concepts and theories in general. Similar to the structure of other mental performances (e.g., intelligence), the construct as outlined in the test plan can be legitimately conceptualized in terms of a single general ability factor, and a large number of narrow ability factors, as supported by the fit of the second-order factor model in the RN Comprehensive Predictor assessment data. This evidence supports the claim that the seemingly multidimensional categories and subcategories are correlated, yet redundant attributes of the underlying general factor of the entry-level nursing ability.

In addition, the relative magnitude of the strong yet distinct factor loadings on each category indicate a congruence of the item designation of the RN Comprehensive Predictor assessment to that specified in the test plan. Maslow's interrelated nursing needs theory lays the foundation for most types of the nursing care concepts widely accepted in current nursing practice. A person must meet the needs at the foundation of the hierarchy before working toward meeting higher level needs.

Consistent with the need theory (Maslow, 1943, 1954), the NCLEX-RN test plan allocates various corresponding weights to the four major content categories in order of the importance of client needs: entry-level nursing tasks related to meeting physiological needs were rated as the most frequent and important (i.e., 50% of the test items in the PHI category), followed by tasks for meeting client safety needs (i.e., 32% of the items in the SEC category). The least weight on tasks related to need for love, esteem, and self-actualization (i.e., 9% of the items in the PSI and HPM categories).

The results of the CFA conducted in this study echoed the order of importance assigned to each category in the real data: stronger factor loadings fall on the two first-order factors of the PHI and SEC categories; however, the factor effect on the remaining HPM and PSI categories are much lower.

The congruence is observed on the relative magnitude of factor loadings of subcategories. The first-order factor SEC has a stronger effect on the subcategory of MC than on SI, which matches the NCLEX-RN designations of more items (i.e., 20% on average) to the preceding subcategory than the latter one (i.e., 12% on average). Similarly, among the four subcategory variables explained by the first-order factor of PHI, the least factor effect is on BC. This pattern echoed the NCLEX-RN test plan specifications of 9% of the test to this subcategory, compared with at least 12% to the other three subcategories of RR, PA, and PP.

The concordance of the RN Comprehensive Predictor assessment data and the test plan provides strong construct validation evidence for the RN Comprehensive Predictor assessment total score and subscale score as meaningful indicators of the corresponding general entry-level nursing ability construct. Although tasks on this examination are a random set selected from a designated category or subcategory, the resultant model verified in this study shows that the observed category or subcategory scores are adequate indicators for each measured factor of content. As such, area of attention identified by the RN Comprehensive Predictor assessment is based on a subset of representative items for the tested underlying content. Thus, the types of tasks missed on the test provide a reasonable scope for review or remediation. To achieve optimal review or remediation effect, the authors recommend using them as indicators of scope in the effort of thorough review of all possible related tasks rather than limiting to just the tasks missed. The findings of the strong factor loadings of each and every category (i.e., above 0.50 in value) indicate the importance of mastering each content area in the success of becoming an entry-level nurse. No redundant scale is specified in the test plan, but rather all contribute to the successful performance on the general entry-level nursing ability. Reviewing each and every content area with equal attention is important to master the skills, knowledge, and ability for entry-level nursing practice.

The authors are confident in the findings of this study partly due to the sample used to conduct the CFA. The sample was composed of nursing students enrolled in three programs (i.e., ADN, BSN, and diploma) across 50 states and U.S. territories. The sample of 39,113 students is not a convenient sample, but rather a representative sample of the nursing graduate population in key demographic variables of gender, ethnicity, primary language, and location. Having a large and representative sample normally leads to a better estimate of the population parameters because it can guard against some of the harmful consequences of unusual cases and assumption violation. Except for the demographic variables of gender, ethnicity, and primary language, no missing data on any of the observed variables were analyzed in the CFA model. This indicates that no missing data problem is present in this study that biases the results.

Conclusion

This study demonstrates the CFA analysis procedure, a useful technique in validating a nursing scale or testing a nursing theory. Although CFA was rarely seen in studies published in the nursing journals from 1982 to 2004 (Watson & Thompson, 2006), its use in nursing science has increased in the 21st century (Kääriäinen et al., 2011). It is noteworthy that most of those articles adopted a first-order factor model without exploring whether an alternative model, such as a second-order factor model, may be a better fit to the tested nursing concepts or theories. The application of the CFA analysis in the current study enhanced the need to take such factor models into consideration in future nursing science research on the development and testing of a nursing instrument or theory.

References

  • Barbaranelli, C., Lee, C.S., Vellone, E. & Riegel, B. (2014). Dimensionality and reliability of the self-care of Heart Failure Index scales: Further evidence from confirmatory factor analysis. Research in Nursing & Health, 37, 524–537. doi:10.1002/nur.21623 [CrossRef]
  • Bentler, P.M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238–246. doi:10.1037/0033-2909.107.2.238 [CrossRef]
  • Bureau of Labor Statistics. (2015). Registered nurses: Job outlook. Retrieved from http://www.bls.gov/ooh/healthcare/registered-nurses.htm#tab-6
  • Byrne, B.M. (1998). Structural equation modeling with Lisrel, Prelis, and Simplis: Basic concepts, applications and programming. Mahwah, NJ: Lawrence Erlbaum Associates.
  • Byrne, B.M. (2001). Structural equation modeling: Perspectives on the present and the future. International Journal of Testing, 1, 327–334. doi:10.1080/15305058.2001.9669479 [CrossRef]
  • Byrne, B.M., Shavelson, R.J. & Muthen, B.O. (1989). Testing for equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105, 456–466. doi:10.1037/0033-2909.105.3.456 [CrossRef]
  • Carroll, J.B. (1993). Human cognitive abilities: A survey of factor-analytic studies. Cambridge, England: Cambridge University Press. doi:10.1017/CBO9780511571312 [CrossRef]
  • Chen, F.F., West, S.G. & Sousa, K.H. (2006). A comparison of bifactor and second-order models of quality-of-life. Multivariate Behavioral Research, 41, 189–225. doi:10.1207/s15327906mbr4102_5 [CrossRef]
  • Child, D. (2006). The essentials of factor analysis (3rd ed.). New York, NY: Continuum International Publishing Group.
  • Comrey, A.L. & Lee, H.B. (1992). A first course in factor analysis (2nd ed.). Hillside, NJ: Lawrence Erlbaum Associates.
  • Curran, P.J. (1994). The robustness of confirmatory factor analysis to model misspecification and violations of normality (Doctoral dissertation). Retrieved from http://www.unc.edu/~curran/pdfs/Curran%281994%29.pdf
  • Field, A. (2009). Discovering statistics using SPSS: Introducing statistical methods (3rd ed.). Thousand Oaks, CA: Sage.
  • Gorsuch, R.L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates.
  • Gustafsson, J.E. & Balke, G. (1993). General and specific abilities as predictors of school achievement. Multivariate Behavioral Research, 28, 407–434. doi:10.1207/s15327906mbr2804_2 [CrossRef]
  • Hambleton, R.K., Swaminathan, H., Cook, L.L., Eignor, D.R. & Gifford, J.A. (1978). Developments in latent trait theory: Models, technical issues, and applications. Review of Educational Research, 48, 467–510. doi:10.3102/00346543048004467 [CrossRef]
  • Hattie, J. (1985). Methodology review: Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9, 139–164. doi:10.1177/014662168500900204 [CrossRef]
  • Hu, L. & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. doi:10.1080/10705519909540118 [CrossRef]
  • Jensen, A.R. (1998). The g factor: The science of mental ability. Westport, CT: Praeger.
  • Johnson, W., Bouchard, T.J. Jr.. , Krueger, R.F., McGue, M. & Gottesman, I.I. (2004). Just one g: Consistent results from three test batteries. Intelligence, 32, 95–107. doi:10.1016/S0160-2896(03)00062-X [CrossRef]
  • Jöreskog, K.G. & Sörbom, D. (1996). LISREL 8: User's reference guide. Chicago, IL: Scientific Software International.
  • Kääriäinen, M., Kanste, O., Elo, S., Pölkki, T., Miettunen, J. & Kyngäs, H. (2011). Testing and verifying nursing theory by confirmatory factor analysis. Journal of Advanced Nursing, 67, 1163–1172. doi:10.1111/j.1365-2648.2010.05561.x [CrossRef]
  • Kamphaus, R.W., Winsor, A.P., Rowe, E.W. & Kim, S. (2005). A history of intelligence test interpretation. In Flanagan, D.P. & Harrison, P.L. (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (2nd ed., pp. 23–38). New York, NY: Guilford.
  • Kyngas, H. (1999). A theoretical model of compliance in young diabetics. Journal of Clinical Nursing, 8, 73–80. doi:10.1046/j.1365-2702.1999.00213.x [CrossRef]
  • Maslow, A.H. (1943). A theory of human motivation. Psychological Review, 50, 370–396. doi:10.1037/h0054346 [CrossRef]
  • Maslow, A.H. (1954). Motivation and personality. New York, NY: Harper and Row.
  • Maslow, A.H. (1970a). Motivation and personality (2nd ed.). New York, NY: Harper & Row.
  • Maslow, A.H. (1970b). Religions, values, and peak-experiences. New York, NY: Penguin.
  • McLeod, S.A. (2007). Maslow's hierarchy of needs. Retrieved from http://www.simplypsychology.org/maslow.html
  • Mulaik, S.A. & Quartetti, D.A. (1997). First order or higher order general factor?Structural Equation Modeling, 4, 193–211. doi:10.1080/10705519709540071 [CrossRef]
  • National Council of State Boards of Nursing. (2012a). 2011 RN practice analysis: Linking the NCLEX-RN examination to practice. Retrieved from https://www.ncsbn.org/12_RN_Practice_Analysis_Vol53.pdf
  • National Council of State Boards of Nursing. (2012b). 2013 NCLEX-RN test plan. Retrieved from https://www.ncsbn.org/2013_NCLEX_RN_Test_Plan.pdf
  • National Council of State Boards of Nursing. (2016). NCLEX-RN examination: Test plan for the National Council Licensure Examination for Registered Nurses. Retrieved from https://www.ncsbn.org/RN_Test_Plan_2016_Final.pdf
  • O'Neill, T. & Reynolds, M. (2006). Assessing the unidimensionality of the NCLEX-RN. Retrieved from https://www.ncsbn.org/2005.04_ONeill_-_AERA_-_Assessing_the_Unidimensionality_of_the_NCLEX-RN.pdf
  • Rantz, M.J., Zwygart-Stauffacher, M., Mehr, D.R., Petroski, G.F., Owen, S.V., Madsen, R.W. & Maas, M. (2006). Field testing, refinement, and Psychometric evaluation of a new measure of nursing home care quality. Journal of Nursing Measurement, 14, 129–148. doi:10.1891/jnm-v14i2a005 [CrossRef]
  • SAS Institute. (2013). SAS 9.4 language reference concepts. Cary, NC: Author.
  • Schumacker, R.E. & Lomax, R.G. (2004). A beginner's guide to structural equation modeling (2nd ed.). Mahwah, NJ: Lawrence Earlbaum Associates.
  • Spearman, C.E. (1904). ‘General intelligence,’ objectively determined and measured. American Journal of Psychology, 15, 201–293. doi:10.2307/1412107 [CrossRef]
  • Steiger, J.H. (1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25, 173–180. doi:10.1207/s15327906mbr2502_4 [CrossRef]
  • Steiger, J.H. & Lind, J. (1980, May). Statistically based tests for the number of common factors. Paper presented at the annual meeting of the Psychometric Society. , Iowa City, IA. .
  • Stevens, J.P. (1996). Applied multivariate statistics for the social sciences (3rd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.
  • Tabachnick, B.G. & Fidell, L.S. (2007). Using multivariate statistics (5th ed.). Boston, MA: Allyn and Bacon.
  • Thompson, B. (2000). Ten commandments of structural equation modeling. In Grimm, L.G. & Yarnold, P.R. (Eds.), Reading and understanding MORE multivariate statistics (pp. 261–283). Washington, DC: American Psychological Association.
  • Tucker, L.R. & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1–10. doi:10.1007/BF02291170 [CrossRef]
  • Watson, R. & Thompson, D.R. (2006). Use of factor analysis in Journal of Advanced Nursing: Literature review. Journal of Advanced Nursing, 55, 330–341. doi:10.1111/j.1365-2648.2006.03915.x [CrossRef]

Test Specifications for the 2013 NCLEX-RN and the RN Comprehensive Predictor 2013 Assessment

NCLEX Client Need Major Category and Subcategories2013 NCLEX-RN % of Items Assigned% of Items on ATI RN CPScored Items on ATI RN CP
SEC
  MC17 to 232030
  SI9 to 151218
HPM6 to 12914
PSI6 to 12913
PHI
  BC6 to 12913
  PP12 to 181523
  RR9 to 151218
  PA11 to 171421

NCLEX-RN 2013 Factor Model Specifications

Hierarchical LevelFactorVariable
OneSECMC
SI
OnePHIBC
PP
RR
PA
TwoNursing abilitySEC
HPM
PSI
PHI

Sample Characteristics for RN Comprehensive Predictor 2013 Assessment Data

Program/DemographicGroupn%
ProgramADN19,57550
BSN18,87948.3
Diploma6591.7
GenderFemale27,92671.4
Male4,33311.1
Unidentified6,85417.5
EthnicityAfrican American2,8587.3
Asian1,7384.4
Caucasian22,51857.6
Hispanic2,1615.5
Native American1880.5
Other7341.9
Unidentified8,91622.8
Primary languageEnglish31,27079.9
French1070.3
Spanish2600.7
Other8222.1
Unidentified6,65417
Total39,113100

Model Fit Statistics for NCLEX-RN 2013 Factor Models

Fit IndicesFit CriterionNCLEX-RN Model in RN Comprehensive Predictor Assessment
Model χ2 (df)379.451 (18)
p< .0001
AGFI⩾0.950.995
RMSEA<0.060.023
SRMR<0.050.011
CFI⩽0.950.995
NNFI⩾0.950.992
Authors

Dr. Liu is Senior Psychometrician, and Dr. Mills is Director, Research and Applied Psychometrics, Ascend Learning, Leawood, Kansas.

The authors thank Jerry Gorham for his valuable comments and suggestions that greatly contributed to improving the final version of this paper.

Address correspondence to Xin Liu, PhD, Senior Psychometrician, Ascend Learning, 11161 Overbrook Road, Leawood, KA 66211; e-mail: Xin.Liu@AscendLearning.com.

Received: August 18, 2016
Accepted: February 15, 2017

10.3928/01484834-20170619-05

Sign up to receive

Journal E-contents