Research in Gerontological Nursing

Focus on Methods 

Investigation of Longitudinal Data Analysis: Hierarchical Linear Model and Latent Growth Model Using a Longitudinal Nursing Home Dataset

Juh Hyun Shin, PhD; In-Soo Shin, PhD


The appropriate use of the data analysis method in a longitudinal design remains controversial in gerontological nursing research. The objective of the current study is to compare statistical approaches between a hierarchical-linear model (HLM) and a latent-growth model (LGM) in random effects, variance explained, growth trajectory, and model fitness. Secondary analysis of longitudinal data was used. Two variables were chosen to demonstrate the comparison between statistical methods. The HLM was superior in addressing unbalanced data in repeated-measures analysis of variance (ANOVA) and multivariate ANOVA because its nested data structure and random effects could be estimated. The LGM had advantages in modeling growth trajectories and model-fit comparisons. Superior to the HLM, the LGM reported more acceptable data fit, reporting a quadratic model, and successfully differentiated between and within components. The current research provides some evidence for applying appropriate statistical methods when addressing longitudinal datasets in gerontological nursing research. [Research in Gerontological Nursing, 12(6), 275–283.]


The appropriate use of the data analysis method in a longitudinal design remains controversial in gerontological nursing research. The objective of the current study is to compare statistical approaches between a hierarchical-linear model (HLM) and a latent-growth model (LGM) in random effects, variance explained, growth trajectory, and model fitness. Secondary analysis of longitudinal data was used. Two variables were chosen to demonstrate the comparison between statistical methods. The HLM was superior in addressing unbalanced data in repeated-measures analysis of variance (ANOVA) and multivariate ANOVA because its nested data structure and random effects could be estimated. The LGM had advantages in modeling growth trajectories and model-fit comparisons. Superior to the HLM, the LGM reported more acceptable data fit, reporting a quadratic model, and successfully differentiated between and within components. The current research provides some evidence for applying appropriate statistical methods when addressing longitudinal datasets in gerontological nursing research. [Research in Gerontological Nursing, 12(6), 275–283.]

A longitudinal design examines tendencies of change in measurements in the same organization over time (e.g., several data collections from a participant, or clustered data divided by geographical area) (Hayat & Hedlin, 2012). In gerontological nursing research, longitudinal analysis is essential because researchers should consider the physical and psychological changes of older adults that continuously impact outcomes (Edjolo, Proust-Lima, Delva, Dartigues, & Pérès, 2016). Traditionally, researchers frequently used repeated measures of analysis of variance (RM ANOVA) and multivariate analysis of variance (MANOVA). RM ANOVA discerns effects of more than one predictor variable on an outcome variable (Krueger & Tian, 2004). A MANOVA addresses many outcome variables, testing vectors, and reporting differences unreported by ANOVAs (French, Macedo, Poulsen, Waterson, & Yu, 2008).

Many facets of longitudinal research should be considered cautiously because issues may include attrition, closing an organization, refusal to participate, and latent effects in the longitudinal design (Finch, 2016). MANOVA has disadvantages in treating missing data because drop-off measurements cannot be included in the MANOVA data analysis (Hayat & Hedlin, 2012). Different time points and groups in an experimental design are predictor variables (i.e., within-subjects factors) in an RM ANOVA model; researchers measure outcomes at the group level (i.e., between-group comparison). When variables are independent, RM ANOVAs allow examination of predictors, time variables, and concurrent effects of multiple variables (Sidani & Lynn, 1993).

Furthermore, the assumption of independence of observations may be violated and the probability of a type 1 error may increase when using traditional regression models due to an incorrect division of variances (Woltman, Feldstain, MacKay, & Rocchi, 2012). Same-time intervals between measures may not be possible because participants' symptom onset may differ, and survey instrument responses may accrue at different times (Krueger & Tian, 2004). A MANOVA assumes linear relationships among pairs of outcome variables. Researchers should consider power when addressing cubic and quadratic effects (French et al., 2008). A MANOVA is appropriate when the assumption of sphericity is cautious, although a RM ANOVA is more appropriate when collecting a measurement only twice and a sphericity assumption using Mauchly's test can be accepted (Armstrong, 2017; Loerts, 2008; Stanley & Hollands, 2014). In longitudinal studies, time can be considered a random effect (Armstrong, 2017); thus, researchers may have difficulty deciding the correct modeling for covariance (McNeish, Stapleton, & Silverman, 2016) or interpretations and estimations of different levels when addressing discrete dependent variables (McNeish et al., 2016).

Review of a Hierarchical Linear Model and Latent Growth Model in Longitudinal Designs

A hierarchical-linear model (HLM) provides a multilevel or mixed-effects design—a composite structure of ordinary least squares regression to expectations of shared variance of dependent variables when independent variables differ by level (Raudenbush & Bryk, 2002; Woltman et al., 2012). An HLM is an essential statistical technique to address nested data, such as patients in nursing homes or hospitals (Min, Park, & Scott, 2016). A HLM permits at least two levels of data structure (Lininger, Spybrook, & Cheatham, 2015), simplifying regression techniques and used in expectations, data decreases, and inferences of cause and effect relationships in experimental designs (Gelman, 2006). Researchers can investigate individual change rates and whether individual/group-level features align with change rates (O'Connell & McCoach, 2004). Researchers using HLM consider the intraclass (intra-nesting/clusters) correlation coefficient (Seo & Li, 2009), the proportion of between-group variance divided by total variance (Raudenbush & Bryk, 2002). Previous researchers reported that HLM yields more comprehensive data analysis information than other methods. For example, researchers reported improved or deteriorated differences by increasing one unit of an independent variable, whereas in traditional longitudinal data-analysis methods (e.g., RM ANOVA), researchers report only statistically significant differences on overall mean scores between groups without determining within-subjects changes with the independent variable (Shin, 2009). Furthermore, HLM precisely explains the extent of effects and direction of variables' relationships, restricting the undervaluing of macrolevel factors and the overvaluing of microlevel factors (i.e., organizational level) (Min et al., 2016).

By using a HLM, it is possible to evaluate independently the outcomes of individual independent variables and group-level averages (Gelman, 2006) and to integrate multilevel theory into the practical model. Furthermore, the characteristics of a growth-trajectory model (e.g., linear, quadratic, cubic, growth model, cubic model, two-piece wise), may be examined using a HLM, which is better than a MANOVA in determining slope differences by covariate, proportion of variances, and explanation of variances by independent variables. Researchers should explain interaction effects between independent variables and confirm between and within variances at all levels (Woltman et al., 2012).

The latent-growth model (LGM) is an advanced statistical method derived from a structured equational model, developed to offset the disadvantages of longitudinal data (Chou, Bentler, & Pentz, 1998). A LGM curve in which time (i.e., individual-level growth) is involved in the statistical model allows evaluation of the individual differences of parameters (i.e., growth/development), like the HLM (Diallo & Lu, 2017). Latent equations facilitate attaining individual levels of growing variation with possible factors that describe the variation (Curran, Harford, & Muthén, 1996).

Each consecutive measurement over time is an essential underlying growth factor, reflecting actual differences over time (Curran et al., 1996). Results from the HLM and structural-equation modeling (SEM) were comparable in parameters and error terms (Chou et al., 1998; Heck & Thomas, 2000; Wendorf, 2002). A HLM requires that (a) all related independent variables are included; (b) all related random effects are included when addressing outcome continuous variables; (c) R (i.e., covariance of within-cluster residuals) is appropriately specified; (d) within-nested residuals have multivariate normal distribution; (e) G (i.e., covariance of random effects) is identified for dependent-variable scales; (f) within-nested residuals and random effects do not vary with other variables (Cov[u, ɛ] = 0); (g) random effects have multivariate normal distribution for all dependent variables; (h) independent variables do not vary with other variables and residuals at all levels (Cov[X, ɛ] = 0, Cov[X, u] = 0); (i) appropriate sample sizes are met for asymptotic inferences at all levels; and (j) missing data are assumed to be missing completely at random or missing at random (McNeish et al., 2016).

Although assumptions are not compatible, the LGM is compatible with the HLM (Chou et al., 1998). The null model in the HLM involves individual-level and random effects, excluding covariates, comparable to the two-step approach of SEM (Wendorf, 2002). The conditional model with covariates in the HLM is comparable to the SEM (Anderson & Gerbing, 1988; Wendorf, 2002). The LGM enables a better fit by assessing factor loadings and error terms (Li, Duncan, Harmer, Acock, & Stoolmiller, 1998; Wendorf, 2002). Time is a latent variable, equal to the random effects in the HLM (Diallo & Lu, 2017). Remarkably, the LGM can incorporate time predictors in that the intercept and slope have random effects in longitudinal datasets (Curran, Obeidat, & Losardo, 2010). Thus, researchers can appraise real predictors of outcome variables considering time-random differences over measurements (Curran et al., 2010).

Previously, few researchers compared these methods with an analysis of a real dataset. The current article addresses customary RM ANOVAs and MANOVAs for analysis of a longitudinal dataset of nursing home residents and provides a conceptual understanding of statistical methods, focusing on a comparison between the HLM and LGM.


The dataset is from a research project supported by the Basic Science Research Program through the National Research Foundation of Korea, funded by the Ministry of Science, ICT & Future Planning project on the “Relationship Between Nurse Staffing and Quality of Care in Nursing Homes” in Korea. The parent project investigated the relationship between nurse staffing and quality of care in nursing homes. Datasets accrued from 46 nursing homes that operated from 2014 to 2017.

Researchers collected information on nurse staffing and quality of care outcomes every 3 to 4 months. In total, 46 organizations participated and researchers accrued a total of seven measurements (45 the first time, 34 the second time, 28 the third time, 19 the fourth time, 15 the fifth time, seven the sixth time, and one the seventh time), and researchers analyzed 137 data points, due to attrition. A total of 93 measurements were not realized because of closing nursing homes and changing businesses. The current research was approved by the Institutional Review Board of a university in South Korea.

Researchers chose two variables to compare statistical methods. The independent variable was the proportion of the long-term care insurance grade from 1 to 5 and the outcome variable was the proportion of residents with depression. The long-term care insurance grade was classified by ranking physical and cognitive functional limitations in activities of daily living determined by the long-term care committee of the National Korean Insurance Corporation in Korea (Korean Long-Term Care Insurance, 2019). The committee determines a total of five grade rankings: first-grade beneficiaries are older adults who are completely physically and cognitively dependent; second-grade beneficiaries are mostly dependent; third-grade beneficiaries are partially dependent; fourth-grade beneficiaries have limited dependence; and the fifth-grade was developed for older adults with dementia.

Researchers summarized all organizational variables by descriptive statistics, then estimated a total of three models. The HLM permits developing a model of Level 2 variances, which is impossible when using repeated measures. The HLM assumes (a) independence of error terms, normal distribution, and homogeneity of variance; (b) independence of independent variables with error terms at Level 1; and (c) independence of independent variables with error terms at Level 2. The authors added the coefficient of variation (within nursing home standard deviation divided by within nursing home mean) as Level 2 covariates to control individual nursing home differences in variability of interpretation. The authors modified the coefficients defined within nursing homes such as depression outcome variables and independent variables (i.e., staffing variables) at a specified time between nursing homes at Level 2, working as fixed and random effects (Nezlek, 2001).


Table 1 summarizes descriptive statistics of participating organizations. The average number of beds was 87.76. Of 46 organizations, 40 were nonprofit, and of residents, 39.1% were third grade long-term care beneficiaries and 23.9% were second-grade long-term care beneficiaries. Finally, the study results of traditional approaches (RM ANOVA, MANOVA) and the HLM, LGM were compared.

Organizational Characteristics of 46 Nursing Homes

Table 1:

Organizational Characteristics of 46 Nursing Homes

Traditional Approach: Repeated Measures Analysis of Variance and Multivariate Analysis of Variance

With an RM ANOVA, the required assumption of sphericity was not met (significant chi-square = 43.84, p < 0.001) and compound symmetry was biased in that the variance and covariance in the population differed. Researchers used adjusted degrees-of-freedom (df) univariate tests (Greenhouse & Geiser, 1959; Huynh & Feldt, 1976) because this analysis supplies adjusted-group differences when the assumption of sphericity is not met (O'Connell & McCoach, 2004). Epsilon was smaller than 0.75 (F = 43.8, p = 0.40) in all three adjustments: Greenhouse and Geiser = 0.398, Huynh and Feldt = 0.445, and lower bound = 0.250.

Table 2 displays RM ANOVA results, showing that the effect of time was not statistically significant, based on the study by Greenhouse and Geiser (1959). However, the effect of time was statistically significant in the MANOVA results based on Wilks' lambda. The RM ANOVA and MANOVA results showed several disadvantages of strict assumptions and missing data management. Gerontological nursing research frequently cannot meet assumptions and the same measurements without missing information. Therefore, the HLM and LGM approaches can be used to investigate longitudinal data.

Repeated-Measures Analysis of Variance (RM ANOVA) and Multivariate ANOVA (MANOVA) Based on Five Time Points

Table 2:

Repeated-Measures Analysis of Variance (RM ANOVA) and Multivariate ANOVA (MANOVA) Based on Five Time Points

Comparison of the Hierarchical-Linear Model and Latent-Growth Model Approaches

Researchers using a HLM can choose potential covariances because the model does not consider the assumption of compound symmetry (Armstrong, 2017). The authors used seven observations (measurements) in the HLM analysis, whereas observations in the RM ANOVA included only the first through fifth observations (measurements), respectively (Table A, available in the online version of this article). Composite symmetry (i.e., all variances and covariances are the same) in HLM shows the plot of individual changes of each measurement's trajectory and individual changes over time (Morris, Payton, & Santorico, 2011). A HLM can assess how much variance to attribute to each level in an unconditional model, the proportion of variance explained in a conditional model, and model comparison information between conditional models, permuting to measure within-subject variation and between-subject variation, unavailable in a RM ANOVA (Bryk & Raudenbush, 1992; Herrin et al., 2015; Lininger et al., 2015).

Hierarchical-linear model.

Table A.

Hierarchical-linear model.

First, using the HLM (i.e., random effects and variance decomposition) approach, the authors structured the data hierarchically (i.e., individual change of nursing homes, county, city, and province). The nested measurement (137) was Level 1 (time) and the total participating nursing homes (46) were Level 2 (patients/residents) units (Table A). Because a HLM contains fixed and random effects, the authors recommended the maximum likelihood to estimate parameters as in a study by Ma, Mazumdar, and Memtsoudis (2012). HLM 7.0 software maximized the possible estimates of fixed and random parameters (Raudenbush, Bryk, Cheong, Congdon, & du Toit, 2011). The HLM was successful for within- and between-group regressions to investigate change over time for residents with depression and the proportion of long-term care beneficiary grade (Table 3). The unit increase of Level 1 was identified by the nested Level 2 cluster, separately identifying the slope of Level 2. Consistent with previous research (Herrin et al., 2015; Shin, 2009; Woltman et al., 2012), the variance of the lowest level of each organization could be fixed at 1, supporting the interpretation, because the authors weighed the probable variance of the united outcome measures in the HLM.

Longitudinal Analysis Results Using a Hierarchical Linear Model

Table 3:

Longitudinal Analysis Results Using a Hierarchical Linear Model

The HLM (137 points) permitted inclusion of 67 more data points than a RM ANOVA (70 points), confirming results from Table 3 and the superiority of the HLM over RM ANOVA when addressing unbalanced data with missing information. RM ANOVA assumed each time point was measured at the same time point. MANOVA can also handle an imbalanced sample size across measurements using statistical computer packages (e.g., SPSS) (French et al., 2008).

The HLM assessed unconditional and conditional models, reporting statistical differences. The unconditional model in the HLM excludes covariates and estimates only measurement models in Level 1 and random effects in Level 2; the conditional model permits the estimation of covariates or predictors (Bryk & Raudenbush, 1992; Wendorf, 2002). Thus, the HLM evaluates the quantity of variability at each level (Min et al., 2016), permitting researchers to compare rational inferences and exact predictions of means, with conventional no-pooling and complete-pooling estimates (Gelman, 2006). The grand mean of the proportion of residents with depression in the null model (unconditional model) was 6.96 (ψ0j = 6.96, SD = 1.96), and 5.08 (ψ0j = 5.08, SD = 1.26) in Conditional Model 2. The authors used the null model to estimate the proportion of between-group variance in the total variance without considering covariates. Variances in conditional Level 2 diminished substantially from 162.84 (the null model) to 51.90 using the proportion of residents from the first to fifth grade as a covariate. The deviance (poor fit) also decreased from 971.83 (the null model) to 940.97 (Conditional Model 2). Differences between the null model and Conditional Model 2 were 971.83 to 940.97 = 30.86, df = 1 (p < 0.05); thus, the conditional model was statistically significant, explaining the variance in the dependent variable better than the null model.

Second, in using the LGM approach (i.e., growth trajectories and model-fit comparison), measurement estimates of the conditional model in the HLM were analog to those of the SEM. The proportion of residents with depression was reported over time. The proportion of residents with a long-term care insurance grade was added as a covariate in the conditional model in the HLM. However, the current study focused on investigating trajectories and model comparisons without covariates. The authors compared three models (i.e., linear, quadratic, cubic) to find which model fit nursing home data well. Unlike the HLM, the LGM permits evaluation of various model-fit indices including chi-square, non-normed fit index, comparative-fit index, and root-mean-square error of approximation (RMSEA) (Wendorf, 2002). The linear and quadratic models are a good fit, but not the cubic model (Table 4). For the current study, the authors chose the quadratic model because the trajectory is more similar to the quadratic model than the linear model. More information was reported: nursing home residents with depression increased at first, then gradually decreased.

Model Comparison: Linear Versus Quadratic Model in Depression

Table 4:

Model Comparison: Linear Versus Quadratic Model in Depression

The model-fit index reported a more acceptable data fit (RMSEA = 0.001, comparative fit index = 1.0), than the linear LGM, which was impossible to attain in the HLM. Consistent with previous research (Chou et al., 1998; Diallo & Lu, 2017; Khoo & Muthén, 2000; Stoel, Van den Wittenboer, & Hox, 2003), the authors also reported equivalent estimated parameters for the HLM (Table 3). The difference was the covariance structure of outcomes. The HLM compiles residual variance of outcomes as one component, whereas the multilevel LGM differentiates between and within components (Diallo & Lu, 2017), which is a major advantage over the HLM. Researchers can examine changes in residual variances for diverse population outcomes using the multilevel LGM (Diallo & Lu, 2017) by disaggregating between- and within-cluster components.


The appropriate use of the data analysis method in a longitudinal design remains controversial. In gerontological nursing research, appropriate comparisons and evaluations of various statistical methods are important to build concise examinations of relating factors on residents' outcomes. Using a longitudinal nursing home dataset, the authors report that the HLM and LGM produced comparable results to traditional RM ANOVA and MANOVA in (a) addressing unbalanced data, (b) explaining variance, and (c) developing models about the relationship between slope and intercept.

In longitudinal research, it is hard to have balanced data without missing data because each health care organization has varying bed sizes, residents, or patients in a rapidly changing health care environment, posing a challenge throughout the decades. RM ANOVAs examine differences in outcomes of dependent variables at different time points, differences with different independent variables, and outcomes with differences in time and independent variables. RM ANOVAs require stricter data collection timing, follow-up measurements, and satisfaction of the sphericity assumption. Sphericity is the statistical measure of the variances in differences between all probable couples of groups that are likely to be the same in the mean levels of predictors (Kim, 2015). Each measurement should be homogeneous for the F test (Shin, 2009). When addressing unstable datasets with missing data in a longitudinal design, the HLM is more appropriate than RM ANOVAs and MANOVAs, if the sample size is sufficient (Hayat & Hedlin, 2012), and can answer more questions than RM ANOVA.

The HLM and LGM are appealing because the specification of the model is possible when assessing time effects and growth rates (Chou et al., 1998). In a RM ANOVA, data were aggregated, ignoring nursing home differences and each individual difference, whereas data were disaggregated when using the HLM, which reported the proportion of the total outcome variation (i.e., intraclass correlation coefficient) among health care organizations (e.g., nursing homes, by dividing Level 3 residual variance for the intercept). Thus, researchers can report the explained variance.

The authors were able to examine whether the primary data value was statistically different, indicating different growth among participants. Previous researchers also supported advantages reported in the current study (Doorenbos, Given, Given, & Verbitsky, 2006; Lee & Seo, 2013; Miyazaki & Stack, 2015; Seo, Macy, Torabi, & Middlestadt, 2011). The LGM handles nonlinear characteristics in a longitudinal design and integrates correlates; thus, researchers have the best answers to research questions (MacCallum, Kim, Malarkey, & Kiecolt-Glaser, 1997). The current study reported the major superiority of LGM compared with HLM in that LGM levels differentiate between and within components. Consequently, residual variances of outcome measurements over time are possible using the multilevel LGM consistent with the HLM (Diallo & Lu, 2017). The multivariate-growth model shows that researchers can (a) construct linear and quadratic forms, (b) directly assess the association between intercepts and slopes in and across variables, and (c) incorporate time effects into the model. Further research should study multivariate longitudinal data analysis.

As a final note, current and pioneering statistical methods should be considered; the multivariate-growth model compiles the HLM and SEM with the LGM (Curran et al., 2010), permitting the addition of nested upper levels of datasets by allowing a concurrent separation in the between- and within-group models (Diallo & Lu, 2017; Heck & Thomas, 2008).


The current study has limitations in that this analysis and interpretation builds on a longitudinal survey study and causal relationships could not be applied. However, the correct estimation of variance at each level made interpretation more valid. Although this analysis pooled seven longitudinal descriptive datasets, the sample (N = 46) was relatively small. Thus, study results cannot be applied to the whole population of Korea. Further research with a large database should be conducted.


The scientific and appropriate use of longitudinal data analysis is necessary in gerontological nursing. The current study revealed the efficiency of HLM and LGM compared with traditional RM ANOVA and MANOVA in that the specification of the model is available when assessing time effects and growth rates and addressing missing data. Finally, the multilevel LGM successfully differentiated between and within components, permitting different model-fit indices, reporting quadratic model outcomes, stating that nursing home residents with depression increased at first, then gradually decreased, which was not possible in the HLM.

Further research with a large sample should study multivariate longitudinal data analysis. The current study's main objective was to compare methods for conducting longitudinal data analysis. In the application of the HLM and LGM, the growth trajectory and adequacy of data should be explained in greater detail in future studies.


  • Anderson, J.C. & Gerbing, D.W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103, 411–423. doi:10.1037/0033-2909.103.3.411 [CrossRef]
  • Armstrong, R.A. (2017). Recommendations for analysis of repeated-measures designs: Testing and correcting for sphericity and use of MANOVA and mixed model analysis. Ophthalmic & Physiological Optics, 37, 585–593. doi:10.1111/opo.12399 [CrossRef]28726257
  • Bryk, A.S. & Raudenbush, S.W. (1992). Hierarchical linear models: Applications and data analysis methods. London, England: Sage.
  • Chou, C., Bentler, P.M. & Pentz, M.A. (1998). Comparisons of two statistical approaches to study growth curves: The multilevel model and the latent curve analysis. Structural Equation Modeling, 5, 247–266. doi:10.1080/10705519809540104 [CrossRef]
  • Curran, P.J., Harford, T.C. & Muthén, B.O. (1996). The relation between heavy alcohol use and bar patronage: A latent growth model. Journal of Studies on Alcohol, 57, 410–418. doi:10.15288/jsa.1996.57.410 [CrossRef]8776683
  • Curran, P.J., Obeidat, K. & Losardo, D. (2010). Twelve frequently asked questions about growth curve modeling. Journal of Cognition & Development, 11, 121–136. doi:10.1080/15248371003699969 [CrossRef]21743795
  • Diallo, T.M.O. & Lu, H. (2017). Consequences of misspecifying across-cluster time-specific residuals in multilevel latent growth curve models. Structural Equation Modeling, 24, 359–382. doi:10.1080/10705511.2016.1247647 [CrossRef]
  • Doorenbos, A.Z., Given, C.W., Given, B. & Verbitsky, N. (2006). Symptom experience in the last year of life among individuals with cancer. Journal of Pain & Symptom Management, 32, 403–412. doi:10.1016/j.jpainsymman.2006.05.023 [CrossRef]17085266
  • Edjolo, A., Proust-Lima, C., Delva, F., Dartigues, J.F. & Pérès, K. (2016). Natural history of dependency in the elderly: A 24-year population-based study using a longitudinal item response theory model. American Journal of Epidemiology, 183, 277–285. doi:10.1093/aje/kwv223 [CrossRef]26825927
  • Finch, W.H. (2016). Missing data and multiple imputation in the context of multivariate analysis of variance. Journal of Experimental Education, 84, 356–372. doi:10.1080/00220973.2015.1011594 [CrossRef]
  • French, A., Macedo, M., Poulsen, J., Waterson, T. & Yu, A. (2008). Multivariate analysis of variance (MANOVA). Retrieved from
  • Gelman, A. (2006). Multilevel (hierarchical) modeling: What it can and cannot do. Technometrics, 48, 432–435. doi:10.1198/004017005000000661 [CrossRef]
  • Greenhouse, S.W. & Geiser, S. (1959). On methods in the analysis of profile data. Psychometrika, 24, 95–112. doi:10.1007/BF02289823 [CrossRef]
  • Hayat, M.J. & Hedlin, H. (2012). Modern statistical modeling approaches for analyzing repeated-measures data. Nursing Research, 61, 188–194. doi:10.1097/NNR.0b013e31824f5f58 [CrossRef]22551993
  • Heck, R.H. & Thomas, S.L. (2000). An introduction to multilevel modeling techniques. Mahwah, NJ: Lawrence Erlbaum.
  • Heck, R.H. & Thomas, S.L. (2008). An introduction to multilevel modeling techniques (2nd ed.). New York, NY: Routledge.
  • Herrin, J., St. Andre, J., Kenward, K., Joshi, M.S., Audet, A.M. & Hines, S.C. (2015). Community factors and hospital readmission rates. Health Services Research, 50, 20–39. doi:10.1111/1475-6773.12177 [CrossRef]
  • Huynh, H. & Feldt, L.S. (1976). Estimation of the box correction for degrees of freedom from sample data in randomized block and split-plot designs. Journal of Educational Statistics, 1, 69–82. doi:10.3102/10769986001001069 [CrossRef]
  • Khoo, S.T. & Muthén, B. (2000). Longitudinal data on families: Growth modeling alternatives. In Rose, J.S., Chassin, L., Presson, C.C. & Sherman, S.J. (Eds.), Multivariate applications in substance use research: New methods for new questions (pp. 43–78). Mahwah, NJ: Lawrence Erlbaum.
  • Kim, H. (2015). Statistical notes for clinical researchers: A oneway repeated measures ANOVA for data with repeated observations. Restorative Dentistry & Endodontics, 40, 91–95. doi:10.5395/rde.2015.40.1.91 [CrossRef]
  • Korean Long-Term Care Insurance. (2019). Long term care insurance. Retrieved from
  • Krueger, C. & Tian, L. (2004). A comparison of the general linear mixed model and repeated measures ANOVA using a dataset with multiple missing data points. Biological Research for Nursing, 6, 151–157. doi:10.1177/1099800404267682 [CrossRef]15388912
  • Lee, C. & Seo, D. (2013). Trajectory of suicidal ideation in relation to perceived overweight from adolescence to young adulthood in a representative United States sample. Journal of Adolescent Health, 53, 712–716. doi:10.1016/j.jadohealth.2013.06.013 [CrossRef]23910569
  • Li, F., Duncan, T.E., Harmer, P., Acock, A. & Stoolmiller, M. (1998). Analyzing measurement models of latent variables through multilevel confirmatory factor analysis and hierarchical linear modeling approaches. Structural Equation Modeling, 5, 294–306. doi:10.1080/10705519809540106 [CrossRef]
  • Lininger, M., Spybrook, J. & Cheatham, C.C. (2015). Hierarchical linear model: Thinking outside the traditional repeated-measures analysis of variance box. Journal of Athletic Training, 50, 438–441. doi:10.4085/1062-6050-49.5.09 [CrossRef]
  • Loerts, H. (2008). Multivariate ANOVA and repeated measures. Retrieved from
  • Ma, Y., Mazumdar, M. & Memtsoudis, S.G. (2012). Beyond repeated-measures analysis of variance: Advanced statistical methods for the analysis of longitudinal data in anesthesia research. Regional Anesthesia & Pain Medicine, 37, 99–105. doi:10.1097/AAP.0b013e31823ebc74 [CrossRef]
  • MacCallum, R.C., Kim, C., Malarkey, W.B. & Kiecolt-Glaser, J.K. (1997). Studying multivariate change using multilevel models and latent curve models. Multivariate Behavioral Research, 32, 215–253. doi:10.1207/s15327906mbr3203_1 [CrossRef]26761610
  • McNeish, D., Stapleton, L.M. & Silverman, R.D. (2016). On the unnecessary ubiquity of hierarchical linear modeling. Psychological Methods, 22, 114–140. doi:10.1037/met0000078 [CrossRef]27149401
  • Min, A., Park, C.G. & Scott, L.D. (2016). Evaluating technical efficiency of nursing care using data envelopment analysis and multilevel modeling. Western Journal of Nursing Research, 38, 7–20. doi:10.1177/0193945916650199 [CrossRef]
  • Miyazaki, Y. & Stack, M. (2015). Examining individual and school characteristics associated with child obesity using a multilevel growth model. Social Science & Medicine, 128, 57–66. doi:10.1016/j.socscimed.2014.12.032 [CrossRef]25589033
  • Morris, T.L., Payton, M.E. & Santorico, S.A. (2011). A permutation test for compound symmetry with application to gene expression data. Journal of Modern Statistical Methods, 10, 448–461.
  • Nezlek, J.B. (2001). Multilevel random coefficient analyses of event-and interval-contingent data. Personality & Social Psychology Bulletin, 27, 771–785. doi:10.1177/0146167201277001 [CrossRef]
  • O'Connell, A.A. & McCoach, D.B. (2004). Applications of hierarchical linear models for evaluations of health interventions: Demystifying the methods and interpretations of multilevel models. Evaluation & the Health Professions, 27, 119–151. doi:10.1177/0163278704264049 [CrossRef]15140291
  • Raudenbush, S.W. & Bryk, A.S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage.
  • Raudenbush, S.W., Bryk, A.S., Cheong, Y.F., Congdon, R. & du Toit, M. (2011). HLM 7 for Windows [computer software]. Skokie, IL: Scientific Software International.
  • Seo, D.C. & Li, K. (2009). Effects of college climate on students' binge drinking: Hierarchical generalized linear model. Annals of Behavioral Medicine, 38, 262–268. doi:10.1007/s12160-009-9150-3 [CrossRef]
  • Seo, D.C., Macy, J., Torabi, M.R. & Middlestadt, S.E. (2011). The effect of a smoke-free campus policy on college students' smoking behaviors and attitudes. Preventive Medicine, 53, 347–352. doi:10.1016/j.ypmed.2011.07.015 [CrossRef]21851836
  • Shin, J. (2009). Application of repeated-measures analysis of variance and hierarchical linear model in nursing research. Nursing Research, 58, 211–217. doi:10.1097/NNR.0b013e318199b5ae [CrossRef]19448525
  • Sidani, S. & Lynn, M.R. (1993). Examining amount and pattern of change: Comparing repeated measures ANOVA and individual regression analysis. Nursing Research, 42, 283–286. doi:10.1097/00006199-199309000-00007 [CrossRef]8415042
  • Stanley, J. & Hollands, M. (2014). A novel video-based paradigm to study the mechanisms underlying age- and falls risk-related differences in gaze behavior during walking. Ophthalmic & Physiological Optics, 34, 459–469. doi:10.1111/opo.12137 [CrossRef]
  • Stoel, R.D., Van den Wittenboer, G. & Hox, J. (2003). Analyzing longitudinal data using multilevel regression and latent growth curve analysis. Metodología de las Ciencias del Comportamiento, 5, 21–42.
  • Wendorf, C.A. (2002). Comparisons of structural equation modeling and hierarchical linear modeling approaches to couples' data. Structural Equation Modeling, 9, 126–140. doi:10.1207/S15328007SEM0901_7 [CrossRef]
  • Woltman, H., Feldstain, A., MacKay, J.C. & Rocchi, M. (2012). An introduction to hierarchical linear modeling. Tutorials in Quantitative Methods for Psychology, 8, 52–69. doi:10.20982/tqmp.08.1.p052 [CrossRef]

Organizational Characteristics of 46 Nursing Homes

Bed size (mean, SD)87.76 (13.39)
Ownership (n, %)
  For profit6 (13)
  Not for profit40 (87)
Occupancy rate (%) (mean, SD)89.7 (13.54)
Operation duration (years) (mean, SD)9.0067 (1.03)
Location of organizationsa (n, %)
  Large metropolitan city (≥1,000,000)13 (28.9)
  Local small city (50,000 to 500,000)26 (57.8)
  Rural area (<50,000)6 (13.3)
Long-term care insurance (n, %)
  1stb6 (13)
  2ndc11 (23.9)
  3rdd18 (39.1)
  4the7 (15.2)
  Unrated4 (8.7)
Hospital chain (n, %)
  Yes42 (91.3)
  No4 (8.7)
Religious establishment (n, %)
  Yes18 (39.1)
  No28 (60.9)
Hours per resident day (mean, SD)
  RN11 min, 5 sec
  Certified nursing assistant10 min, 31 sec
  Qualified care worker2 h, 42 min, 28 sec
Turnoverf (mean, SD)
  RN5.94 (20.93)
  Certified nursing assistant9.73 (25.71)
  Qualified care worker12.18 (56.67)

Repeated-Measures Analysis of Variance (RM ANOVA) and Multivariate ANOVA (MANOVA) Based on Five Time Points

RM ANOVA (Tests of Within-Subjects Effects)
SourceType III Sum of SquaresdfMean SquareFp ValuePartial η2
  Sphericity assumed267.714466.9292.9410.0290.184
  Greenhouse & Geiser267.7141.592168.1642.9410.0850.184
  Huynh & Feldt267.7141.778150.5382.9410.0780.184
Error (time)
  Sphericity assumed1183.4865222.759
  Greenhouse & Geiser1183.48620.69657.185
  Huynh & Feldt1183.48623.11951.191
  Lower bound1183.4861391.037
  Pillai's trace0.6133.9564100.0350.613
  Wilks' lambda0.3873.9564100.0350.613
  Hotelling's trace1.5823.9564100.0350.613
  Roy's largest root1.5823.9564100.0350.613

Longitudinal Analysis Results Using a Hierarchical Linear Model

ItemMean (SD)
Null Model (Unconditional Model)Conditional Model 1Conditional Model 2
Fixed effect
  Initial status (ψ0j)6.96**(1.96)6.95** (1.98)5.08** (1.26)
  Growth rate (Υ00)0.13 (0.22)−0.01 (0.25)
Random effect
  Initial status (intercept)162.84**165.94**51.90**
  Growth rate (proportion of grade beneficiaries)2.23**
  Level 129.4129.3823.76
Statistics for current model
  Deviance (badness of fit)971.83970.83940.97
  Number of estimated parameters224
Model comparison
  χ2 statistic29.85**

Model Comparison: Linear Versus Quadratic Model in Depression

Modelχ2dfCFIRMSEALo 90Hi 90
Linear & quadratics5.47261000.184

Hierarchical-linear model.

Notation of the two-level growth model: The Level-1 coefficients are denoted by π and the Level-2 coefficients by β. The Level-1 and Level-2 predictors are ati and Xpq, and the random effects are now eti and rpi, respectively.
Repeated measure Level 1 Yti=π0i+π1iαti+π2iαti2++πpiαtiP+etipFor i = 1,…, n institution, where αti is the measure at time t for nursing home i and πpi is the growth trajectory parameter p for nursing home i associated with the polynomial of degree P (i.e., p = 0, …, P). Each nursing home is observed on Ti occasions. Note that the number and spacing of measurement occasions may vary across nursing homes. Repeated measure Level 2 πpi=β0i+q=1QpβpqXqi+rpiwhere Xqi is either a measured characteristic of the nursing home's background or of an experimental treatment. βpq represents the effect of Xq on the pth growth paramenter; and rpi is a random effect with a mean of 0. The set of P + 1 random effects for nursing home i are assumed to be multivariate normally distributed with full covariance matrix, T, dimensioned (P + 1) X (P + 1).

Dr. J.H. Shin is Associate Professor, Ewha Womans University, and Dr. I.-S. Shin is Professor, Department of Transdisciplinary Security, Dongguk University, Seoul, South Korea.

The authors have disclosed no potential conflicts of interest, financial or otherwise. The current research was supported by the National Research Foundation of Korea grant funded by the Korean government (Ministry of Science, ICT & Future Planning) (2017R1A2B4003282).

Address correspondence to Juh Hyun Shin, PhD, Associate Professor, Ewha Womans University, 205 Helen Hall, 52, Ewhayeodae-gil, Seodaemungu, Seoul, South Korea, 120-750; e-mail:

Received: February 15, 2019
Accepted: June 19, 2019


Sign up to receive

Journal E-contents