Fit Index Sensitivity in Multilevel Structural Equation Modeling

Abstract

Multilevel Structural Equation Modeling (MSEM) is used to estimate latent variable models in the presence of multilevel data. A key feature of MSEM is its ability to quantify the extent to which a hypothesized model fits the observed data. Several test statistics and so-called fit indices can be calculated in MSEM as is done in single-level structural equation modeling. Accordingly, problems associated with these measures in the single-level case may apply to the multilevel case and new complications may arise. Few studies, however, have examined the performance of fit indices in MSEM. Furthermore, recent findings suggest that evaluating fit at each level separately is advantageous to evaluating fit for the overall model. Therefore, the purpose of the present study was to evaluate the sensitivity of several fit indices to misspecification in the cluster-level model under varying multilevel data conditions including the intraclass correlation coefficient, sample size configuration, and severity of model misspecification. Furthermore, three methods of level-specific fit evaluation were compared. Results from a Monte Carlo simulation study suggest that fit indices are affected by the ICC of model indicators and sample size configurations in MSEM. With the exception of the SRMR, all fit indices were less sensitive to cluster-level model misspecification at low indicator ICCs, large overall sample sizes, and smaller numbers of clusters. Discrepancies in fit information between two methods of level-specific fit were observed at low ICC values. Finally, two fit indices rarely used in SEM applications revealed desirable properties in certain simulation conditions. Implications of the simulation results are discussed and a program for implementing level-specific fit evaluation in the R statistical language is provided.