Examination of the Parameter Estimate Bias When Violating the Orthogonality Assumption of the Bifactor Model

Abstract

Educational and psychological constructs are normally measured by multifaceted dimensions. The measured construct is defined and measured by a set of related subdomains. A bifactor model can accurately describe such data with both the measured construct and the related subdomains. However, a limitation of the bifactor model is the orthogonality assumption that proposes that there are no correlations between the measured construct and its associated subdomains and among the associated subdomains (i.e., orthogonal). This assumption requires that all items perfectly measure the specified constructs so that no correlations exist among all the constructs. In other words, test developers need to write items perfectly, measuring the primary and one subdomain factor only. However, test items are inherently flawed in practice and can rarely be written perfectly. To force correlated factors to be orthogonal can result in a loss of information and can lead to distorted and untrustworthy parameter estimates in bifactor solutions. Precision of parameter estimates is an important issue in any assessment because parameter estimates are considered to be decisive criteria for finalizing item performance and respondents' ability level. The purpose of this study, therefore, was to investigate the parameter estimate bias of different levels of orthogonality violation among factors of the bifactor model. Since the orthogonality violation cannot be controlled and true parameters are unknown in real data, an extensive series of simulation studies were generated to evaluate a proposed bifactor model with various orthogonality violations among the subdomains. Results indicated that levels of orthogonality violation had no significant influence on intercept and theta parameter estimates but did have a significant influence on discrimination parameter estimates. Higher levels of orthogonality violation could result in severely distorted discrimination parameter estimates. Orthogonality violations between two subdomains could only distort parameter estimates of the involved subdomains and the primary construct but not the other subdomains. Among all of the theta parameter estimates, the estimates of the primary dimension were most trustworthy. Item length had no significant influence on either item or person parameter estimates.