Bi-factor Multidimensional Item Response Theory Modeling for Subscores Estimation, Reliability, and Classification

Abstract

In recent years, there has been increasing interest in estimating and improving subscore reliability. In this study, the multidimensional item response theory (MIRT) and the bi-factor model were combined to estimate subscores, to obtain subscores reliability, and subscores classification. Both the compensatory and partially compensatory MIRT models are defined with bi-factor structure. A Monte Carlo study with 1,500 examinees was carried out for each model to examine two different test lengths (30 and 60 items) and five levels of item discrimination between primary and specific abilities (.50, .75, 1.0, 1.25, 1.50). The Markov Chain Monte Carlo (MCMC) with the Gibbs sampling method was applied to simultaneously estimate the expected a posteriori (EAP) subscores for primary and specific ability dimensions. Results were evaluated in light of estimation accuracy and fit, subscore reliability based on the Bayesian marginal reliability, and subscore classification based on subscore separation index. Despite a very minimum computing intensity for the MCMC simulation, both bi-factor compensatory and bi-factor partially compensatory models produced higher subscores reliability resulted from lower bias and reduction in the error variance of EAP subscores in all ability dimensions. These improved subscores reliability that also arrived at a higher discrimination level and for a longer test. This study found the bi-factor compensatory model to show better potential in classifying the magnitude of distinction between specific abilities and primary ability. Whereas, the bi-factor partially compensatory minimized the classification of subscores between the specific and primary abilities.