Applications of Exploratory Q-Matrix Discovery Procedures in Diagnostic Classification Models

Abstract

Diagnostic Classification Models (DCM) use a Q-matrix to determine which skills are required to correctly answer items on large-scale assessments. DCMs are fit under the assumption that the Q-matrix is correctly specified. Misspecification of the Q-matrix is problematic for several reasons; problems with model convergence, poor model fit, and inflated model parameters. The current study examines the use of probabilistic estimation of the Q-matrix for cognitive diagnosis modeling in order to allow for uncertainty to help shape the construction of the Q-matrix. Two DCMs, the DINA and the DINO, were estimated for common reading comprehension tests using an EM algorithm and the goodness of fit was checked. Models using a probabilistic Q-matrix showed better fit and lower slip and guess parameters, suggesting that the probabilistic model provided more accurate Q-matrix specification and more accurate prediction of examinee skills.