The Hierarchical Testlet Response Time Model: Bayesian analysis of a testlet model for item responses and response times

Abstract

Computer-based testing makes it possible to record an examinee’s response time on an item. This information can be an important factor to understand the examinees, as well as the items (Marianti, Fox, Avetisyan, Veldkamp, & Tijmstra, 2014; van der Linden, 2007). Most response time scoring models are based on unidimensional Item Response Theory (IRT) models. If tests are composed of testlet items, then the assumption of local independence for IRT models is likely to be violated. The purpose of this study is to introduce the Hierarchical Testlet Response Time (HTRT) model to address local dependence among items, and to evaluate the impact on parameter estimation when fitting a response time model to item response and response time data that have been influenced by testlet effects. The study compares the HTRT model with the Hierarchical Framework model (van der Linden, 2007), and explores the relationship between item characteristics and examinee ability as well as response time, which is examined using real and simulated data. The Bayesian estimation using the Markov Chain Monte Carlo (MCMC) method was applied to the investigation of response time. The HTRT model generated better parameter recovery than the Hierarchical Framework model. The HTRT model recovered all parameters very well, with a small magnitude of errors. The current results demonstrate that the Hierarchical Framework model had very good recovery of both the item difficulty and time intensity parameters, but fairly poor recovery of the item discrimination and time discrimination parameters. The examinee ability and speed parameters showed poor recovery, due not to bias but to dramatically increase random error.