A Study of Bayesian Estimation and Comparison of Response Time Models in Item Response Theory

Abstract

Response time has been regarded as an important source for investigating the relationship between human performance and response speed. It is important to examine the relationship between response time and item characteristics, especially in the perspective of the relationship between response time and various factors that affect examinee's responses. The purpose of this study was to examine different scoring models using response time data in conjunction with item response models. In this study distinctive response time models incorporated in IRT were compared, and the relationship between item characteristics and examinee ability as well as response time were examined using real and simulated data. Bayesian estimation using Markov Chain Monte Carlo (MCMC) methods for Thissen's (1983) lognormal response time model, Wang and Hanson's (2005) 4PL RT model, and van der Linden's (2007) hierarchical framework were applied to the investigation of response time on real data. Overall, van der Linden's (2007) hierarchical framework showed the most reasonable outcomes from the real data analysis when it was compared with the 4PL RT and Thissen's models. Compared with Wang and Hanson's (2005) 4PL RT model in the simulated data analysis, the hierarchical framework also showed better results as follows: (1) better recoveries in item and examinee parameter, (2) reasonable explanations in delineating relationships between response time and other related parameters in the model. There were no clear relationships among speed-related parameters across the models when the relationships between the response time-related parameters were investigated across the response time models. This was due to the different definitions and different parameterization procedures of the speed-related parameters based on the response time model.