An Examination of the Improved Wald Test for Differential Item Functioning Detection with Multiple Groups

Abstract

Methods for identifying items that function differentially (DIF) across groups have over time become increasingly essential for establishing validity in psychometric testing. There is a vital need for simultaneously comparing multiple groups, evidenced by the widespread proliferation of international assessments in educational testing. Historically, DIF methods have been formulated to only address pairwise group comparisons; methods that address the two or more groups’ case in a latent variable context are a comparatively recent phenomenon. This study evaluates the effectiveness of Wald-1, a newly developed DIF detection approach for multiple group comparisons. Data were simulated under the three-parameter logistic (3PL) Item Response Theory (IRT) model, with the explicit design of simulation conditions and parameters guided by empirical data. Results were examined in the context of statistical power and Type I error rate under various combinations of experimental conditions: (a) the number of test groups, (b) the number of candidate items with DIF, and (c) the number of anchor items with DIF. The results indicate that Wald-1 performs well in the identification of DIF items, collapsing across all other variables such as the number of groups, conditional on the existence of a DIF-free anchor set. The effectiveness of Wald-1, practical implications of the results, and considerations of future research are examined and discussed thoroughly. Reliable and effective anchor selection methods are prerequisite of excellent performance for Wald-1 detecting DIF.