Loading...
The Impact of the Item Types and Number of Solution Steps of Multiple-Choice Items on Item Difficulty and Discrimination and Test Reliability
Atalmis, Erkan Hasan
Atalmis, Erkan Hasan
Citations
Altmetric:
Abstract
This study examined two multiple choice item-writing guidelines addressed by Haladyna, Downing, and Rodriguez (2002). One is related to using the "None of the Above (NOTA)" option, the other is about the plausible number of options for a multiple-choice item (MCI). These two guidelines were empirically tested using one-step and multi-step problems to identify their impact on item characteristics (item difficulty and item discrimination) and test characteristics (test reliability). Three forms with MCIs were generated and administered to approximately 1500 7th and 8th grade students in the United States and Turkey. Bi-factor Item Response Theory (IRT) was applied to assess dimensionality related to the number of solution steps of items. Multiple regression models were employed to determine the degree of impact these item-writing guidelines had on item and test characteristics for MCIs with one step solution (MCI with one-step solution) and those with more than one step solutions (MCIs with multi-step solution). The results show that item characteristics do not change significantly across the conventional MCIs with four options, MCIs with three options, and MCIs with NOTA option. The interaction between solution steps and the three MCI types had no significant impact on item characteristics. For the test with MCIs with a one-step solution, the findings demonstrate that four options are significantly more reliable than the NOTA options and not statistically different from three options. For the test with MCIs with multi-step solutions, four options are not statistically different from three and NOTA options. Compared to MCIs with four options, the results support that MCIs with NOTA options are preferable for MCIs with multi-step solutions while three options are desirable for both MCIs with one-step solutions and multi-step solutions.
Description
Date
2014-08-31
Journal Title
Journal ISSN
Volume Title
Publisher
University of Kansas
Research Projects
Organizational Units
Journal Issue
Keywords
Educational psychology, Educational tests & measurements, Item writing guidelines, Multiple choice questions, None of the above, Non-parametric irt model, Number of options, Option elimination