Detection of Attribute Hierarchies and Classification Accuracy: the Value of the Hierarchical Diagnostic Classification Model in Formative Assessment Practices

Abstract

The assumption that learning occurs sequentially, or in steps, is a common consideration in K12 education. As the landscape of student education continues to advance, driven by efforts to incorporate tools that offer educators and students feedback capable of identifying areas a student is excelling or struggling in, cognitive diagnostic models are emerging as potentially effective and efficient tools. Despite the value of diagnostic models, there are concerns regarding the application of these models when as learning hierarchy is present or theorized; applying nonhierarchical cognitive diagnostic models when an attribute hierarchy is present or applying hierarchical cognitive diagnostic models when an attribute hierarchy is not present influences the classification accuracy of the models. This study was designed to evaluate the efficiency of the HDCM in statistically testing for an attribute hierarchy, therein providing researchers with evidence and support for subsequent model application. By evaluating the results from a formative assessment designed based on the structure of a theorized attribute hierarchy, this study highlighted the model fit variations and student classification differences. The results of this study indicate that the HDCM does in fact provide a means for researchers to investigate the presence of a theorized hierarchy. Additionally, this study highlights the potential classification differences noted between models.