A Psychometric Examination of the School Implementation Scale

Abstract

The purpose of this study is to further enhance the existing reliability and validity evidence related to the School Implementation Scale (SIS) and its use for evaluating the implementation of different integrated academic and behavior tiered Response-To-Intervention (RTI) models. Previous methodology conducted for validation of the instrument has been expanded to examine the factor structure of the measurement instrument and the relationship between those factors across two overall state-implemented educational initiatives and across two different types of integrated academic and behavior multi-tiered systems of support within and between two states. The exploratory and confirmatory factor analyses that have been conducted in both SPSS and Mplus on the five datasets (Midwestern Collaborative Work, Western Effective Behavioral and Instructional Support Systems, Midwestern School-Wide Positive Behavior Supports, Western Schoolwide Positive Behavioral and Instructional Supports or Response-To-Intervention, and both the Midwestern and Western states combined) consisting of the common 28 School Implementation Scale items reveals that the 4-factor structure specified for the SIS does not adequately fit any of the datasets. The SIS, however, initially consisted of 31 items. Statistical analysis of the 2014 Midwestern state’s dataset with 31 items also reveals that the data does not adequately fit the 4-factor structure specified for the SIS. Therefore, the 4-factor structure may have been an incorrectly specified model for the School Implementation Scale, and should be reviewed then revised. This study concludes that the SIS with its current 4-factor structure, for either 28 or 31 items, is not a valid and reliable measure of the implementation level of integrated academic and behavioral multi-tiered systems, and should not be used across various state populations and different forms of multi-tiered support systems.