Differential construct definitions of six change score models within a correlational research context

Abstract

Six change score models were comparatively evaluated within the correlational research context. The models compared included raw change, corrections of raw change for unreliability in x, correction of raw change for unreliability in both x and y, a regression correction, the raw residual model, and the base-free measure of change. The data were simulated for nine different parameter conditions. The manipulated parameter values were reliability coefficient values for x, y and w where x and y were the components of change and w was an outside variable, relative variability of x and y, colinearity between x and y, and relative validity coefficients for x and y. A set of true and two sets of observed change scores (total of 18 models) were generated for 2000 cases under each condition. Correlations among scores between models within and across conditions were generated. A principal component analysis was used to investigate the commonality of the change score models regarding the construct definition of change when w was considered and when w was partialed from the change score models. The latter analysis investigated the possible differential impact of w on the construct definition of change.

The findings revealed that model differences do exist between the change scores under most of the parameter conditions, particularly for $\sigma\sb{x} = \sigma\sb{y}$ where $\rho\sb{xy} < .50$ and $\sigma\sb{y} > \sigma\sb{x}$ where $\rho\sb{xy} = 0.75$ when $\rho \sb{xx\prime} \not= \rho\sb{yy\prime}$. Selected parameter conditions had differential impact on discrepancy models versus residual models. Discrepancy models were more susceptible to manipulations of x and y variability, while the base-free measure of change was most affected by different reliability levels and colinearity coefficients. Removal of w had differential impact on the change score models.

The results of this study lead to a conclusion that change scores in the form of any of the models are not sufficiently stable across research conditions to provide confidence in their use. Those conditions most favorable to change scores are rare in practice and use of a single variable (y) will result in an equal amount of information.