Improving the Evaluation of Summer Interventions: How Testing Intervals Affect Summer Learning Loss Measurements

Abstract

This study considered how the timing of assessments affects summer learning loss estimates and the evaluation of interventions designed to reduce summer learning loss. By utilizing an efficient assessment called reading curriculum-based measurement (R-CBM), measurements of seasonal learning were calculated with a minimized school influence schedule that compressed out as much school time from the summer vacation period as possible to compare with a more traditional schedule. Measuring based on the minimized school influence schedule resulted in summer learning loss of 17.370 words per minute compared to learning loss 5.564 words per minute. Based on these measurements, summer learning loss was underestimated by 219%. The study also revealed that controlling for testing interval by including the testing dates does not completely account for this underestimation because the learning growth rate at the end of the year is significantly greater than the rest of the school year. The growth rate at the beginning of the year was not significantly different than the rest of the year but this may be because of the tendency of teachers to focus on relationship building and classroom management at the beginning of the year. Follow-up interviews with teacher showed the increased growth rate at the end of the year possibly occur because teachers meaningfully change instruction to maintain the attention of their students as the school year end approaches. This study also considered if the patterns in the data support the findings of previous research of a compensatory effect of schools. The results show children on free/reduced lunch learn at a slower rate than children not on free/reduced lunch during the school year, but then the growth rates are similar during the summer. The effectiveness of Summer Boost, a learning loss intervention, were also considered. While utilizing the minimized school influence model improved the outlook for Summer Boost, the resulting gains were not statistically significant. However, this result may be because of a small sample size (the treatment group included 68 students) rather than the quality of the program.