Strategic Adolescent Reading Intervention (STARI) Evaluation
Overview
Middle school students who struggle with their reading are at a greater risk of not graduating from high school. Yet helping these students catch up is particularly challenging due to the changes in motivation and engagement that occur during this period of development.
Developed by the SERP Institute, Harvard University, and Wheelock College, the Strategic Adolescent Reading Intervention (STARI) is year-long program intentionally designed to address the motivational barriers faced by struggling middle school readers. STARI tackles gaps in students’ basic reading skills, but unlike other interventions, it seamlessly integrates those skills with complex comprehension tasks and grade-level reading instruction and gives a central role to student motivation. Students receive the STARI intervention in addition to their regular English Language Arts class, during an elective period or an intervention period. All STARI curriculum materials are freely downloadable. A small-scale randomized trial of STARI, conducted in school year 2013-14 in a Northeastern state, found statistically significant positive effects on students’ word recognition, reading fluency, and morphological awareness.
Based on the strength of this evidence, in 2017 SERP was awarded an Education and Innovation in Research mid-phase grant from the U.S. Department of Education to support scale-up and evaluation of STARI in several high-needs school districts. The intention was to implement STARI in four urban school districts for three school years (2018-19, 2019-20, and 2020-21). This timeline, however, was deeply impacted by the COVID-19 pandemic, making it challenging to implement STARI as intended and creating significant barriers for data collection. Due to these exceptional circumstances, STARI implementation was extended to include a fourth school year (2021-22) in two of the four study districts, thereby making it possible to evaluate the STARI’s effects during that school year.
The evaluation, which focuses on STARI’s effects in 2021-22, was conducted during an unprecedented time when struggling adolescent readers fell even further behind because of the instructional disruptions caused by the pandemic. In 2022, only 31 percent of U.S. eighth-graders could read at a proficient level, down from 34 percentage points prior to the pandemic. STARI’s design makes it well-positioned to address these learning gaps and to help struggling adolescent readers make up lost ground.
Additional Project Details
Agenda, Scope, and Goals
The purpose of the evaluation is to examine whether STARI’s effects on students’ reading skills can be replicated in other settings. Specifically, there are two research goals:
- To assess whether, after one year of intervention, eligible students assigned to a STARI class have stronger foundational reading skills and more positive reading behaviors than students not assigned to a STARI class.
- To assess whether, after one year of intervention, eligible students assigned to a STARI class have higher levels of general literacy achievement and better academic performance in their English Language Arts classes than students not assigned to a STARI class.
In addition to building evidence on STARI’s effectiveness across different contexts, the evaluation also aims to contribute to the literature on strategies for helping middle school students who are reading below grade level catch up.
Design, Sites, and Data Sources
The evaluation was conducted in 11 middle schools in two urban school districts during the 2021-22 school year. Students in these schools were eligible for STARI if they were reading more than one year below grade level. A total of 398 eligible students were eligible.
The effect of STARI was evaluated using a student-level random assignment research design. A lottery-like process was used to determine which eligible students would be offered STARI for one school year (STARI group) and which students would not be offered STARI (control group). Random assignment was conducted separately by school and by grade level. The effect of STARI for eligible students was evaluated by comparing the outcomes of students in the STARI group and the control group. Students in the control group could take any of the other “business as usual” classes offered at their school, including another reading intervention, another academic class, or a non-academic elective, so the findings represent the effect of STARI over and above the effect of these other classes.
Students’ foundational reading skills and reading behaviors were measured by administering a reading test and survey to students in Spring 2022. Students’ scores on English Language Arts (ELA) state tests and their marks in their ELA courses were obtained through student records. Sample sizes are smaller than expected due to low parent consent rates during the pandemic.
Despite these limitations, the evaluation’s timing creates an important, policy-relevant context for examining the replicability of STARI’s effects and an opportunity to build evidence on strategies for helping struggling adolescent readers catch up.
