Educational practices that prove effective often involve a substantial investment of training and support time. Unfortunately, that is the very characteristic that makes it difficult and costly to expand the programs on a large scale. A key challenge when scaling up is to maintain high levels of effectiveness at an affordable cost.

MDRC’s evaluation of the United2Read expansion focuses on a scale-up strategy that employs technology to reduce the implementation cost of a proven intervention. Funded by an Educational Innovation and Research grant from the U.S. Department of Education, this study examines whether an alternative delivery model is as effective as the original delivery model and whether it can serve as a viable way to cut program costs and expand the intervention.

The intervention, Assessment-to-Instruction (A2i) professional support technology, is a data-driven system designed by and for teachers. A2i technology integrates several key features: First, it offers online adaptive assessments to measure students’ skill levels on several dimensions of reading. Second, it uses information collected through the assessments to compute recommended amounts and types of literacy instruction for each student. Finally, to make it easier to follow through on those recommendations, the system provides embedded lesson planning linked to a district’s reading program, as well as data visualization tools and online professional development tools.

This intervention has strong evidence of efficacy based on randomized controlled trials and quasi-experiments conducted since 2005 in 28 schools in Florida and Arizona. These studies repeatedly demonstrated that schools using A2i can accelerate gains in literacy during the crucial early elementary grades for all students, including high-need students, children living in poverty, English learners, and children receiving special education services.[1]

In these studies of A2i, the system was delivered to schools and teachers through intensive in-person training and ongoing professional development, tailored to teachers’ beliefs, skills, and knowledge, throughout the school year. This high level of personal attention makes the intervention, at about $150 per student per year, more expensive than many schools can afford.

To reduce the cost of the A2i intervention, the United2Read team (Digital Promise; Learning Ovations; the University of California, Irvine; and MDRC) designed an alternative, technology-based delivery model. The tech-based model includes online webinars rather than face-to-face workshops, and Professional Learning Communities mediated through live video conferencing (for example, via Zoom, Skype, or Google Hangout). In time, the tech-based model will also allow teachers to upload video of their classroom instruction (to a secure Cloud server) so that the teacher and a coach from Learning Ovations can watch the video together and discuss instructional challenges and ways to overcome them. This approach aims to sustain A2i’s emphasis on personalized professional development while improving coaches’ productivity by reducing their travel time and allowing each coach to work with more schools and teachers.

This evaluation, therefore, focuses on the alternative delivery model as a strategy for expansion and uses a school-level randomized controlled trial design to answer the question “Can a more technology-based, less costly model be as effective as the in-person model for A2i implementation?” The study team has recruited 60 elementary schools from 20 school districts in four states around the country to participate in this evaluation, with half receiving the in-person model and half receiving the tech-based model. Participating schools were assigned to one of these two models through a lottery conducted by MDRC in summer 2018. In the 2018-2019 school year, the two versions of the program are being implemented in kindergarten and first grade; in 2019-2020, the evaluation will be expanded to include second grade; and in 2020-2021, third-grade classes will implement the program. The evaluation will conclude with a comparison of third-grade state reading test scores for students who have been at the study schools for all three years.

While the randomized controlled trial examines the relative effectiveness of the two models, the team will also conduct a supplementary comparative interrupted time series study to assess the effectiveness of the tech-based model compared with business as usual in similar schools.

Results from this evaluation will provide valuable information on the viability of using technology to bring A2i to classrooms across the nation at a cost they can afford.