As part of our study of Making Pre-K Count, an innovative preschool math program, we conducted in-depth qualitative research to better understand how the program produced its effects. The program has several instructional components that are directly measurable, but one key process-related ingredient in the program’s theory of change is less well understood — that is, teachers’ ability to differentiate their instruction to individual students’ needs. What can we learn about a main intervention component that is challenging both to implement and to document?

The Making Pre-K Count study

The intervention in Making Pre-K Count used Building Blocks, a pre-K math curriculum that includes 30 weekly lesson plans consisting of four main components: (1) whole-group activities; (2) small-group instruction, led by a teacher with three to four children at a time; (3) Hands On Math Centers, offering activities children can play with on their own or in a small group, with or without a teacher; and (4) computer activities. In the study, the curriculum was implemented by teachers over two school years and was supported by 11 days of teacher training and ongoing, in-classroom coaching provided by Bank Street College of Education. Teachers were trained in differentiating the math content and activities, an instructional approach that involves keenly observing children and tailoring instruction to meet each student’s learning styles, abilities, skills, and interests.

The program successfully changed teachers’ math practices in the pre-K year, but those impacts did not translate into consistent effects on children’s outcomes at the end of pre-K. By the end of kindergarten, however, as described in a newly released report, the program did lead to slightly improved math skills, more positive attitudes toward math, and stronger working memory skills for children.

The Making Pre-K Count study is the result of a partnership between Robin Hood, one of the country’s leading antipoverty organizations based in New York City, and MDRC. Additional funding was provided by the Heising-Simons Foundation, the Overdeck Family Foundation, and the Richard W. Goldman Family Foundation.

A deep dive on the theory of change

When we were planning the study, the Building Blocks program developers highlighted differentiating small-group math instruction as one of the key ingredients in its theory of change. They also acknowledged that it could be difficult to implement and that there is little knowledge in the field about how teachers differentiate within a classroom setting and how it affects children’s learning outcomes. In particular, research is lacking on how teachers in pre-K approach differentiated instruction.

With many of the other main curricular components measured quantitatively via coach logs, teacher surveys, and structured classroom observations, our study team decided that an intensive, qualitative study of this aspect of Building Blocks, consisting of classroom observations and teacher interviews, would be more useful to the field than a general set of interviews about curriculum implementation. We observed 20 out of 87 Building Blocks classrooms, identifying instances of teachers differentiating instruction for individual children and groups of children. Then we interviewed those teachers immediately after the observation to understand their reflections on their work.

We found that teachers were differentiating their instruction in line with how they were taught in Building Blocks training sessions. But while teachers reported using a variety of approaches to differentiate instruction for students with stronger skills, they were more limited in the approaches they used for children struggling with math. For struggling learners, teachers approached differentiating through repetition and making curricular activities easier. In interviews, some teachers expressed doubt that students with lower skills at the start of the year would be able to learn new math skills, or they expressed hesitancy about challenging children who struggle.

The pre-K impact study found small, positive impacts on two measures of children’s math skills for children entering pre-K with higher receptive language skills but not for those with lower language skills. In our study, teachers commented unprompted that children who were struggling with math had lower verbal skills and that children excelling in math had higher verbal skills. It may be that teachers’ self-described hesitance to challenge struggling children influenced how different children experienced the program.

Our research can help explain our findings and inform the theory of change for Building Blocks as well as the strategies for supporting teachers who implement it in the future. For example, in future Building Blocks replication efforts, coaching or technical assistance could add a focus on helping teachers more confidently differentiate instruction for struggling students.