Leigh Parise: Policymakers talk about solutions, but which ones really work? Welcome to Evidence First, a podcast from MDRC that explores the best evidence available on what works to improve the lives of people with low incomes. I’m your host, Leigh Parise.

The pandemic has led to unfinished learning for a broad swath of students. It has also exacerbated existing disparities in student outcomes by race and ethnicity, income, and geography. Many states and school districts are eager to scale up high-dosage tutoring programs, a proven way to accelerate student learning. But there are challenges getting these programs successfully off the ground, like recruiting tutors in a tight labor market, fitting sessions into already packed school days, and ensuring consistent student participation.

To help practitioners address these challenges and ramp up their high-dosage tutoring programs, the University of Chicago’s Education Lab and MDRC launched the Personalized Learning Initiative (or PLI) in 2021 in collaboration with a consortium of researchers and funders. PLI is supporting and studying the expansion of high-dosage tutoring programs in school districts across the country. As part of that project, members of the PLI research team have had the opportunity to engage with multiple school districts and tutoring vendors that have already been trying to implement such programs on a rapid timeline.

Today I’m joined by Barbara Condliffe, a senior research associate at MDRC, who’s leading the implementation research on the PLI study. Barbara’s here to discuss this important work and some of the early lessons we’re learning. Barbara, thanks so much for joining me. I’m really excited that you’re here.

Barbara Condliffe: Thanks, Leigh.

Leigh Parise: I am personally especially excited to be talking to you about this work. When I think about work that we have had the opportunity to do over the more than a decade that you and I have both been here, this work feels so directly relevant to what schools are grappling with every single day. So I’m really excited that you’re here. I know that you’ve been out in the world getting to work closely with practitioners for several years now and have been seeing the kinds of things that they’re wrestling with. I think it’ll be instructive for people to hear some of what you’ve been learning and some of what you’ve been seeing on the ground. This is great.

Why don’t we start at the top? Tell us about the goals of PLI and about our partners on this study.

Barbara Condliffe: The PLI study is a collaboration with our partners at the University of Chicago’s Education Lab. They have done a lot of groundbreaking work already to demonstrate the benefits of high-dosage tutoring. The PLI study, in the wake of the pandemic, seeks to understand whether and how we can replicate the effects of those prior studies as districts and states and charter management organizations try to scale these programs to serve many more students.

In addition to that, we are also getting to work with these local partners to help design more affordable alternatives, because we know that one of the key barriers to scaling high-dosage tutoring is the high cost. This study is both helping and documenting what it takes to scale these programs while also testing their effectiveness. And, importantly, we’ll eventually be able to say [that] not only do these types of programs work, but who is best served by the different types of tutoring programs that are out there. It’s a really big study: To date, the study has involved over 18,000 students, 100 schools in eight localities across the country, and it is still going. It’s a really exciting project to be a part of.

Leigh Parise: Wow, that’s incredible: almost 20,000 students across so many different locations. This is really cool to get to talk about. It’s possible that I should have started here, but say a little bit about what high-dosage tutoring is?

Barbara Condliffe: We define high-dosage tutoring as being consistently scheduled tutoring [sessions] with a trained tutor and a small group of students—for example, four students to one tutor—that’s held at least three times a week. Importantly, this is not homework help; this is tutoring that involves a structured curriculum that tutors have been trained to deliver. That’s the form of tutoring—that sort of small group, high-dosage [tutoring]—for which we have the most evidence.

As I mentioned earlier, the study is also helping to put in place alternatives to high-dosage tutoring that try to preserve the quality while dialing down some of the cost drivers. We are learning with our partners about things like integrating more education technologies, which might allow us to increase the group size, for example.

Leigh Parise: That, I think, is really helpful for people to hear. I have been in education research now for almost 20 years, and last summer, Barbara (as you know), I took a deep dive into what it would look like to [include] a high-dosage tutoring program as part of my daughter’s school’s after-school program. The people who run the program asked me to put together the research and think about what the key components would need to be. When I shared the research with the after-school team, I think they were shocked at how complicated it would be to implement this [program] well—and understand the level of dosage necessary, the training that would need to happen, the connections to what was happening in the classroom. I’m glad that you walked through that, because I think people [think], Oh, tutoring; right, my kid gets tutoring, but thinking about what the research says are really effective forms of tutoring feels like a central part of this study. I appreciate you walking through that.

One follow-up question. When you talked about this study and what we’re trying to accomplish with EdLab [the Education Lab], you said, “We’re focused on scaling.” I’m curious for you to say a little bit more about what you’ve been hearing as you’re working with districts and charter organizations across the country. Is this an issue that they’re grappling with, trying to figure out how to reach more students? Can you say a little bit about what you’re hearing from school administrators or district administrators about that?

Barbara Condliffe: Sure. As the listeners may know, tutoring was among the investments that states and school districts could make with their ESSER [Elementary and Secondary School Emergency Relief] funds. Given the prior evidence of tutoring’s effectiveness, I think there was a lot of interest across the country in launching these programs. But doing so to reach the number of students that could benefit from the program is a challenge and it’s complex.

As you mentioned, Leigh, in helping your school design a program, tutoring itself is a relatively straightforward intervention; it’s not that hard to get our heads around. But putting all the pieces into place can be complicated. And you were talking about an example of just your one school. When you’re in a district office and you might have a high number of schools trying to launch this thing, it gets even more complex. One complexity that we are seeing a lot is the challenge of identifying, hiring, training, and retaining a tutor workforce. To preserve these small groups, we need a lot of tutors, and staffing is a major challenge. That’s something we’ve heard about at every level. When we talk to school leaders, when we talk to coordinators, they’re often sharing the challenges they’ve faced when recruiting and then retaining a tutor workforce.

It’s been really interesting to be hearing about these challenges and also seeing some of the really creative solutions that our partners have come up with to address these challenges. My colleague, Rebecca Davis, released a brief that’s on our website that talks about the different models we’ve seen. Some of our partners have done this in-house. The City of Chicago, for example, launched its Tutor Corps program, where CPS [Chicago Public Schools] managed the hiring of this really large corps of tutors—they did the vetting, the hiring, and then they worked with partners to do the training of the tutors. That’s one model.

Other models—for example, Fulton County Schools is working with both internal employees and also a slate of external vendors that the district vetted to say, “Here are your choices, schools,” and then schools were able to pick a vendor that did the hiring of the tutors themselves. We’ve also seen examples like in New Mexico, where they’ve said that one of the reasons it’s hard to recruit is getting folks who can come to the school because of distances they might have to travel in a state with a lot of rural areas—or perhaps because of schedules. So they launched a virtual program where they recruited and hired folks who could work virtually. Those are just some of the examples of the different ways we’ve seen folks address this challenge.

Leigh Parise: Thanks for sharing that. I love that you immediately went to Right, let me give you some concrete examples of what we’re seeing on the ground and what we’re hearing from people. Thank you for that.

I know this study initially launched in 2021—a wild time for the world at that point—but we’re a few years in. Is there anything that you can share about what the project has learned so far?

Barbara Condliffe: Recently our colleagues at the University of Chicago released a technical brief with some preliminary findings on the impacts of high-dosage tutoring—so that small-group, high-dosage tutoring—from school year 2022, 2023, in two of our participating localities (Chicago and Fulton County). Now, this is preliminary, but because of the interest in the field, the team felt it was really important to put these initial findings out there. What we’re finding is that overall, and for reading, the impacts on student achievement that they’re seeing are still too noisy. The sample sizes aren’t where they need to be for us to feel confident in there being conclusive findings, and we don’t see any statistically significant differences in student learning.

However, for students who were assigned and received high-dosage tutoring in math, we’re seeing really promising results. For students who received math tutoring, the impact on their learning is equal to about two-thirds of a typical year of learning. As you think about the scale of unfinished learning that we’re seeing, particularly after the pandemic, this shows that—even as Chicago and Fulton County were scaling up anything that had been done in these prior studies of tutoring—this preliminary evidence suggests we’re able to realize impacts that are pretty similar to those much-smaller-scale studies and making a real dent in helping students to catch up.

I think also we’ve been sitting side by side with these localities since 2021 as they’ve been launching these programs, so we’ve learned a lot about what it takes to develop the infrastructure, both at the district or state level—the entity overseeing the program—as well as the school level. We’re learning about key ingredients that are not entirely surprising to folks who’ve been doing this kind of work for a long time, but do take really careful planning and thinking through of resources, in terms of the time for folks to oversee these programs. For example, we’re learning about the importance of having data systems in place for the tutors to be entering student attendance and student progress at tutoring, as well as things like routines for folks at the school level, at the district level, to be making sense of that data and acting on it.

Sometimes programs are put in place pretty quickly, and in the early years, we see that some of those pieces of infrastructure aren’t yet in place. Over time, if we can help folks get those pieces into place, I think the implementation of that core of tutoring, that consistent student-tutor interaction, can happen much more easily.

Leigh Parise: Great, thank you for sharing that. I think I’m hearing—what you’re saying is, it takes time to get some of these structures into place, and it is important to be thoughtful about what the infrastructure needs to look like and who the staff are going to be. Then down the road—hopefully not too far down the road—you’ll be able to reap the benefits of getting these systems into place, so that you can be serving more students and making sure that they are seeing some of the gains that we’re seeing initially. Which is exciting.

You mentioned attendance, which on the one hand feels obvious: Students need to be attending these sessions in order to actually reap the benefits. But it’s probably something that people don’t quite think about as often as they should, or might not immediately think, Right, of course that’s one of the things you’re going to need to have a plan for. Can you say a little bit about why it’s so hard to ensure adequate attendance, especially when tutoring is taking place during the school day?

Leigh Parise: It’s a great question, because one early lesson we learned in this study was the importance of the first thing you said: making sure that tutoring is happening during the school day. Some places across the country have said, “Oh, it’s really hard to schedule tutoring and there’s a lot of important stuff happening during the school day. Let’s make these after-school programs.” Our observation, as we’ve been launching the study, has been that that’s really hard to pull off well. An after-school [program] is more of an optional activity, and so students may or may not sign up—the students who need it most. So you see challenges with things like making sure the right students are getting tutored and achieving strong dosage in after-school [tutoring].

We have worked within our sites to push the in-school models. Yet student attendance at tutoring is among the most commonly talked about challenges when we survey folks who coordinate these programs, and school leaders, and what have you. What we found is, first off—the elephant in the room, I’d say—is that students can’t attend tutoring if they aren’t in school when tutoring is held in school. We know that student absenteeism is a challenge nationwide, aside from the tutoring. So it would be a big ask to expect that tutoring programs are going to move this complex issue of student absenteeism from school.

And yet, even when students are in school, we are sometimes seeing that they are not consistently getting the tutoring program. Why is that? We’re finding that tutor absence can be a challenge. Sometimes we talk to sites where they’d say, “I lost some tutors during the school year”—so tutor attrition, delays in hiring. Tutor absence, just like teacher absence, can be a challenge.

But even when tutors and students are both in the building, sometimes sessions get canceled. I know we were both teachers, Leigh, and we know this: School schedules are packed and there are a lot of demands on the time of students. So sometimes we found that schools were having to cancel tutoring sessions in favor of other activities. Sometimes it was assemblies, or field trips, or state testing. We also heard from tutors—and sometimes teachers—that teachers sometimes didn’t want to allow their students to go to tutoring because they were concerned about them missing classroom instruction, or the student needed to go get other interventions or services. Tutoring doesn’t want to be taking away from other important things that students might need. That’s why it is so important that tutoring be scheduled and be built into students’ schedules, the school’s schedules, from the start, so that that time can be protected.

Additionally, we heard a lot about the importance of making sure that everyone in the building—even if [one is not] delivering tutoring, is fully aware of and on board with this initiative. So school leaders, teachers—making sure that they understand the benefits to their students of receiving that consistent dosage of tutoring and know who’s going to be pulled for tutoring (and when) so they can help get students there. That is just so important to making sure that students get that dosage and that we avoid things like session cancellations.

Leigh Parise: I appreciate those examples of how some schools are trying to deal with this challenge by building it into the schedule and then making sure that there’s broad awareness and buy-in. You can’t underestimate the importance of making sure that everybody is aware that this is something that the school is doing, to both support the students and to support the teachers and what’s happening in the classrooms more broadly. Is there anything else that you want to share with listeners that you’ve seen schools doing to try to address this challenge, related to attendance and making sure that students are getting everything that they can from their tutoring?

Barbara Condliffe: I mentioned earlier the importance of building that infrastructure piece, those data systems. Something we saw pretty early on was that it’s really hard—even when there is good buy-in—for folks who are overseeing tutoring programs to intervene early if there is a challenge with student attendance if they don’t have good data. Seemingly small things can make a big difference. Sometimes tutors, if they are externally hired, didn’t necessarily have access to the student information systems where they could enter information on student attendance. Or tutors didn’t have time in their schedules to take attendance; things like that. So making clear what those expectations and routines are for data entry, making sure folks have the right access—and, again, making sure that there’s someone at the school who is keeping their eye on this so that we can intervene early and make sure that the students who’ve been identified for this resource are optimizing it or having the chance to benefit as much as possible.

Another thing I’ll mention is that we also need to make sure that the folks who are at the core of this intervention, the students and the tutors—[that] we are attending to their motivations for showing up every session and getting the most out of the intervention. For the most part, in surveys, we heard tutors speaking of [finding] real enjoyment in their sessions with the students. Still, we saw some differences between schools, and the extent to which they embraced tutors as a part of their school community and made them feel like a welcome member of the staff. You’ve got to think—as we think about these challenges with things like tutor attendance and turnover and absence—that making sure that these tutors who are coming into the building or joining virtually feel like they are part of the school community is going to make a difference there.

Finally, students themselves—we also heard a lot, particularly from tutors and from the coordinators and teachers, how happy (oftentimes) students were to go to tutoring. But we also know that sometimes tutoring might not be the thing a student wants to do, and so they might be preferring to stay with their friends in the classroom. Or perhaps for older students, we sometimes heard about some stigma associated with being pulled. So thinking carefully through those things and helping to address challenges students might be facing or concerns they might have about going to tutoring and making tutoring a positive part of school culture. We saw some schools rewarding students for consistent attendance, giving students extra time at recess for doing a great job with tutoring, giving them points at the school store. Celebrating their success went a long way as well.

Leigh Parise: That’s great. I see how that could make a big difference for students. And not having to miss the fun stuff in the classroom: so [making] sure it’s scheduled in a way that students aren’t being pulled from the things that otherwise might make them feel excited or motivated about being at school.

Barbara Condliffe: We have more to do to understand the potential here, but we did hear from some schools about things they were doing to bring families in. Even though this is during the school day, it’s still so important that parents are made aware of this resource that their [children] are getting. [They] can be real partners in working with the school if there are challenges with students consistently attending tutoring. I think we see some promising examples of things like parent contracts and showcasing tutoring at back-to-school nights. I think there’s more for the field to understand, there, in how to make them part of these tutoring programs moving forward.

Leigh Parise: I love that: making sure that parents are part of the team. That’s great. You mentioned that we have a lot more to learn. Tell us about what’s next for the PLI study.

Barbara Condliffe: Over the past few years we have seen many different variations of tutoring programs put in place. All the schools involved in PLI share some common principles that they put in place—a structured curriculum, group sizes for those that are high-dosage tutoring, consistent dosage scheduled throughout the week and the year. But there’s also a lot of really exciting and interesting variation in how they have designed and launched these programs, including some of the levers that our team and these localities tweaked to try to help serve more students and bring costs down—things like larger group sizes, integrating more education technology.

Over these next few years, I’m really excited for us to be documenting for the field what these variations were, what we learned about how to implement them well, and where there were challenges—as our partners at the University of Chicago Education Lab dig into understanding the impacts of these different types of tutoring programs and what types of students are best served by the different program types.

We also have a huge number of tutors who are involved in the study, and we surveyed these tutors a number of times over the past few years. I’m looking forward to digging into understanding who constitutes the tutor workforce in these localities and what can they tell us about their experiences with tutoring: What went well, what was challenging, and where are they going next? There’s been some thought in the field: Might tutoring be a pathway into the teaching profession? I’m excited to dig into the data to understand a little bit more about who these folks are and where they might go next in the field of education.

Leigh Parise: That’s great; that’s really exciting. This feels like a situation where the successes and challenges that [the PLI] schools are experiencing—staffing, scheduling, student and family engagement—are all issues that, I imagine, are at the forefront for many schools right now. It does sound like there’s lots of opportunity to get to learn and share lessons across the different places that are participating that I'm sure will be relevant much more broadly.

Barbara, thank you so much for joining me. This has been a really engaging discussion, and I appreciate your focus on what matters for students, practitioners, and policymakers; what we’ve learned; and what some of the future opportunities might be. Thank you.

Barbara Condliffe: Thank you, Leigh. This was great.

Leigh Parise: To learn more about PLI, check out a brief by Marissa Strassberger and our guest, Barbara Condliffe, featuring educators’ advice on high-dosage tutoring programs.

Did you enjoy this episode? Subscribe to the Evidence First podcast for more.