Leigh Parise: Welcome to Evidence First, a podcast from MDRC that explores the best evidence available on what works to improve the lives of people with low incomes. I'm your host, Leigh Parise.

We know the pandemic had devastating effects on learning, and one of the biggest questions on the minds of school leaders is how do we tackle students' unfinished learning? One strategy that's often mentioned is individualized instruction, sometimes called individualized or personalized learning, and it can include things like tutoring and other differentiated supports. But what does that really entail and how do you do it well? And how would schools actually implement individualized support for students?

Today I'm joined by Andy Rotherham, co-founder of Bellwether, a national nonprofit focused on improving education and life outcomes for underserved students. Andy also wears a bunch of other hats, including being a contributing editor to US News and World Report, and he's a senior editor at The 74, an education news and analysis publication. He also writes the very popular Eduwonk blog, which of course I read, and he's now a member of the Virginia Board of Education.

Also, full disclosure, I'm very lucky that he's also on the board of a nonprofit that I started with my Olympic gold medalist brother, Steve Mesler, called Classroom Champions. Also joining me is my dear colleague, William Corrin. He's the director of MDRC's K-12 education policy area. William is a former high school social studies teacher in the Charlotte-Mecklenburg Public School system and was the district director of research, evaluation, and assessment in Evanston, Illinois. William has been at MDRC for 18 years, and he's led numerous large-scale studies of educational programs. He's also the person who hired me at MDRC about 11 years ago, and so I'm very excited to get to have the conversation with these two guys today.

Andy Rotherham: Well, it's great to be here. Thanks for having me. Leigh, I just tremendously enjoy the work we get to do together at Classroom Champions, and William and I, we compare notes on professional stuff and also music, and so I'm not going to let him off the hook without talking about his band and so forth during this podcast.

William Corrin: I'm William Corrin, and I'm the director for K-12 education here at MDRC. And, yes, I moonlight a little bit as a bass player and try to make that go, but it doesn't pay the bills. But it is nice to have a colleague in the field, like Andy, we could talk about education stuff, but then we could also lighten it up and talk about things like events that he and his wife support and put on and what happens to me on stage and in the studio.

Andy: Yeah, I enjoy comparing notes on music. We haven't been able to get you guys down. My wife and I host these private concerts in this little one-room schoolhouse called the Schoolhouse Series, but first the pandemic disrupted that. Obviously, we had to put that on hold because it's a wonderful acoustic, acoustics of it are great, but the ventilation a little less so. So we had to put that on hold. And then now with my kids, the age they're at and sports and particularly my daughter who plays ice hockey, finding weekends where we can get a really good act, get the schoolhouse, and where we're not off somewhere is proven really challenging.

But those are one of those fun things that we get to do because it brings people together across all kinds of lines of difference to just relax, enjoy live music, support working musicians. And I feel like something we can talk about just how contentious things are right now—we just need opportunities to have a break from that and to experience things, and a lot of the artists we get sort of deliberately transcend various lines of difference and that's really fun.

Leigh: All right. So I want to start out talking about the idea of individualized instruction. Lots of people want to use this as a strategy to help get students to where they need to be. And this is especially true, I think, now that the most recent NAEP scores show a really steep decline in reading and math scores for kids a couple of years after the pandemic first started. What do you guys think about this idea of individualized or personalized instruction?

William: Right. I mean, I think for today it's going to be nice to talk a little bit about personalized or individualized learning, but the idea is how do we meet individual students where they are? Education in schools [is] about serving groups of kids altogether, but we also know that there's variation on in terms of where different kids are. We know it varies across districts, we know it varies across schools, and we know it varies within classrooms. So, the idea is how do you think about supporting the individual kid in the ways that that kid needs help? Ideally, you can do that for groups of kids at the same time, but there's some cases where it—just say for various reasons—kids [are] falling behind on something or kids having an issue at home. And so how do you personalize or individualize the support for that student?

And the value of digging into that topic is that there's actually evidence for things like tutoring, high-dosage tutoring done well and evidence behind some types of adaptive math education technology that are both different ways of trying to help the individual kid, understand where that student is, and then target the support for that student so it can best address the learning needs or other challenges that the student is facing. Whether we call it personalized or individualized or something else, the main point is how do we ensure we're paying attention to all kids?

Andy: And that's probably especially important right now, just given the scale of what we're facing. And we talk about learning loss in these broad [ways]; the NAEP scores just recently came out and was focused on that. But you're seeing those impacts are average impacts and kids are going to be impacted more or less, and we're going to have to have some kind of a customized strategy to help kids, particularly the kids who are furthest behind, who not surprisingly were the kids who were furthest behind before the pandemic. If we're serious about catching them up, it's going to be hard to do that if we just spread interventions around like peanut butter, as they say, rather than really target them on what do individual kids need to get back to where they need to be.

Leigh: Do you have a sense for what schools are doing now to address some of the needs that students have?

Andy: It's completely haphazard. If you mean these out-of-school needs, it's haphazard, it's politicized. In some cases, schools are being asked to do a lot and then you've got a bunch of people asking them to do even more. I think, as of most things when we talk about schools at large—50 states, 13,000 school districts, you can find examples to—there's a thousand facts or a thousand theories kind of thing going on. So, I would say, if I had to use a word, I would say it's haphazard. And that's also how parents are experiencing this and how they're responding. And families, they don't think about this stuff all the time, they have a variety of needs, people are trying to live their lives, they have various stressors and so forth on them, other things going on. So, the response there is haphazard as well. Not because parents don't care deeply or want to do the right thing, but life is challenging and time-constrained and all of that.

I think that's why we're going to continue to see a general unevenness in the recovery. And really since the beginning of this pandemic, everything has been K-shaped. The affluent have been able to skate along. When the economy tanked in the early part of 2020, they were all ordering their food from DoorDash and so forth. And other people were really struggling to figure out how to make ends meet in the initial weeks of that. You saw a K-shaped academic impact. I mean, I wouldn't want to say this has been easy for anybody, but it's certainly been more challenging disproportionately for kids who are already furthest from opportunity, which in our country that's low-income students, racial and ethnic minorities, and so forth. And we're still seeing that play out, so I would say haphazard and uneven are the watchwords still of the day. And what kids need from us is a little less haphazardness and a little more unevenness in terms of what they need from government and the policy community right now.

Leigh: So personalized or individualized instruction could be one of the approaches that policymakers take if they're interested in adapting a less haphazard and more focused strategy to dealing with unfinished learning. But the problem is that when you hear individualized or personalized learning, I think there are some people who take issue with that or it makes them nervous or they want to challenge that idea. Andy, do you want to say a little bit more about that? I feel like you travel in a lot of different circles and could give us probably some perspective here. What do you think are some of the different ways that people interpret that term or that approach?

Andy: Yeah. I mean, one of the great things about my job, Leigh, is I do get to travel in different circles. As I said, we do evaluation at Bellwether and so I interact with the research community; I'm on advisory boards for different research studies and stuff and that's fascinating work. I also get to interact with media and the policy and the political communities. And so it's really a fun place to be at the cross currents of these because you pick up on different trends and patterns. And I think the conversation, there's a couple, as with anything in education there's like the bad faith reading as soon as you say personalized or individualized, there's sort of bad faith readings of that that aren't particularly useful. But among the more thoughtful debates, you have people talking about this in the terms of what's technically possible. How much can you actually individualize at scale? Because what we are talking about is mass customization, technically what's possible there, what should we expect? What role can technology play? Questions like that.

And then you also have this fascinating debate. Karim Ani, for instance, Citizen Math has this argument he makes that if you look right now at the country and what we're struggling with at large, why on earth would we want to make things more personalized and individualized? We're focusing a whole lot on the unum and not much on the pluribus, and that maybe we need to rethink that. I think that's actually a fascinating argument for this. And as we think about—are we going to unbundle school more and some of the customized options that people avail themselves of? People of means availed themselves during the pandemic; we're going to see more of that. I think that's actually just from a values perspective, a super important conversation to have. And where do you want to personalize and where do you not?

William: Andy raises a really nice point in terms of it's not about just doing one thing or the other thing or the other thing. It's how do things work together in concert? So, we think that because there are multiple studies of tutoring programs that demonstrate that these tutoring programs help students catch up 2, 3, 4, 5 months of learning, that's great and we should put that into play and figure out how to do that at scale and try to help kids catch up. At the same time, what are the broader strategies that are going to touch more kids that may be already at scale or closer to scale so that we're trying to move everybody along the way?

To your earlier question, Leigh, one of the things that's interesting is I've started to hear some stories about there are parents will say, "Well, my kid doesn't need a tutor or my kid doesn't need that help." And so my perspective is—or with my own kids even, although they're older now—doesn't everyone want their kid to get some individualized attention that's really focused on where they are, whether they're behind, whether they're ahead, whether they're on grade level? Doesn't everybody appreciate some kind of personalized approach? People are happy to say, "Hey, my kid's in the gifted and talented program." That's another version, maybe it's more small group than individualized, but it's trying to recognize maybe where a kid is and where his or her potential may lie. But in all these cases, I think all of us should want the ability to have our kids engaged with some sort of support that's paying attention to who your student, your child is as an individual, what they know and where they can go.

Andy: Well, I think there's a revealed preference thing here, too, isn't there? That you see there's a multibillion-dollar industry in the affluent availing themselves of various kinds of personalized experiences and support, whether that's packaging their kids up to go to college or it's tutoring on various tests and so forth. People who have the means to avail themselves of those experiences for their kids disproportionately do so. And there's a lesson in that, and so it's figuring out how do we small-D democratize that to create those experiences for a wider range of kids and get them now?

Leigh: I think that's a really important point. Actually, one of the things I want to talk about—so, you talked about these kinds of choices that people are making and particularly those who have the means to figure out what it is their particular child could benefit from. When you think about the real-world considerations, though, from a school district’s perspective, what are the kinds of choices that they need to be making or how should they be thinking about implementing different kinds of programs or different kinds of tools to try to support individual students to meet them where they are, which is one of the places where you started, William?

William: So, it's tough, right? This interacts with resources and resources broadly defined; it's not just about: do you have the funding to do these things? We have federal policies trying to push a bunch of money into schools related to pandemic relief. So, it's not just about the money, but it's about people's time and focus, too. There's a lot going on as schools have been trying to deal with whatever level of closure they may have gone through a couple years ago aside of where they are now. How do you balance doing the individualized work against where you're going to put your resources? So, in an ideal world from a research perspective, we would be trying to do projects where we can learn about some of the variation that's possible when you attack these strategies. I'm excited that we're now doing a second project with Reading Partners.

We did a first evaluation with Reading Partners, their version that was in-person tutoring with kids, elementary school kids. And we saw they had an impact on the reading scores for these kids and improved their reading when they're exposed to the tutoring program. We're just starting a second project with them now that they'd already started experimenting with a virtual tutoring model actually before the pandemic hit, but now it's become that much more important. Now we're going to learn about, "Well, what does it mean to do Reading Partners virtually?" If you can do it virtually, then all the geographic constraints that are related to getting a tutor to a school face-to-face with a kid disappear.

Now, there may be some trade-off, what's the nature of the relationship for example, when you get to see a kid a couple times a week in person versus you're seeing them on the screen? That remains to be determined, but if you can increase access because you're no longer going to be bounded by the geographic constraint, which affects your resources around time and money to do the travel, etc., then that's sort of an important thing to learn. I think the more we look at these types of programs and can look at if you do them in different ways, particularly if those different ways engage resources differently, then you start to move towards being able to create, I think, a meaningful strategy about how to scale something.

Leigh: And in the districts that we're working with, that you're working with in some of MDRC projects, or, Andy, that Bellwether is engaging with, how do you think they're dealing with this set of choices ahead of them or the considerations that they need to make to figure out what it will take to actually implement these kinds of programs or tools to support students? And particularly in light, as you mentioned, Andy, of the recent NAEP scores that came out that really show these huge declines, that probably for a lot of school district leaders or school administrators weren't a big surprise because they're experiencing…they're engaging with students and parents every day and hearing from their teachers and they know that students came back and were struggling quite a bit.

Andy: Yeah, I think the NAEP scores were not...I think the surprise would've been if we didn't see that kind of impact. And so I think what you saw, they were sort of a....I don't know what you want to call it, just a high-profile signal or a taboo break, whatever, for people to talk about just what a catastrophe the pandemic was. Not because of the pandemic per se, but because of the policy choices that were frequently made and where we are. And some people just want to relitigate that for political reasons, but the more sort of important serious question seems to me is, "Okay, what do we do now? This impact is really serious and what do we do now?" And that's what I think was the importance of those is it starts to tee up the “what now” conversation. So, it was important.

I mean, look, we work with lots of school districts at Bellwether; it's a big part of the work we do. And we have an academic advising practice that's in schools and school districts, and they also do implementation support and other things. I think you're seeing of a range—without getting to any sort of specific clients—but just like you're seeing a range of approaches, there's a lot of urgency and a lot of concern. The capacity issues though are real; it's what William was talking about a minute ago about how do you find ways to get around that is a lot of concern. More generally, though, I think you see our clients are a self-selected group—they're hiring a firm that's focused on equity and doing this kind of work. And so they obviously want to catch kids up.

So, I don't know if there's anything approaching a representative sample…if you look nationally, I mean we're seeing essentially, I would say business as usual. You have teacher strikes in a number of places around the country; I was sort of surprised that hasn't actually gotten more attention. And then just not strikes, but resistance: they try to do voluntary extra days in LA and the teacher's union said “no” and that kind of stuff. A lot of the money hasn't been spent; a lot of it's actually being spent on infrastructure and so forth, which is important. I mean, schools weren't well-ventilated before the pandemic, and I think people appreciate how much more important that is now and they have aging physical plants and all that. So, it's not unimportant, but that's not necessarily [addressing] learning loss, and a lot of the money just hasn't been spent.

And that's just a fascinating thing that's been going on for a while with relatively little attention to it. I think people act like this $190 billion is just out the door; a lot of it is still sitting in accounts. And so around the country, I think we have to have a serious conversation about what's the urgency that people feel and are we simply seeing an extension of the same dysfunctional politics that led to bad choices during the pandemic just continuing in other ways now? I think that's a conversation the sector doesn't like to have, but that we need to find a way to have. And I think that points up a couple of things for this conversation.

One is, okay, this evidence and how do we figure out, how do you take things that work under a controlled set of circumstances and make them work more generally? And we should certainly talk about that a great deal. And then, second, why does this field wrestle just so much with evidence? And you see that across a range of issues—reading is getting a lot of attention now. Why is it so hard to get engagement with where there is—oftentimes the research is conflicting or there's various ethical or value considerations, whatever—but we have a bunch of issues [where] that's actually not the case; it's a pretty clear research base and yet we still struggle to do it and implement it. We should talk about why.

William: Yeah, that's good. So, coming off the heels of what you're talking about, it’s lack of capacity. If you want to bring research to bear within your district, you have to have some people who have some time to actually focus on it and learn about it. You have to have programs that are producing administrators and teachers who have had some learning in graduate school or as part of their training as they prepared for the profession to be thinking about what does research mean and how do you use it and how do I apply it? I also think that there are a few places where we do have clusters of evidence that's better. I think reading is a nice example, I think tutoring is a very good example in this moment. And so more people should be trying to dig in on that.

I do think there's often some questioning that happens about, well, how does that study or how does that body of literature apply to my context that comes up? There's a book called Common-Sense Evidence that Nora Gordon and Carrie Conaway wrote. It's really a pretty good book, and it's a very practical guide directed at education leaders about how to think about research. And it's about how do you think about other research that's out there that's not your own—as well as how do you think about answering questions that are important to you yourself? So, what's the role of a school district, both as a consumer of research but also as a generator of new knowledge? Because at the end of the day, what I hope work that we do at MDRC and some of our peer organizations and the research work that Bellwether does….my hope is that we're continuing to put better information, ideally the best information possible in front of people so that educators can make better decisions, the best decisions they can.

And I often try to make that distinct from perfect information to make perfect decisions because I don't think that's in the realm of possibility. I think if we hold the standards way too high, then people get paralyzed and they won't do anything. Encourage people to take what you know, be thoughtful about it, and then put something into play. And not in every case do we have a randomized controlled trial that tells us something. And in some cases, as Andy referenced earlier, we might have one and it was in a particular set of circumstances where implementation was done very well and everybody's really focused and on board with it. Now you're trying to think about how does this idea apply in a new context?

So, the idea is we'll give it a shot and pay attention to what you're doing and try to catalog the places where it just didn't go so well. But again, all these things, there's a piece out there that requires a little bit of extra effort and sometimes people don't have the capacity. I think that's where the field benefits in some ways from some of the more formal research-practice partnerships that have sprung up, particularly for a number of the larger urban districts over the past—I mean ever since the Chicago Consortium, so it's a lot of years—but I'd say in the past decade we've seen more of an explosion of that kind of practice. It's a way to add capacity to generate and have somebody else help you interpret evidence and to try to answer questions that feel important to the students that you're serving.

Andy: Yeah, I feel like you're onto something. I think it's underappreciated how much either in a school leadership position, if you're leading a district, or in a policymaking position, you're making the best decision you can based on imperfect information. I don't think that's fully…and that's always the case. Even if something, as you said, there's a good body of evidence like reading, you're implementing these things in dynamic situations and so the information is never perfect and often the evidence base on a lot of other issues is really incomplete. And you have to figure out what does the preponderance of evidence say? What's the best choice I can make here with the information in front of me?

That is underappreciated, but the risk I see with a lot of this is you see this pattern again and again in our sector, which is something that works well under controlled conditions and then being implemented rapidly and widely where those conditions are not present or without the key elements that actually made it effective—sort of the light-touch thing. Then it becomes this sort of weak version of it with little or no fidelity to the core elements of the initial intervention, which I think [creates] two bad effects. One, substantively it's not good for kids; you don't see results. And, second, good ideas can sometimes get discredited. And it's not that it actually wasn't a good idea; it's that it wasn't implemented well.

And you could see a risk for tutoring. I don't dispute what you're saying about the evidence base on high-quality tutoring, but you can certainly see a scenario where, in a few years, the popular narrative becomes tutoring really didn't put a dent in this problem. And it wasn't because of tutoring per se; it was because of the way it was implemented and delivered at scale.

William: So, there's two sides of this coin. You implement something the way it was supposed to be implemented, the way it was designed and how well they do with that? And there's a twist on that, which is: do you thoughtfully adapt to your circumstances, so you stay true to the principles of implementation but acknowledge you might not be able to do everything exactly the same way, but you're clear about the key concepts and the key needs. I think the other side of the coin is: how is what you're doing different from what's already available to your students? I think often what is overlooked is sometimes people say, "Oh, this is a proven program; I'm going to bring it in." But they don't really acknowledge that they may actually have some things going on that are already pretty good or they're competitive with that [new intervention], so they're really not going to see a difference from it.

I think one of the advantages that something like tutoring has had is: if you don't have individualized support for your students in place already, you're talking about some focused energy that's going to go into something that students may not be getting somewhere else, so you have a difference-maker. There are other cases where people try different kinds of interventions, where it's sort of duplicative of something else that they're already doing and then it looks like it's not really having an added effect. But they maybe shouldn't have done it in the first place because they were already doing something that had those features, so it's also how do you help people think through where are the gaps in the practices that I'm putting into place in my district or in my school to serve kids? Where are kind of the holes? And now let me investigate how I fill those holes versus grabbing at the bright shiny thing that everybody's talking about in the moment.

I think there's also that pre-thinking that's important to making choices about how you might change your practice or how you might bring in a particular intervention that's focused on: where am I filling a gap? Where am I doing something that's really truly going to represent a difference from what I currently do now?

Andy: I think the keyword there is choice and making choices because there's a bad habit in the sector: we add lots of new things and you don't sort of clear out the others and you see that across a range of issues. It's one of the things that exhausts teachers. I mean, an obvious example you see is with standards, we keep adding stuff to them because people think new things are important and we never take [any] out, so they become completely unmanageable. But you also see this with district initiatives. A lot of what our strategic advising practice is doing is helping people clarify, build a fact base, and actually clarify where those choices have to be made because you've got things that are either not working in concert or at odds or just too much to get done. But you have to really build a fact base and analyze it to figure out where that's going on. So that this issue of you have to make choices—this field loves to go with “and” when, in fact, you do need to actually make an “or” kind of choice.

William: What's interesting about that is this is a place of where research is super underutilized. This is a place where if you're uncertain about how to go. One of the challenges when we do lottery-based studies or randomized controlled trials is people are concerned about the comparison group: “oh, they're not going to get something and I don't want to deny something to students.” But there's also a lot of power and tests that can be randomized or other strong designs that are a little bit more: “I'm going to compare option A to option B and try to learn from that.” So, I've been doing B for the past 10 years, [and] A is really interesting. Instead of just adding A on top, let me do A in some of the places and do it systematically and let me see how kids do compared to the kids that are getting B. And then I can make a resource decision later about which one of these things I want to do if they perform similarly, but one was less resource intensive—maybe I'm going to take that one. If A performs better than B, maybe it's time to sunset B and have more people doing A. And I think there's really a lot of space for districts to be more creative in those kinds of tests within their settings to try to understand whether something really might offer some value that's not already there and still be providing supports that they think are going to be good for students across the board. I think there's a lot more room to do that kind of exploration.

Andy: And big, fast things. We do sort of real fast A/B kind of testing. I think [when] people sometimes think of evaluation, they think only of large studies over time, either using longitudinal data, various kinds of quasi or experimental methods. And that's really important over time for evaluating things, but, in the near term, if you're a school leader, really fast feedback on how is something performing, how is it working, what do you potentially need to change. Again, that you're making these decisions in real time. So, you want to make the information as least imperfect as you possibly can. Seems like a place that we should be….as I said, we do that and I know you guys, I think, do some of that, but it needs to be much more common. We wait often a long time to get these kinds of results—often to the point where the caravan's kind of moved on—and we need to be doing that much more in real time.

William: It depends on your question and it depends on your program—sort of the timing of that, too. If it's a whole-school reform effort that you're trying out, we know from the field this is going to take three to five years for that to truly take hold. And you have to give yourself the time to do it. But if you're trying to figure out: am I doing well enough engaging students in a particular program? I can assess that every week or every two weeks: I do cycles and see are they really participating [and] for how much? Are they engaged in this? And I can make adjustments on the fly through a quicker study that maybe gives me a stronger program faster.

I think we're going to have to make t-shirts that say something about least imperfect and share those around. But what I've been wrestling with is: are there any out of the box, big ideas that this is the time to do them or to try them in some places? And maybe I'm just a prisoner of what I already know and what I've experienced, so it's hard to come up with it. But just trying to think about: is there a radical thing that's worth doing in education that wouldn't be misaligned with what we know already but might give us more of a chance to have larger change at larger scale?

Andy: I'd love to hear your answers, too. I'm not going to let you guys off the hook. I mean, look, I think anyone who has a big idea like this and is keeping it to themselves, that's irresponsible. So, I don't have a whole bunch up my sleeve, but it seems like a couple of things. One is time. And really since the spring of 2020, people have been saying, "Hey, we have lost a lot of time; this has been disruptive. How do we make that time up?" Ideas that [once] seemed radical and got you called all sorts of names….but in the summer of 2020, why didn't we have kids in more places going to school in tents outdoors? By that point, early 2020, no one knew what was going on; it was a really scary time. By summer of 2020, we had some sense of how this virus spread, some basic mitigation strategies and so forth.

So why weren't we already being like, "Okay, we've lost this time; let's make it up"? And people forget, we talk about remote school and so forth. In the spring of 2020, millions of kids across this country got sent home with, at best, a packet, often with nothing. And when that was done, school was over and there was just not a lot of energy around. So, I think one of the places to go is time and what do we do now with time to help kids catch up? Sort of a really ambitious plan. And I do worry: we pushed an awful lot of money out from Washington and so is there an appetite? All these things cost money—is there an appetite to spend more? If we actually made just a catastrophic mistake of sending a bunch of money out in kind of ill-defined ways, is there money to really think about how do you play around with time?

And then, second, these interventions like you've been talking about, but maybe how do we think about empowering parents? And we have these ideas….so, for example you say, "Why don't we send money to parents?" And people freak out. That's, of course, one of these weird issues in education where sending money directly to parents is….like education's the one place where Democrats don't want to do that and it's the one place where Republicans do want to do it. So, the politics are strange, but what could we do to give parents an opportunity to avail themselves of different things for their kids and try to drive it that way from the demand? And then hopefully that they'll be supply, and you're going to get unevenness and so forth.

And there are examples of this. There are international examples: like every kid in Iceland gets, when you're a teenager, you get a small amount of money every year to spend on stuff like this. So, these aren't wild [ideas], but what could we do here with that? Those are the two things I think about because I don't think the school system itself, as we talked about, we weren't good at doing this before the pandemic. And to the extent people tend to catch up from serious education deficits in this country, it's individualized.

This is actually something I like about the system: it's a system built on second chances. People who didn't have a good experience in high school can still find colleges that are open enrollment or community colleges; they can continue. And we've got this system of second chances that helps people figure out different ways; you can go into all kinds of various postsecondary options and certificates. There's lots of different ways for people who want to move, but it's all individual, it's not systemic, it's the person deciding this is what I want to do and here are the systems that I can try to avail myself of. So I am skeptical of the system's ability to do this at scale.

What about you guys? What are the secret interventions that are in the files of MDRC that you guys haven't shared?

William: I wish we had more, especially at the K-12 level. Before I dig into that, you mentioned the time; it's really interesting. The other thing that I always think about is we've set up a K-12 education system and in more and more places you're getting pre-K thrown in. And you're basically talking about—call it a 14-year plan to get through secondary school and be ready for some mix of college and career opportunities. And we're very beholden to this 14-year concept. One of the things I try to wrestle with and I don't know what it would look like….kids may accelerate and decelerate at different times and different content areas and are there better ways for us to adapt for that? Should we have different expectations around, should everybody be somewhere between 17- and 19-years-old when they graduate high school or should we have more latitude there? And is there a way to have more latitude that doesn't exacerbate existing inequality?

So yeah, we lost a lot of time, but should some parents have the choice or the option—the way you're talking about it—to think, "Well, maybe my kid's going to need another year of education and they're going to be a year older by the time they finish high school because I need them to cover the content. For them to do that is just going to take extra time that I don't want to jack into their summer or I don't want to have them not doing sports or music or arts after school because I'm going to have them doing some kind of academic enrichment or academic support program." And, so, is there a way to loosen up how we think both about time and the immediate—and how do you spend the days per week on education with kids and what are the things they're doing?

And maybe we can be more efficient with that, but also thinking about that longer scale and are there ways for us to think about flexibility or at least offer some options for flexibility that a subset of families might want to take advantage of or might not be horrified by. Intervention-wise, the best we got is probably stacking things up. There's certainly stuff that we know about. Well done, individualized support, tutoring, and mentoring can be a very powerful thing. I do think there's something about how do you make sure there are strong relationships in place because I think the relationships between adults and students help keep students engaged and encouraged and kind of plugging away. And if I've fallen away, I think one of the biggest challenges about the pandemic is the disengagement of students from school.

There are students who are still in the system where they're plugging away and they might be two or three years behind where we'd like them to be, but then there are a bunch of students who have disconnected. How do we think about interventions that may or may not officially connect them to school but connect them to stuff so that they're actually building knowledge and skills so that they're not lost. And that they don't hit 24- or 25-years-old kind of at sea without the ability to connect to jobs that feel meaningful or interesting without some sort of ability to have a vision and a pathway that's going to support their own future, if they have families that'll support their families, etc. In some ways, that's a group I'm particularly concerned about—the ones who disconnected and how to bring them back.

Andy: I think that's a big problem with the pandemic. The one thing I'd sort of both push on and see a through line there is, you talked earlier, you don't want the academics to come at the expense of extracurriculars, like music or sports. I actually think these are all part-and-parcel. Good enrichment activities like arts, music, sports—like the skills and the things you learn in those—they help with the academics. I feel like it's all much more bundled up. We talk about these as if it's like these two separate lanes. This is all….it's helping kids discover their passions and also helping them learn how to do things, how to be good at things, how to be bad at things. And that can be okay, too.

Particularly on sports, there's a lot more we could do to make it more inclusive. But these things, I think they're part-and-parcel in terms of supporting and re-engaging kids, so some of these various things that we tend to think of as extracurricular, out-of-school experiences might actually be ways to help start to reengage. And, again, teach some of those different kinds of skills as well.

Leigh: Andy, the points that you just made about really thinking about the kinds of activities and opportunities that get kids engaged….it could be all part of the individualization argument, if it's really about what are the things that are going to spark students and the things that are actually going to motivate them and help them be engaged. Because that's all part of how they're spending their time and how schools are thinking about spending their resources and doing what they need to do for each student as they are on their journey. But, ideally, their journey stays connected to the system and gets them some kind of credential at the end—or gets them connected to something that is going to later allow them to build on that for more success in life.

In some ways, I feel like people put that set of music and extracurriculars and sports as like, "Well, that's one thing, but the thing we really need to address is academics,” when you could easily make the argument, I think, that they're connected in really important ways to help students really be engaged in school.

Andy: So, I've tried not to, Leigh, on this podcast, be like every time you and William say something, "Hey, and here's another interesting thing we're doing at Bellwether." But I can't lay off of this one because we do have this initiative called Beta, which is a big new ideas initiative. And our first big Beta project, we're calling it Assembly. It's basically like: it's not as if all these supports we're talking about don't exist or people aren't availing themselves, they are. It's just if you have the means to….it's disproportionately the affluent who's doing it and these things work. And so why don't we figure out a way, as we talked earlier, to small D democratize them so more people can take advantage of them.

And there are policy things there, whether that's education savings accounts or different kinds of grants or whatever it is to help parents. There's a whole bunch of pieces, but the basic idea is you've got all these different things and you can actually assemble an education that is much more customized with some of these different pieces. We don't have it all figured out; we're in the middle of it right now and obviously the big challenge is how do you do this in a way that's equitable for all kids? What we're trying to figure out: how would you actually build that? What could it look like and how would you assemble a more customized education?

It's an exciting space and does get to some of the themes here that William was talking earlier about: how do you make some of this stuff stackable? And we were talking about how do you make it customized and then your point on how do you bundle this stuff together? So, stay tuned on that: it's complicated from a practice and policy standpoint, but potentially hugely impactful for young people.

Leigh: I think that this topic is clearly going to continue to be an important topic for many years. I know that both MDRC and Bellwether will have more to say about it over time as we learn with our partners and from what's going on in schools. So, maybe we should say something like, "This is a conversation that is to be continued." But for today I will say thank you so much to both of you for joining me for this discussion. It was really great to have you.

To learn more, visit MDRC.org. Did you enjoy this episode? Subscribe to the Evidence First podcast for more.