Leigh Parise: Policymakers talk about solutions, but which ones really work? Welcome to Evidence First, a podcast from MDRC that explores the best evidence available on what works to improve the lives of people with low incomes. I’m your host, Leigh Parise.

In 2014, three Ohio colleges set out to adapt a student support model—pioneered by the City University of New York—called Accelerated Study and Associate Programs (or ASAP). The program offers high-touch, proactive advising with full-time enrollment, financial assistance, and other supports. MDRC’s evaluation of the ASAP Ohio program found that it doubled graduation rates for community college students after three years. MDRC, with support from Arnold Ventures, recently released exciting findings from a long-term follow-up study of the ASAP Ohio program, showing that the program not only increased attainment of associate’s and bachelor’s degrees after six years, but—notably—also increased the earnings of students.

Today we’re talking with Christine Brongniart, the university executive director of CUNY ASAP, and Colleen Sommo, a senior research fellow in MDRC’s postsecondary education policy area, to learn more about the CUNY ASAP model, its replication across the country, and the latest findings from MDRC’s study of the program in Ohio.

Christine, Colleen, thank you so much for joining me today. It’s great to have you here.

Leigh Parise: Christine, let’s start with you. You spend a lot of time thinking about the ASAP model. Tell us a little bit about the model. What are its goals and what’s its approach to accomplishing them? And then after that, maybe a little bit about how the model was adapted for the Ohio context.

Christine Brongniart: CUNY ASAP, wow, it has a history now. We are currently recruiting our seventeenth cohort of students. We were founded in 2007 with the support of New York City’s Center for Economic Opportunity—now known as NYC Opportunity—under then-Mayor Bloomberg. There was a bold goal. We knew, as a system, that we had to do better in terms of associate’s degree completion. At the time ASAP was conceptualized, we had a three-year associate’s degree completion rate—across our community college system—of 13 percent, which lagged the national urban community college average graduation rate of 16 percent. We had to do better as a university system. We had a moral imperative from the City of New York to do better, given the very connected ecosystem CUNY is a part of, in terms of workforce development, in terms of the ways in which our students are citizens of our community who, of course, need access to…but also completion support through our system.

So that’s really where CUNY’s ASAP was conceptualized. Urgency to do better, bold completion vision, taking that 13 percent completion rate and not only imagining doubling [it], but setting the bar even higher at a 50 percent three-year graduation rate. We had to prove very quickly that that was achievable, so that really set the motion forward in terms of the way we had to ground our work in rigorous benchmarking, rigorous real-time data management and oversight—and very much in real time, looking at that as a system, in the ways in which we could collectively, in aggregate, report back to our city partners that this was a worthy investment.

So no matter how we pieced the components together, the only way forward for us was a comprehensive approach. We knew that in isolation, in pockets, there are certain practices that are effective to establish and maintain academic momentum and create a connected community [and] deliver relevant on-time support. So CUNY ASAP is really constructed on that weaving together—that holisticness—of ensuring that students are receiving robust financial, academic, and personal supports in a way that is on time (meaning proactively delivered). And that’s only made possible through the way in which we approach academic advisement: ensuring that caseloads are modest, that that relationship and core foundational connection is established early, and again, all the ways in which we know that advisement works  is only through the use then of real-time student-level data. That’s not just academic data, that’s personal data: What’s going on in students’ lives, what’s going on in terms of the push and pull of personal demands [and] other kinds of demands for their time that would otherwise derail their academic progress.

So that is ASAP’s approach. It’s all about those holistic supports. It’s weaving it all together as one comprehensive package. And I would say that maintaining full-time enrollment is the key. Maintenance of academic momentum through full-time enrollment is absolutely critical. We provide academic momentum support, not only through the provision of the academic advisement [and] the robust financial supports, but [by] ensuring the students really use the entire academic year, winter and summer course-taking [and] immediate continuous enrollment through corequisite courses, as needed, in their first year. Again, it’s all about maintaining that momentum.

I’d say another feature that’s really important to highlight with our CUNY ASAP model is our consortium structure. I think that that administrative structure is not often highlighted in terms of the “secret sauce.” We’re often asked, “What are the most effective components?” The response to that is not the components in isolation but, again, the innovation being the holisticness of the delivery of the components. But I’d say the consortium structure here at CUNY ASAP is another key to our success. We work very, very collaboratively with our nine implementing ASAP partner colleges that run this program and support students, day in and day out. Our central office—I’m situated in the Office of Academic Affairs—provides really meaningful administrative relief to our colleges.

So we’re maintaining a certain rigor of fidelity, cross system, in terms of the ways in which we impart and maintain standard policies and practices. But it’s about the efficiency too, really making sure that those public tax dollars are utilized in the most effective and efficient way to ensure we maintain healthy enrollment, [and] to ensure that we’re able to effectively manage all of the robust student financial supports. And, again, it allows us to really ground a culture of continuous improvement. We really co-own the program across the system with our college partners.

Leigh Parise: I want to pause for one second to reflect on a couple of the things that you shared—which, first, I really appreciate, and I imagine listeners will as well—the whole history and the way that, as a system, you approached this. And there’re a few things that I just need to call out that as you were talking, I was like, “Yes, yes, yes. That.”

First, it seems like the system really took responsibility. You used the language, “We had to do better as a system.” I think that is something to acknowledge, that that was something that CUNY said. “We have to figure out how we’re going to serve our students better and what it is they need, what’s going to support them, and how are we going to make sure that they feel supported?” You talked about that— connecting what’s happening in their personal lives—and I want to call that out. And then, also, it’s amazing that you said you’re recruiting the seventeenth cohort of students. Which, if I’m doing my math right, means that some of your first cohort of students are 35 years old, probably, now—or older. [It’s] just really exciting and impressive that you’re having such an impact on students and, likely (I would think), on the city of New York.

A couple of other things to note—maybe just wearing my MDRC hat—that got me especially excited. You talked a lot about how having this centralized consortium structure really helps with maintaining fidelity. I think a lot of our research work and support that we do with our partners focuses on program implementation, and fidelity is often so key to actually seeing whether you’re going to make it through to having an impact on the people that you’re serving. So that’s exciting. And then you also mentioned responsible use of public dollars, rigorous benchmarking and data management. All of that, I think, is probably one of the reasons that you’ve had so much success over so many years. Thank you for sharing all that background.

Christine Brongniart: Oh, sure. Thank you. As you said, I think the all-in-ness, in terms of the system commitment, it really did galvanize buy-in, meaning it just became a nonnegotiable. All community colleges at that time were in, at a modest level. Our first cohort was just over 1,100 students. While not at scale—and maybe while the vision, even then, was never scale, necessarily—it was with the interest of turning what completion culture looks like on its head. I think that that was really what the underlying premise was: We put all resources in, both fiscal and [the] brain trust of university leadership and leadership at the colleges themselves. “What can we do and how high can we set the bar?” So the all-in-ness, I think, was a key to success—meaning if this started in pockets as a pilot at one particular community college, I don’t know that we would be where we are today with regard to scale.

I’d say, also, this consortium piece too, it’s just so fascinating to see how... Again, this is really the co-ownership piece. We really don’t see that modeled across CUNY like we do here. There really is a flatness between the central office and our partner colleges—where we do, at the central office, make a lot of key strategic decisions, but we’re really more advocates for continuation of funding. More recently, in the spring, we testified before the higher education oversight committee at the city council. We are able to bring voice to our work and represent our work as a consortium, and, again, continue to advocate to support our current scale and prove our value to the city and be active advocates in that space. It’s a unique approach that I think is really effective and has potential for inspiration across other systems as well.

Leigh Parise: Yeah, I think so too. Well, speaking of which: Nice segue. I know when the initial findings came out and CUNY and MDRC started to share them, a lot of people said, “Wow, this is really impressive. But that’s New York City. That can’t work in other places. The context is so different.” So say a little bit about how the model was adapted for the Ohio context, which is really what we’re here to talk about, today, in more detail.

Christine Brongniart: Absolutely. Within CUNY, we were really curious about implementation context. We fully acknowledge and embrace this idea that CUNY is a very specific implementation setting, not only because of its particular urban characteristics. And, yes, the majority of our students are more traditional in terms of age and their progression from the New York City public school system to our community colleges. We really did want to build the evidence base and see if this model and its structure still have the level of impact that MDRC’s randomized controlled trial of CUNY ASAP demonstrated.

That’s really the motivation in our replication work. Working in partnership with three Ohio community colleges, knowing that the demographic context is different, knowing that the structural and administrative features of the way that the programs were going to be managed—[that] was an interesting feature that was adaptive. We entered the work in a spirit of wanting to understand the adaptability potential of the model. So that’s where the curiosity was started. And, again, our commitment to building our evidence base was a primary motivation as well.

Adaptations were modest, from our vantage [point], in our national replication work, where—more and more—replication fidelity is not defined as rigidly as [when] we started. Meaning, when we began this work, we had a very specific sense of how the model should be implemented, and we were sticklers for staying as close to fidelity as possible. I think we’ve learned over time that due to resource constraints and other structural features... Every community college system is unique and therefore adaptation is inevitable.

We’ve been defining it, more and more, as proximity to fidelity. There are certain core tenants we hold to be absolute, necessary for consideration, in terms of being a replication of the model. But then there [are] other features that are adaptable—and, therefore, if it’s in proximity to fidelity, then that works. That has to work. So in the case of Ohio, making very modest modifications, say, in the monthly incentive—that’s the most commonly known adaptation. In CUNY ASAP we provide all of our students with an unlimited semester-based MetroCard. It’s a really high-value incentive. That really drives up our cost per student. But for a grounded rationale—meaning access for our students to be able to travel across the city, across all five boroughs, to get from home, to work, to school, to wherever they need to go—that is a huge relief of out-of-pocket costs, so that’s a really valuable incentive.

But in the Ohio context, of course, metro system wasn’t an option. So again, just for the sake of preserving the intentionality of that incentive, which is [that] it’s not only to alleviate out-of-pocket costs associated with attending school, but it’s also an incentive to show up every month, meet with your academic advisor—it serves that purpose as well. As long as that core feature was preserved, as far as our replication definitions were concerned, it was still in fidelity with the model.

Leigh Parise: Great. Thank you so much. All right, Colleen, the ASAP model has a lot of evidence behind it, showing that it dramatically increases student success. Tell us about the latest findings from MDRC’s study of the Ohio programs and why you think they’re important.

Colleen Sommo: Sure. So as you noted, MDRC conducted an evaluation of the ASAP Ohio programs. Through a randomized controlled trial—or essentially using a lottery—we were able to compare students who are offered the opportunity to participate with students who had the more standard college experience. A few years ago, we reported that the Ohio programs were nearly doubling the three-year graduation rate. Given how successful this program has been at improving degree completion, we were very excited to have the opportunity to continue to track these students for longer, looking at longer-term degree completion, as well as a first look at earnings.

Our latest report shows that after six years, the program continued to have large impacts on graduation. Forty-four percent of the program group graduated, compared with 29 percent of the control group. During this longer follow-up period, we’re also finding that more students in the program group have gone on to earn a bachelor’s degree as well. While these findings on increased degree completion are newsworthy on their own, we’re also excited because this report includes our first look at ASAP’s effects on labor market outcomes. And what we’re seeing is that six years after entering the program, students in the program group are earning about $2,000 more per year, on average, than students in the control group. This is an increase of about 11 percent, and because both groups are working at similar rates, the increase is likely coming from higher paying or steadier work, which is really great news.

Leigh Parise: Great, thank you. For people who might not know that we’re not always, in studies, able to actually track people into the labor market to look at their earnings, can you say a little bit about why this was such a unique opportunity and how we were able to do that?

Colleen Sommo: I think that this is a unique opportunity because, first of all, having these really large effects on degree completion is an important first step in order to expect to see longer-term effects on things like earnings. You want to see a large effect on a more proximal outcome to be able to set yourself up, usually, to see this longer-term outcome. So that, I think, is one of the most important things.

The other thing is that in order to access these data, it really required a partnership between MDRC and CUNY, as well as the Ohio Department of Higher Education and the Ohio Department of Jobs and Family Services. So, I think, building on Christine’s earlier point about this importance of having a consortium and having different people involved along from the beginning, [this partnership] really set us up to be able to have access to these data and be able to provide this information.

Leigh Parise: Great. I think it’s really exciting, and I imagine that it will catch the attention of even more policymakers and practitioners than the already really impressive findings did, because I think that’s a north star: Can we really change some of the economic outcomes for students participating in various programs? So thanks for saying a little bit more about that.

All right, Colleen, one other follow-up question. I know, in the report, some people might take a look at it and think, “Oh wait, students who participated in ASAP are still only making just under $20,000.” And they might think, “That’s kind of a low number.” I’m curious about your reaction to that and whether you can unpack that number a little bit for us.

Colleen Sommo: Yes. Thank you for raising that. There are some important things to keep in mind when interpreting this number. First, the average earnings include every student in the study, regardless of their employment status. So there are many students in that average who are not working at all, [and] many who are working part-time or working inconsistently. This means that we shouldn’t interpret that average earnings number as a full-time annual salary. It’s a conglomerate of all sorts of people doing different things at that point in time.

Also, the averages only reflect earnings in the state of Ohio. So while we feel confident that this captures the majority of students, based on larger statewide migration trends, students employed outside of Ohio will have underestimated earnings in our data. And finally, I just want to say that about a quarter of the students were still enrolled in school during Year 6, so we might expect to see averages increase as more students graduate and begin to enter the labor market.

Leigh Parise: Great. Thank you so much. That was really helpful. All right, Colleen, tell us a little bit more about what’s next for the study. What are some of the questions that we’re still hoping to answer?

Colleen Sommo: Sure. We’re going to continue to collect information on degree completion through 8 years of follow-up, and we’ll be able to extend the wage analysis through 10 years of follow-up. This will allow us to see whether the impacts continue to grow over time. We’re also hoping to do some exploratory analyses to look at work patterns, including things like industry and consistency of employment. Our next set of findings will be released in 2025, so stay tuned for that.

Leigh Parise: I love it—previewing what’s to come. Thank you.

All right, Christine, let’s go back to you for a minute. This is, I think, one more piece of evidence showing the effectiveness of the ASAP model. It would be good for people to get to hear a little bit from you about what’s next for the field. What are the things that you think higher education leaders, policymakers, and other people should be doing with these findings?

Christine Brongniart: These findings, in terms of increased employment and earnings in the Ohio demonstration context, are fantastic. I think it just provides even more of the return-of-investment narrative that sometimes gets lost as individual institutions or state systems consider replicating a model such as ASAP. Which, of course, comes with a commitment of upfront costs: You have to staff these programs, [and provide] the resources in terms of the financial benefits to students. These are real dollars that need to be committed. While the staffing piece adds meaningful capacity at the institutional level, we’re dealing [with] an environment where enrollment is challenging, revenue streams are impacted, and there’s scarcity in terms of available resources.

I think we’re seeing a tide shift, meaning I think more and more large pots of public dollars are being meaningfully directed to systems, and the policymaking voice in this has been very active in [its] advocacy.

These findings certainly propel the conversation forward, in terms of this model being a sound investment. I think that that’s really critical, that you’re seeing other state systems—like the State University of New York—allocate a huge, meaningful size and proportion of their recent Fiscal Year 2024 budget to create a transformation fund. “Transformation” doesn’t necessarily mean you start from scratch. While there always should be room for innovation, I think allocating resources to grounded, evidence-based models like ASAP that are demonstrating long-term returns—intermediate returns, we’re looking at six years out [and] already seeing gains in employment and earnings; that’s really exciting—I think that that’s really where I see the field headed, in terms of this scaled system approach of replicating evidence-based, evidence-proven models like ASAP. We recently stood up a new partnership with the State Higher Education Executive Officers Association (SHEEO), which supports the launch of a completion coalition learning community that’s going to bring together five higher education leaders across state systems to do a deep exploration, not just on the implementation of ASAP, but really grounding the long-term, sustained case for investing public dollars in a scaled effort.

We’re continuing to push forward at CUNY, to realize our vision. We state our vision, as an entity of CUNY, to impact national public higher education policy and investment that focuses on increasing student success, completion, and equity. We know that this model not only has dramatic impact for all students, but we look at that continuously for subgroups of students. We know that that 50 percent graduation rate—and getting closer to that rate across all subgroups—is holding, even through the last few years of unique challenges.

That’s where the momentum is leading. It feels like there’s right energy, even at the federal level, with the new round of the Postsecondary Student Success Program. I think there is even more grounding and opportunity to make these kinds of investments that really do pay off, both to the individual students and to the states as well, in terms of economic health.

Leigh Parise: Thank you so much, Christine. I will say, it certainly feels to me like you are well on your way to having that national impact that you talked about, which is really exciting. And I know that Colleen and [those of us] at MDRC are excited to get to be a partner in that.

All right. Thank you both so much for joining me. This was really great.

Christine Brongniart: Thank you. Thank you so much for having me.

Colleen Sommo: Yes, same here. Thanks for having me.

Leigh Parise: Thanks so much to Christine and Colleen for joining me.

If you want to learn more about CUNY ASAP, you can check out our earlier podcast with Christine from a couple of years ago. A related podcast you can check out is an earlier episode with Amanda Janice Roberson of the Institute for Higher Education Policy and Jinann Bitar of the Education Trust, [which is] specifically focused on equity in college completion. To learn more about this work, you can also visit mdrc.org. Did you enjoy this episode? Subscribe to the Evidence First podcast for more.