Policymakers talk about solutions, but which ones really work? Welcome to Evidence First, a podcast from MDRC that explores the best evidence available on how to improve the lives of low-income people. I am your host, Katie Beal.

Today we’ll be talking about the Accelerated Study in Associate Programs, or ASAP. It’s a program developed by the City University of New York (or CUNY) that provides comprehensive support services to community college students. MDRC’s evaluation of ASAP at CUNY community colleges found that it nearly doubled graduation rates — some of the largest impacts found among programs for community college students.

To see if the program could work beyond New York, CUNY, MDRC, and the Ohio Department of Higher Education worked with three Ohio community colleges to implement the ASAP model. Recent findings from MDRC’s evaluation show that the Ohio programs had similarly large impacts on student outcomes, illustrating that the program can be successfully replicated and serve as a model for community colleges across the country.

To learn more about the Ohio results and what it takes to replicate and scale the ASAP model, I spoke with Camielle Headlam, a research associate at MDRC, and Christine Brongniart, the university executive director of CUNY ASAP.

Katie: Thanks for being here today. Christine, why don’t you start by giving us a bit of background on the CUNY ASAP model?

Christine: CUNY ASAP — Accelerated Study in Associate Programs — has an interesting origin story. Essentially, it was an experiment in 2007. It was a call to action by the city of New York. The Center for Economic Opportunity wanted to address issues around social mobility and poverty alleviation, and it was a call out to all agencies across the city and CUNY’s response was ASAP. Essentially, it was a response that scanned the field of effective, evidence-based practices that really did make a difference in terms of ensuring that students were enrolling, gaining academic momentum, staying enrolled, and facilitating full-time enrollment. All of the data that showed that offering comprehensive support services, all tightly knitted under one umbrella, dedicating resources to ensuring that students are accessing a full range of supports — that’s the grounding principle of ASAP.

Financial support is pretty critical to the model. The model really allows for students to enroll full time without having a burden of out-of-pocket costs: Providing tuition gap assistance. Ensuring any unmet need between aid received and tuition is alleviated. MetroCards — ensuring that students have the resources they need to travel to and from school, to and from work, to and from home. Textbook support — students really do bear a burden sometimes of having to make choices between buying their textbooks and, say, paying a utility bill. We really wanted to make sure that those financial supports were in place and that wouldn’t be a barrier for full-time study.

Comprehensive academic supports — structured supports that ensure students can stay enrolled full time and get the courses they need when they need them — it’s pretty critical to the model. This is really fulfilled through structured pathways, ensuring that students are able to consolidate their schedules in blocks when they need them. Morning blocks, afternoon blocks, evening weekend blocks, early registration options, so that students are able to essentially get the classes again as they open and when they need them, and they aren’t shut out to those courses that they need. Early enrollment in developmental education is something that’s pretty critical to the model: to address any developmental need immediately and continuously. When we fulfill that through blocked course offerings in that first year — so ASAP students can take those blocked courses together — it promotes the sense of community within the ASAP cohort, but again, it allows students to prioritize fulfilling their community developmental education requirements.

I’d say the next piece of our model around personalized supports is the primary feature of our model: intrusive advising, responsive proactive advising. That was something that we knew through the literature would be an effective practice. And that’s where we’ve spent a lot of our resources in terms of really refining our academic advisement model. Students have personalized advisement from the moment they enter to the moment that they graduate. Advisers have a modest caseload ratio of 100 to 150 students. Essentially they’re able to meet very intensively and consistently with their students and really have kind of a level of rapport that allows them to provide support that is very proactive and ensures that students, if they’re experiencing challenges, experiencing any, need additional academic supports, if they have other life experiences, we can give them appropriate referrals to other campus-based or citywide resources. That advisers really tune in to every student in their caseloads’ particular needs. We also have very intentional academic supports in the way of program-dedicated tutoring. We have career specialists who are dedicated to serving our ASAP students and ensuring that they’re developing career-readiness skills.

All of these supports are wrapped under one umbrella. Students are having an experience where they’re able to go to one office, meet with one trusted adviser, and understand all of the net that’s there to support them consistently, through their academic journey.

Camielle: I would agree with Christine. What we saw in the research is that all of the components of CUNY ASAP advising — financial supports, tutoring, career counseling — these are components that were being offered by different colleges and universities. But students were really using these supports in isolation. They might see an adviser when they need help, maybe they go to the financial aid office. But what we saw is that CUNY ASAP pulled the supports together in a cohesive manner and set guidelines for how students should use these different supports. That seemed to make the difference for students.

Katie: MDRC conducted a random assignment evaluation of the ASAP program at three community colleges and found that the program nearly doubled graduation rates. Tell me about that partnership between MDRC and CUNY.

Christine: I think that the partnership between CUNY ASAP and MDRC really did solidify what we have already been seeing in terms of the evidence of the model’s impact through completion rates. It allowed CUNY ASAP to rigorously test that model and be able to make a case to the city to continue to fund and sustain and potentially grow the program. That study really did have a profound impact on the trajectory of CUNY ASAP’s expansion. Essentially the findings were released, which confirmed this doubling effect on three-year rates — I believe that was released in 2014 — and right on the heels of that release we received a commitment from Mayor de Blasio, who had just entered into the administration, that he wanted to make a bigger investment in CUNY ASAP.

Katie: And then how did the ASAP Ohio Demonstration come about?

Camielle: When we found that CUNY ASAP almost doubled graduation rates, it was really an exciting finding. It was actually an unprecedented finding. It was the largest impact that we’ve seen on graduation rates in over a decade of research. Naturally we started asking questions. We wanted to know whether the program would be effective outside of New York. Would it work in another state? Would it work for other student populations? With these questions in mind, we partnered with CUNY to share information about ASAP, to tell people about the evaluation findings, but also to explore whether other states and other colleges would be interested in adopting the model and testing it to see if it would work for their students, if it would work in their localities. Fortunately, there was some interest. In Ohio, we found three community colleges: Lorain County Community College, Cuyahoga County Community College, and Cincinnati State Technical and Community College. These three colleges agreed to work with us and to be a part of the ASAP Ohio demonstration. CUNY started working with the colleges and provided technical assistance to teach them about the model, to help them learn how to implement the model, and the colleges began implementing their own programs based on that CUNY ASAP model in 2015. And I must say, a lot of credit goes to these colleges. It was a lot of work for them to create such a comprehensive program so quickly. And after piloting the program for just one semester, MDRC started rigorously evaluating it by randomly assigning students to either be in that program and receive all of those services or to be in the general population, the control group, and just receive regular services at the college.

Katie: In January, MDRC released three-year results from that evaluation. What did the study find?

Camielle: We found that just like ASAP in New York, the Ohio programs nearly doubled the percentage of students who graduated after three years. This was a really impressive finding because many successful programs, when they’re trying a new location, they actually don’t achieve the same impact. The question we get a lot is, “What made this work? How did the Ohio programs increase graduation rates?” And what we see from the evaluation is that the programs led more students to enroll full time, to persist from semester to semester, and to earn more credits, which eventually led to their degrees.

We also see that the Ohio programs led students to have a very different college experience than those students in the control group that were getting general services. When we surveyed students about a year after entering the study, more Ohio program students reported that they used services such advising, tutoring, and career counseling, and what was interesting is they didn’t just use those services, they also used them more frequently. For example, when it comes to advising, students in the program had three times as many advising meetings as those in the control group. It’s becoming clear that this type of support is really essential to getting students to graduate.

One thing I should say is that services are expensive. The Ohio programs did cost more, because they are providing a lot of support to students, including financial support. But because so many more students in the program graduated, the program actually lowered the cost per degree for the colleges.

Katie: That’s really great! Now tell me how the population in the CUNY ASAP study differs from the population in the Ohio study.

Camielle: The population in Ohio was a bit different from CUNY. When we studied the CUNY ASAP program in New York, many of the students were a traditional-age college students. And for the evaluation, all of the students had to be in developmental education courses. That was interesting because when we went to Ohio, we saw that many more students were nontraditional — this means the students were adult learners. They often were working full time or they had children that they were caring for. We also included students who were in developmental education, but those who were college-ready as well. The fact that the program still achieved similar results suggests that the CUNY ASAP model is really effective for a range of students.

Katie: I know that the ASAP program is being replicated in colleges across the country. Christine, can you talk a little bit about that?

Christine: CUNY ASAP does continue to support other institutions who are interested in replicating the model, and we have found partners in this work that are committed to delivering the full comprehensive support services as CUNY ASAP does. We are currently replicating beyond Ohio in California. Through the support of Arnold Ventures, we were able to provide technical assistance to Skyline College, which is in the San Mateo County Community College District. After they launched their program, they used the momentum that they built through this work to engage all of their colleges within the district system. Another replication effort that’s currently underway is Nashville GRAD. It was a commitment through the mayor’s office in Nashville to implement the model at Nashville State Community College. They launched their program this past fall. CUNY ASAP is also supporting two colleges in the West Virginia Community and Technical College System, and they’ll be launching their replication programs in Fall 2020. We also supported the replication of the model at Westchester County Community College. This is particularly exciting because it continues our partnership work also with MDRC. MDRC will be studying the Westchester County replication, which is known as Viking Roads.

Katie: CUNY has rapidly expanded the program over the past several years. You’ve gone from serving about 4,000 students in 2014 to now serving 25,000 students per year. That’s an enormous level of growth. What are some of the lessons you’ve learned as you’ve scaled the program?

Christine: The lessons have been ongoing. We really did have a very specific focus on maintaining our own program fidelity and delivering quality services and trying to not dilute those. When you’re starting small and you’re running a program that has a key feature of having a cohort community philosophy, as you grow and grow, maintaining that community becomes more challenging. Our colleges have been really creative in trying to find ways to bring together the ASAP community through different ways of segmenting their students according to academic fields of study, according to other interests. Students who need some additional academic support, so serving probation students more robustly.

We had to grow very quickly in terms of our advisement capacity and ensuring that the one-adviser-to-150-student ratio was maintained. But we were welcoming so many new advisers that we had to be much more intentional with our onboarding in training, in ongoing professional development offerings, and that is a key function of our office. I think that that was, a lesson learned through scale is how to effectively onboard staff and ensure that as they’re entering, they’re all delivering the same philosophy and grounding principles of CUNY ASAP’s advisement approach, which is very much built around growth mindset and ensuring that it’s not a deficit-focused approach. It’s very much ensuring that students are building the confidence they need to be successful. I think that was a learning for us, is how do we best support a new group of staff coming on in such a rapid succession?

One of the lessons learned, also, was that we really needed to continue to refine the ways in which we were able to support data collection and maintaining that core grounding philosophy that all the work that our program staff do is rooted in the way in which they use data. Ensure that they’re tracking their contacts with students in ways that is meaningful to their work and meaningful to the kind of broader policy of the program and practice. I think that that’s a lesson to us. And within ASAP Central Office is that we really do have to dedicate resources in that area. It was something that I think that we were able to keep pace with as we scaled, and we continue to refine our data-management approach and system. That’s a pretty strong commitment and priority.

Katie: Can you tell me what you’ve heard from students about the impact this this program has had on their lives?

Christine: Yeah, certainly. We have just recruited our thirteenth cohort, so we now have quite a few alumni: We’ve served nearly 58,000 students and our students are now joining our staffing. We have a number of students who are now ASAP academic advisers. We have students who are committed to higher education because they had such a powerful experience on their own journey that they’re looking to CUNY as — they’re returning back to CUNY in their own professional lives. We have alumni who are just telling their stories, and reflecting back on how, as they were pursuing their associate degree, they were encountering issues of homelessness, of food insecurity, all sorts of challenging life circumstances, where they are able to reflect now to say, “Wow. You know, MetroCard support really did save my life for that time. It really did.” Or, “My adviser — I wouldn’t have been able to do it without that support.” That is consistently the message: that the power of that advisement support kept students motivated and driven and focused. Our alumni are always articulating that.

I think the power of that relationship is giving all of our college partners pause to say, “Wow, we really need to look at our general advisement and see how we can build in practices and ensure that our general advisers do have the bandwidth to more intrusively, intentionally serve our students.” That the demonstration of that isn’t that they just reregistered, reenrolled. It’s that students are — just as was demonstrated in Ohio — tapping into the resources of the college community more consistently and robustly. It’s one of those features that I think was an interesting piece of the Ohio demonstration report, that I think that we should be studying here within CUNY as well.

Katie: And what about in Ohio, Camielle? How are the three demonstration colleges thinking about sustaining the program?

Camielle: The sustainability story in Ohio is interesting. One of the colleges, Lorain, they decided that they were really going to commit to this program, and they were going to sustain it and slowly scale it up. They didn’t have the same resources that were available in New York to scale very rapidly, but they are slowly adding new cohorts to their program each year to continue it. Some of how they’re funding this is they renegotiated different utilities contracts to find available funding. They also repurposed funds that aligned with the mission of the program. For example, if they had a scholarship that was intended for low-income students, they might use that to help put students into this program. Lorain has been really creative in how they would continue to maintain this program for their students and how over time, they’ll scale it up to eventually serve at least a thousand incoming low-income students.

Other colleges in Ohio took a different approach. For example, Cuyahoga County Community College decided that they wanted to actually take lessons from their program and infuse it throughout their general services and take those lessons for their general population, while the third college is still doing a little bit of both. They’re taking lessons for their general population, but also trying to identify funding to continue the program. It was interesting to see the different approaches that the colleges in Ohio took to sustaining the program and taking lessons from the demonstration.

Thanks to Camielle and Christine for joining me and sharing these exciting results. To learn more visit mdrc.org. Did you enjoy this episode? Subscribe to the Evidence First podcast for more.