Evidence First Podcast

Leigh Parise: Policymakers talk about solutions, but which ones really work? Welcome to Evidence First, a podcast from MDRC that explores the best evidence available on what works to improve the lives of people with low incomes. I'm your host, Leigh Parise.

Obtaining a college degree remains a critical route to economic mobility, but graduation rates at community colleges have remained low for decades, especially for students with low incomes and for students of color. Through an ambitious initiative called Expanding SUCCESS, MDRC is collaborating with state systems and community college districts with high proportions of students from economically disadvantaged and historically marginalized backgrounds to implement a comprehensive student support program called Scaling Up College Completion Efforts for Student Success (or SUCCESS for short). Joining me today is Dr. Emily Dow, assistant secretary of academic affairs at the Maryland Higher Education Commission (or MHEC), and Elena Serna-Wallender, an evidence-to-practice senior associate at MDRC. We'll discuss the collaboration between MHEC and MDRC and how they're supporting participating institutions.

Emily, Elena, welcome to Evidence First. I'm so glad you're here.

Elena Serna-Wallender: Thank you. I'm thrilled to be here.

Emily Dow: Same. Thank you. It's exciting.

Leigh Parise: Elena, let's start with a bit of background on this model of comprehensive support called SUCCESS. Tell us, what do we know about its effectiveness in supporting students through graduation, and what are some of the hallmark components of the model?

Elena Serna-Wallender: SUCCESS is backed by the strongest evidence base in higher education that we have. We know, from many different evaluations—done by MDRC, done by other researchers—that different kinds of programs or interventions (like student SUCCESS coaching, financial rewards, and tutoring, for example) can have small effects on student outcomes. But combining these kinds of strategies and offering them as a package and for longer periods of time can drive much larger improvements in those outcomes.

Several programs that take this comprehensive approach—like the City University of New York (or CUNY) ASAP program, the ASAP Replication in Ohio Colleges, One Million Degrees and Bottom Line—have in rigorous random assignment studies all shown these larger impacts. The SUCCESS model belongs to that family of comprehensive approaches to student support, and so it brings together a couple of really important components.

First and foremost, it centers on relationships. Students are paired with a dedicated advisor or coach who meets with them monthly and actively stays in touch with them. They're talking about lots of topics, not just what's happening in the classroom. It's a really holistic, high-touch approach.

Second, we want to build academic momentum, so students in the program are required to enroll in a certain number of credits each semester. Usually that's taking a full-time course load. In Maryland, some of the colleges are really wanting their students to focus on at least nine credits per semester, for example.

Third, we want to motivate students to stick with the program and also help them meet basic needs. So every month that students meet program requirements—and that would be meeting with your coach and enrolling in the number of credits that you're supposed to—they get a small financial incentive, like a gift card or cash.

The last component is that we want staff to be empowered by data to the biggest extent possible. That means that coaches who are working with students have access to real-time data about how students are doing in the classroom. It also means that, as much as possible, they have access to tools that give them access to case management and communications technology.

Leigh Parise: Elena, that's really great. Thank you for walking through that. We were recently meeting with someone who specifically called out that part of your title is “evidence to practice.” Just walking through that and talking about how we are going work with the colleges and help them apply what we've learned from the evidence—I think [that] is such a nice example of what we actually mean when we say that. So thank you.

Elena Serna-Wallender: Absolutely. Yeah, it's my favorite thing to do.

Leigh Parise: Love it. Okay. Emily, Maryland is one of the participating states. It would be great if you could talk about some of the reasons for joining Expanding SUCCESS and share a little bit of the background for us on the three participating colleges.

Emily Dow: Absolutely. We were excited when MDRC approached Maryland about this opportunity. In many ways, this is new territory for our agency. We are the state agency that coordinates and regulates postsecondary education in Maryland, and we hadn't really done this work around advising with the campuses. We usually do either true academic affairs—like program review—or financial aid. Thinking about advising and these other kinds of support that students may be getting on the campuses—this is new to us.

I'll also add that MDRC emphasized supporting a diverse array of students. Similarly, we came to the table asking that we engage a diverse array of colleges. What I mean by that is we wanted to identify small, medium, and large community colleges that also served diverse populations—we know that capacity to implement programs like SUCCESS can vary, depending on the size of the campus; we also have a relationship between our smaller campuses being in rural areas of Maryland—to not only reflect the diversity of students being served, but also the diversity of campuses and community college campuses here in Maryland.

Leigh Parise: That's great. I really love that you were able to include such a wide array of colleges—really recognizing the varied needs across the state. As we think about being able to help others learn from what you're doing, I think that's going to be one of the things that will make it possible: that people in other states are able to say, "Right. In our state, we also have a lot of variation in the ways that our campuses operate, and their sizes, and who they're serving."

On that note, for both of you, it'd be great to hear about how MHEC and MDRC are working together to support the institutions as they're focused on implementing this support.

Elena Serna-Wallender: We have been working together for about two years now, and essentially, the MDRC and the MHEC team work as one [laughs] at this point. That's a real testament to Emily and her leadership. MDRC took the lead with the program design process. We have this model—it's got the components that we've already talked about. But the way that that model works on any unique campus varies. There are lots of details to iron out. A lot of it depends on the student population that the campuses are serving, their own specific goals and strategic priorities. We spend a lot of time at the beginning of the design process talking to campus administrators and staff. We really want programs like SUCCESS to last: It's not supposed to be something that you do for a year and then it just goes away.

Because we bring in expertise from working with many other colleges before and doing this same thing, we led the process of working with the colleges to design their programs. The MHEC team was there every step of the way. They were so helpful in translating, you know, “Actually, in the state of Maryland, here are some things that we really prioritize, that we consider thinking about.” Maybe, Emily, you can talk about that.

Emily Dow: I want to echo everything that Elena has said. MDRC has been invaluable in providing technical support to me—in my leadership role—as well as the MHEC staff working on this project. Let me give you some concrete examples of what that means.

We've had weekly meetings between MHEC and MDRC, probably for the past 18 months or so, to discuss progress and consider the challenges that campuses might be facing in recruitment, retention, or data collection. There have been times Elena has brought in other content experts to help us think through and identify challenges, present solutions, while ensuring that we are adhering to the model as best as possible and then also giving it its Maryland flavor.

With MDRC being so hands-on, the team has been great to talk with about what other states and colleges are doing to implement similar programs, and what the research tells us: what works and what doesn't work. It really has been a collaborative effort. We could not have done this without MDRC being as hands-on as they have been. And in particular—not only just for our agency implementation, but again, back to the diversity of the campuses and the diversity of the students that they're serving—we do have to add some flavor; it can't be necessarily a rinse and repeat model. I think that's part of the lesson that I've learned, and MDRC's expertise on that has been, again, invaluable.

Elena Serna-Wallender: This project has a slightly different focus than the typical project for MDRC in that we are explicitly aiming to equip and empower MHEC to do this kind of work moving forward. We are setting up all these tools and templates that we use with the institutions as we launch and monitor and improve the programs. Because we talk so frequently, the MDRC team has gotten a front row seat to understanding how state agencies work. Working with Emily and with her research team makes us a better partner to state agencies and a better research partner as well.

Emily Dow: If I can add to that, we recognize that MDRC could have easily just gone to the three campuses independently and implemented this program, and we, of course, would've given it our blessing and said, “Go.” I think the advantage in leveling this up to our agency is we are able to now think strategically: How can we support advising services? How can we support campuses in their diverse efforts to support students in terms of retention and completion? It's reciprocal, back and forth: They're giving us some critical day-to-day operational project management support, and at the same time, MDRC is getting some insight in terms of how state agencies might work in this environment and really moving the marker around student success.

Elena Serna-Wallender: Absolutely.

Leigh Parise: I want to go back and dig in a bit deeper on one of the pieces about data. You talked about how, as part of this effort, institutions are gathering data that they're using to help continuously improve the program. I think that's something that people might be interested to hear more about: how that's happening and what that looks like. Because that's a goal that probably a lot of state agencies and a lot of campuses have.

Emily Dow: Sure. I can take the lead on that, Elena. Using data to help inform decision-making—that is a priority for our agency from the get-go. We were thrilled that MDRC held those same values for evaluation. Monthly tracking of student participation—that's our primary data point, at this point. That monthly tracking of student participation has been fascinating to review for this past fall semester. Having that data shows that students are participating in this program and does show some promising outcomes, just even to semester-to-semester retention.

The next big question is kind of the elephant in the room: Does it help with long-term retention and, ultimately, completion? I think what's essential in all of this is that our MHEC research team has been part of these conversations from the start, and I know they're excited to collect additional data for the spring semester and start tracking long-term outcomes.

Elena Serna-Wallender: I think something that's really important [about] the way that we talk about data with college staff is that we want it to be a flashlight and not a hammer. What I mean by that is, especially [with] advising staff—folks who are used to having to meet lots of different metrics and goals, and they're constantly spitting out reports for different people (that's a very compliance-based culture; it can be)—we want to make sure that anytime we are talking about data as a SUCCESS learning community that we come to [it] with a spirit of learning and openness and grace. That we really want to know, authentically, how this is working on the ground. Not for any “gotcha” moments or to point a finger at something that's not going well, et cetera. But truly to learn from each other, and to highlight the things that are working well and make sure that everybody across these colleges and at the state level knows, as far as we can tell, why that's going well. And that when there are challenges, we're rallying together to try to address those challenges.

It's a really special opportunity to have that kind of open collaborative space to learn together.

Leigh Parise: Great. I love that. I really appreciate your “it's a flashlight, not a hammer” metaphor. Thank you.

Emily, Elena, this has been a really great discussion. Is there anything else you feel like people should know before we sign off?

Emily Dow: I would add that it's exciting to be doing this work at a potentially statewide level. I'll re-emphasize [that] we support this work on individual campuses. That's great. Campuses need to be engaging with organizations like MDRC and other national partners to make evidence-based change. What's unique here is that there's a potential to do some of this work at a statewide level. We are learning so much about campus operations. So in the same way that MDRC is learning about MHEC operations, MHEC is equally learning about campus operations—which I think we all kind of knew (especially those of us who have been on campuses), but it gives us that visibility from a different perspective. It's been a great experience and I'm looking forward to the continued work.

Before I close out, I would like to give a shout-out to my MHEC team. First and foremost, certainly Secretary Rai. MDRC, I think, approached me and then quickly, thereafter, had a meeting with Secretary Rai. I have this very vivid memory [of] standing in his office and I was like, "You've talked with MDRC; I've talked with MDRC," and all he said was "Go. Do it." And I was like, "Okay, we're doing it. Here we go." Without his support, absolutely we would not be here. He comes from a campus, he knows the value of advising, and he knows the value of these kinds of student support services. It's great to have his leadership and vision.

I also want to recognize a couple of other people on the MHEC team: Dr. Barbara Schmertz, who is our director for research and policy analysis; Dr. Yuxin Lin, who is our associate director for research and policy analysis; and then Victoria Johnson, who has been a staff support in this project so far. They've been a pleasure to work with, along with the MDRC team.

Elena Serna-Wallender: Completely agree.

Leigh Parise: Well, that seems like a great place to wrap up. I hope, Emily, that we get to do more work together in the future. Thanks so much for joining me, Emily and Elena.

Emily Dow: Thank you.

Elena Serna-Wallender: Thanks so much.

Leigh Parise: This is a really exciting collaboration that will hopefully improve outcomes for many students in Maryland and across the country. To learn more about Expanding SUCCESS, visit MDRC.org. Did you enjoy this episode? Subscribe to the Evidence First Podcast for more.