Bottom Line

Pay for Success promises to generate funding to solve complex social problems while at the same time using ideas from the private sector — such as return on investment — to hold governments accountable. For the concept to work, parties in a Pay-for-Success deal must agree on three crucial items: (1) the number of people to be served, (2) the amount of time within which they must be served, and (3) the difference the program must make to be called a success. To balance these goals, someone involved in the deal must understand how a program is meant to work, the system it is meant to work in, and the people it is meant to work for; the strengths and limitations of the existing evidence concerning its model; and the ways it could affect government budgets. To deliver on its promise of accountability, governments must insist on paying for programs only when they show rigorous evidence of success.

What Is Pay for Success?

Policymakers and practitioners alike are focused with renewed urgency on improving the cost-effectiveness of the nation’s social programs. If ineffective programs could be weeded out, promising programs tested, and the most effective programs expanded, taxpayers would get more for their money — maybe even save money. There are plenty of ideas out there about how to improve programs and save money in the long term. But innovation costs money up front, the public and nonprofit sectors are often starved for cash, and there is no guarantee that these ideas will work.

Pay for Success attempts to solve this problem by shifting risk from the government to private investors. In the Pay-for-Success model, a private investor puts up the money for a new program, and if the program achieves results, then the government repays the investor plus a return. Most Pay-for-Success contracts are designed so that the government saves money if the new program succeeds, and some of those savings are then shared with the investor as profit. In other words, Pay for Success offers financial rewards to investors in exchange for taking on the risk of a program’s failure. Pay for Success has excited interest because it frees governments to experiment with promising new approaches, while at the same time holding those new approaches to defined standards of success, something social programs often lack.

The Three Crucial Numbers in a Pay-for-Success Deal

Because Pay-for-Success payments are based on the promise of government savings, parties involved in the deal must agree on a reasonable estimate of how much the government could save if the program succeeds. In the Rikers Island Pay-for-Success project, for example, a program provided skills and strategies to help teenagers detained in New York City’s Rikers Island jail avoid confrontations, with the goal of reducing the number of them who returned to jail. The city would save money if it had fewer people incarcerated, so to estimate the savings the program could generate, the parties had to predict how many people it could keep out of jail. The parties calculated that if the program reduced the number of people returning to jail by 10 percent, for example, then to generate meaningful savings it would have to serve more than 9,200 people. Because investors in any deal have to know when they will be repaid, the program needed to serve that number of people, and achieve those results, within four years.

In general, then, parties in Pay-for-Success deals need to reach agreement on three things: (1) the number of people to be served, (2) within what time period, and (3) with what expected results. To estimate the first two, the partners in the deal must know intimately how the social program involved is meant to work, the system it is meant to work in, and the people it is meant to work for. To predict what kinds of results it is reasonable to expect, they need to be able to absorb and comprehend the existing evidence about programs of this kind. Finally, to judge whether the program will generate sufficient savings, they must be able to estimate operating costs accurately and identify how the changes caused by the program could affect government budgets. Most private investors are not likely to have people with these skills on staff. To fill the gap, many Pay-for-Success projects are undertaken with the help of outside organizations familiar with research evidence, program design, and government cost accounting.

Base Payments on Impacts, Not Outcomes

The central concept of Pay for Success — that the government will only pay after a program has been proven to work — is a simple promise that can be surprisingly challenging to keep. Keeping it depends entirely on how one defines and measures success.

Too often, even sophisticated actors equate program outcomes (for example, the percentage of participants who return to jail within one year) with impacts (the change in that percentage that is caused by the program). MDRC encountered this issue in the Rikers Island Pay-for-Success project. The deal could have been structured so that the city paid back investors for every participant who did not return to jail (which would have been payment based on outcomes). But many of those participants would not have returned to jail whether the program existed or not. If payments had been based only on outcomes, then the city might have paid investors for all of those participants even though the program did not actually make a difference.

Evaluators measure the difference a program makes above and beyond what would have happened anyway by comparing the outcomes of program participants with the outcomes of a comparison group — a similar group of nonparticipants. An independent evaluator’s job is to identify a credible comparison group, one that truly represents what would have happened to the participants if the program did not exist. In the Rikers Island project, for example, incarceration could have declined for reasons unrelated to the program, including changes in policing or court practices. So the evaluation selected a comparison group who would have been affected by those changes the same way as the program participants.

The various parties in a Pay-for-Success project may balk at including evaluation expenses in a deal. Yet if a government ends up paying for programs that do not work, it will not only perpetuate the lack of accountability that Pay for Success is designed to counter, it will actually cost the taxpayers more than if the program had never been tried. When programs work, the government and taxpayers can pay with confidence. When they do not, the government and taxpayers do not pay, and decision makers learn what has not made a difference, a necessary step toward learning what can.

Support Deals That Pay for Rigorously Measured Impacts

Pay-for-Success projects raise new capital to finance taking risks and building evidence, and can increase government accountability by having governments pay only for success. Ultimately, they allow programs to expand if they are proven to work. A bill currently moving through Congress would support state and local governments as they implement Pay-for-Success projects. Such a bill would be a positive step, especially if it advances accountability by requiring rigorous impact evaluations to guide policy decisions. But governments and investors contemplating new Pay-for-Success projects must make sure they have the skills on hand to balance the three critical elements of a deal — or make sure they have partners on hand with those skills.

To learn more about Pay for Success, read Learning from Experience: A Guide to Social Impact Bond Investing, call Timothy Rudd at 212-340-8833, or e-mail Timothy.Rudd@mdrc.org.