Success Academy is a rapidly expanding charter school network in New York City, with schools located in the Bronx, Brooklyn, Manhattan, and Queens. In the 2016-2017 school year, Success Academy served roughly 14,000 students across 41 elementary, middle, and high schools, which at the time was about 13 percent of the students attending charter schools in the city and equivalent to about 1.2 percent of all New York City Department of Education students.

In this paper, MDRC looks back to the beginning of Success Academy and uses naturally occurring lotteries for the seven Success Academy schools in operation during the 2010-2011 school year to estimate the difference in students’ academic achievement caused by the opportunity to enroll in a Success Academy school. This work is part of an independent analysis that MDRC conducted for Success Academy’s 2015 Investing in Innovation (i3) federal grant application. The educational outcomes of elementary students who received placements in their local Success Academy schools by being selected in lotteries are compared with the outcomes of students who entered these lotteries but were not selected for their local Success Academy schools. Because not all students selected in lotteries enrolled in Success Academy schools and some of those not selected through lotteries for their local (or closest) Success Academy school ended up enrolling in a different Success Academy school, MDRC also estimates the effects of enrolling in a Success Academy school (for the lottery sample). A statistical adjustment makes it possible to compare those who won their lotteries and did enroll in Success Academy schools with those who lost their lotteries and did not enroll in Success Academy schools.

The preliminary analysis in this paper focuses on the 4,710 students (3,804 students entering kindergarten and 906 entering first grade) who applied to oversubscribed Success Academy schools in the spring of 2010 for the 2010-2011 school year and who therefore participated in lotteries. Only applicants from this round of admissions could be followed long enough to measure their academic achievement using New York State’s third- and fourth-grade mathematics and reading exams at the time the analysis was conducted. While 4,710 students is a relatively large sample, because the number of applicants greatly exceeded the number of seats available for the 2010-2011 school year, the sample only includes about 700 students who were offered admission. The resulting low ratio of lottery “winners” to all applicants limits the statistical precision of the estimates presented.

Within the constraints of this analysis, MDRC finds that for the subset of students who were selected through a lottery and then enrolled in a Success Academy school, enrollment had large, positive effects on students’ math achievement in third grade, roughly equivalent in magnitude to between one and one and a half typical years of learning for this age group. Effects on reading did not reach conventional levels of statistical significance but are positive and promising. A larger sample size would be necessary to obtain more precise estimates. The program’s effect was also estimated for all lottery winners, not just those who enrolled, and these estimates were similar but of a smaller magnitude.