Since 2010, MDRC has published a series of reports from its ongoing study of small, nonselective public high schools in New York City, called “small schools of choice” (SSCs) by the researchers. The latest findings, released October 16, 2014, include:

• Sustained impacts on graduation with Regents diplomas: Small high schools in New York City markedly increased high school graduation rates for large numbers of disadvantaged students of color, even as graduation rates were rising at other New York City high schools. For the full sample, students at small high schools have a graduation rate of 71.6 percent, compared with 62.2 percent for students in the control situation — a 9.4 percentage point gain. The higher graduation rate was driven by students earning Regents diplomas. These effects were seen among nearly all subgroups of students who attended the small high schools, including special education students.
• Positive effects on college-going: Attending a small high school increased the percentage of students who graduated from high school in four years and enrolled the next year in a postsecondary institution by 8.4 percentage points (to 49.0 percent). Most subgroups, including black males, black females, and students eligible for free/reduced-price lunch, experienced these effects. Small high schools modestly increased enrollment rates in postsecondary schools at every selectivity level, including competitive and very competitive schools, as defined by Barron’s ratings.
• Lower costs per graduate at the small high schools: The small high schools achieved these gains at a lower cost per graduate than that of the high schools attended by their control group counterparts — roughly 14 percent to 16 percent lower. This is in large part because more students successfully graduate from small high schools and fewer need to attend an expensive fifth year of high school.

Small schools of choice are more than just small. They were developed and approved through a competitive proposal process and designed to stimulate innovative ideas for new schools by a range of stakeholders and institutions, from educators to school reform organizations. The resulting schools emphasize academic rigor; strong, sustained relationships between students and faculty; and community partnerships to offer relevant learning opportunities outside the classroom. Each SSC also received start-up funding as well as assistance and policy support from the district and other key players to facilitate leadership development, hiring, and implementation.

Over the years, readers have asked a number of questions about the study, including:

Who asked MDRC to undertake this study?

Nonprofit networks of small schools in New York City asked MDRC, an independent, nonpartisan research firm, to conduct an evaluation to see whether the new schools were making a difference for students. MDRC approached the Bill & Melinda Gates Foundation, which agreed to provide the funding. MDRC conducted the analysis independently.

What question does the study answer?

MDRC’s study addresses the question: “By how much does attending an SSC improve academic outcomes for students who attend them, relative to what these outcomes would have been if these students had not attended an SSC?” The methodology used to address this question is based on a lottery, which, like a randomized control trial, is widely considered to be the “gold standard” for studying effects of educational, social, or medical innovations. This approach relies on the fact that, with a large sample, randomly determining who is assigned to an innovation and who is not creates two groups that are the same on average in all ways, except for their assignment to the program. Any future differences in average outcomes for the two groups thus can be attributed to the innovation. In this way, MDRC has been able to reliably assess the effects of SSCs on high school graduation, postsecondary enrollment, and school costs.

How was the study conducted?

This methodology was made possible by New York City’s high school application processing system (HSAPS), which assigns over 90 percent of the city’s entering ninth-graders each year to high school. Each year, eighth-graders in New York City are required to select in rank order of priority up to 12 high schools that they want to attend; when an SSC has more applicants than spaces, the district uses a randomized process to break ties and assign students to the SSC or to another school in the district. When aggregated across SSCs, this creates two very large groups of entering ninth-graders that are identical in all ways except for their school assignment. The study’s methodology meets the federal What Works Clearinghouse’s highest standard of evidence.

Was the study peer-reviewed? Have the findings been confirmed independently?

The federal What Works Clearinghouse reviewed the study, finding that it met the Clearinghouse’s highest standards of evidence “without reservations.” In addition, findings from the study have been published in the Spring 2014 issue of the peer-reviewed Journal of Policy Analysis and Management. The results have also been independently confirmed by scholars at MIT and Duke University. The new cost analysis paper is currently under review at a peer-reviewed journal.

What schools are included in the study?

The study includes more than 21,000 students — students who enrolled in 105 of the 123 academically nonselective small schools that were created after 2002 and students who wanted to enroll in an SSC but were lotteried into one of approximately 200 other high schools in New York City. Thus, over the course of the study, it has included findings from over 85 percent of the SSCs that were created during the analysis period. This degree of coverage for a “gold standard” randomized trial of a large-scale educational initiative is virtually unprecedented. Even if the remaining 15 percent of SSCs had zero effect on rates of high school graduation, this would not change the overall average effect appreciably.

Most of the SSCs in our study were in either Brooklyn or the Bronx. The schools that control group students attended were quite diverse but were generally older and larger than the SSCs. The study does not compare the SSCs with the large, failing high schools they replaced. Because these schools were no longer available to entering ninth-graders, they were not part of the comparison against which SSCs were judged.

What are the characteristics of the students in the study?

Due to the lottery process that forms the basis of the study, the students who win lotteries for SSCs and the students who lose lotteries for SSCs are virtually identical on all characteristics. The students in the MDRC study are predominantly black and Hispanic and are very disadvantaged:

• 93.9 percent are black or Hispanic,
• 84 percent are eligible for free or reduced-price lunch,
• 25.1 percent were overage for eighth grade, and
• 75 percent entered high school performing below grade level in reading or mathematics.

The study included students who listed an SSC as their first choice, their second choice, and even lower choices. Students who listed an SSC as a relatively low choice experienced effects that are as large as those who listed an SSC as their first choice. In addition, the sample contains students who made themselves known to an SSC by calling it, visiting it, or visiting its desk at a high school fair and students who did not make themselves known to an SSC. Positive effects are as large for students who did not make themselves known as they were for students who did make themselves known. 

What about special education students and English language learners?

New small schools that opened between 2002 and 2007 were not required to serve English language learners or special education students in their first two years of operations while internal capacity was still being built. By their third year of operations, schools that opened between 2002 and 2007 were required to admit students with special needs. Schools that opened in 2008 or later were expected to serve students with special needs from their inception.

The most recent findings demonstrate that SSCs produced a statistically significant 13.4 percentage point impact on high school graduation for special education students. In addition, Table A.1 in Appendix A of the 2014 cost study indicates that, beginning in the 2007-2008 school year (which is the year that our fourth cohort entered ninth grade), SSCs served a higher share of special education students compared with other New York City high schools. However, it is important to note that even though SSCs serve a higher share of special education students, it is possible that they serve a less costly mix of special education students (see the next question for more information).

SSCs serve a somewhat smaller share of students with limited English proficiency compared with other New York City high schools. While we find positive effects of attending an SSC on high school graduation for these students as well, these effects are not statistically significant.

Does the study examine the effects for special needs students enrolled in self-contained classes?

No. Special needs students enrolled in self-contained classes are not assigned to schools by the HSAPS system and thus are not included in this study. Only special needs students who were assigned to schools via the HSAPS lottery process are included in the study. HSAPS is used to assign 90 percent of all New York City high school students to schools; special needs students in need of self-contained classes are not assigned by that system. As noted above, an HSAPS lottery only occurs when a school is oversubscribed — more applicants than seats. Lotteries occurred in 105 of the new small high schools of choice. Many of these small schools also include self-contained classes for special needs students — although they were not a part of this study.