On January 25, MDRC released the latest findings from its ongoing study of new, small, academically nonselective high schools in New York City, called “small schools of choice” (SSCs) by the researchers. The new brief reported that SSCs have: 

• Sustained impacts on graduation with Regents diplomas: Average four-year graduation effects have reached 8.6 percentage points (meaning nearly nine more graduates for every class of 100 entering ninth-graders). This effect is driven by an increase in Regents diplomas attained. 
• Positive graduation effects for virtually every subgroup, including students with low entering proficiency in math and English (levels 1 and 2, in New York City terminology), males and females, blacks and Hispanics, and students eligible for free and reduced-priced lunch.


• A positive effect on a measure of college readiness: a 7.6 percentage point (or 25 percent) impact on scoring 75 or higher on the English Regents exam (which exempts students from remedial English at the City University of New York). There was no effect on scoring 75 or higher on the math Regents exam.


• A five-year graduation effect: Students in the new small high schools are 7.1 percentage points more likely to graduate in five years than their control group counterparts (75.2 percent vs. 68.1 percent).

In the days since the release of the new findings, readers have asked a number of questions about the study, including:

Who asked MDRC to undertake this study?

Nonprofit networks of small schools in New York City asked MDRC, an independent, nonpartisan research firm, to conduct an evaluation to see whether the new schools were making a difference for students. MDRC approached the Bill & Melinda Gates Foundation and the New York City Department of Education (NYC DOE), who agreed to participate. The NYC DOE gave MDRC access to the necessary data, and the Gates Foundation provided the funding. MDRC conducted the analysis independently.

What question does the study answer?

MDRC’s study addresses the question: “By how much does attending an SSC improve academic outcomes for students who attend them, relative to what these outcomes would have been if these students had not attended an SSC?” The methodology used to address this question is based on a lottery, which, like a randomized control trial, is widely considered to be the “gold standard” for studying effects of educational, social, or medical innovations. This approach relies on the fact that, with a large sample, randomly determining who is assigned to an innovation and who is not creates two groups that are the same in all ways, except for their assignment to the program. Any future differences in average outcomes for the two groups thus can be attributed to the innovation.

How was the study conducted?

This methodology was made possible by New York City’s high school application processing system (HSAPS), which assigns over 90 percent of the city’s entering ninth-graders each year to high school. Each year, eighth-graders in New York City are required to select in rank order of priority up to 12 high schools that they want to attend; when an SSC has more applicants than spaces, the district uses a randomized process to break ties and assign students to the SSC or to another school in the district. When aggregated across SSCs, this creates two very large groups of entering ninth-graders that are identical in all ways except for their school assignment.

What schools are included in the study?

The study includes more than 21,000 students — students who enrolled in 105 of the 123 academically nonselective small schools that were created after 2002 and students who wanted to enroll in an SSC but were lotteried into one of approximately 200 other high schools in New York City. Most of the SSCs were in either Brooklyn or the Bronx. The schools that control group students attended were quite diverse but were generally older and larger than the SSCs. The study does not compare the SSCs with the large, failing high schools they replaced. Because these schools were no longer available to entering ninth-graders, they were not part of the comparison against which SSCs were judged.

What are the characteristics of the students in the study?

The students in the MDRC study are predominantly black and Hispanic and are very disadvantaged. For instance, of the students for whom we currently have graduation data (see p. 8 of the new brief):

• 93.2 percent are black or Hispanic,
• 83.9 percent are eligible for free or reduced-price lunch,
• 21.8 percent were overage for eighth grade,
• 70.5 percent were below grade level in reading in eighth grade, and
• 63.9 percent were below grade level in math in eighth grade.

Due to the lottery process that forms the basis of the study, the students who win lotteries for SSCs and the students who lose lotteries for SSCs are virtually identical on all characteristics.

What about special education students and English language learners?

Because of the lottery process, both the lottery winners and lottery losers have the same proportion of students in special education. That means that the MDRC study is comparing like with like (see Supplemental Table 1).

Supplemental Table 2 compares the students in the study sample who win a lottery for an SSC and enroll in an SSC with all the students who enroll in the SSCs (whether they got in by a lottery or not) and with all first-time ninth-graders in the city schools. Looking at the special education row, the 6.7 percent in the program group compares with the overall proportion of special education students in SSCs (15.5 percent), which is very similar to the overall average for all city schools (14.0 percent). That means that SSCs overall serve a similar proportion of special education students as other city schools — even though the MDRC sample (which represents only those students who participated in a lottery) has a somewhat smaller proportion of special education students.

The story is similar for English language learners.