An Experienced School Support Organization at Scale: A Study of The Urban Assembly Network
Overview
Amid modest increases in high school graduation rates for all students, disparities still exist for important student groups, such as students of color and students from low-income families. In response to this equity issue, there has been a growth in third-party "school support" organizations like Urban Assembly. Since 1997, Urban Assembly has supported New York City schools with the mission to improve all students’ economic and social mobility, with a focus on “meeting students where they are,” even when they enter high school performing below grade level.
Under contract with the New York City Department of Education, in the 2019–2020 school year, Urban Assembly supported 23 “unscreened” schools serving roughly 9,000 middle and high school students throughout Manhattan, Brooklyn, and the Bronx. Urban Assembly provides their schools with professional development to school-based staff using two formats: professional learning communities (PLCs) and coaching. They also offer schools five main program areas of support: academic, leadership, social emotional learning, postsecondary readiness, and alumni success.
Despite the significant monetary investment in school support organizations, such as Urban Assembly, there is limited peer-reviewed research on their effectiveness. This study aims to fill that gap and yield lessons for other school support organizations on best practices. Additionally, this study will evaluate the effects of the Urban Assembly school support network on the academic and socio-emotional outcomes of students, as well as apply the most rigorous causal approach to understanding what works to improve their outcomes.
Additional Project Details
Agenda, Scope, and Goals
The research team is conducting a comprehensive study of the landscape of school support organizations working with New York City Department of Education public schools, the implementation of the Urban Assembly model, and the effects of the Urban Assembly model on students. More specifically:
The landscaping strand of work will explore the following research questions:
- What is the landscape of the larger system of high school support in the New York City Department of Education?
- Does the implementation of the Urban Assembly model create differences from other school and student support practices in the New York City Department of Education?
The implementation strand of work will explore the following questions:
- To what extent are the core program areas of Urban Assembly’s model being offered as intended? To what extent do schools participate in the core program areas offered by Urban Assembly as intended?
- What are the conditions that facilitate or challenge Urban Assembly’s ability to support their schools?
- Within Urban Assembly schools, how does the organizational structure of the school and its programming support students?
- To what extent are the core program areas of Urban Assembly’s model (academic, leadership, social emotional learning, postsecondary readiness, and alumni support) being offered as intended in schools? To what extent do students experience the core program areas of Urban Assembly’s model?
- What are the conditions that facilitate or challenge Urban Assembly schools’ ability to support their students?
The impact strand of work will explore the following questions:
- Treatment Contrast: What is the impact of assignment to an Urban Assembly school on students’ experience of the key features of the theory of change overall and by its core components?
- Short-term Impacts: What is the impact of assignment to an Urban Assembly school on students’ outcomes during high school, including: measures of student social and behavioral competencies (including indicators of growth mindset, resilience, response to challenge, and intrinsic motivation) and progress towards high school graduation (including students being on-track at the end of ninth grade, academic credit accumulation and GPA, CTE-specific credit accumulation, New York State Regents examinations, attendance, and mobility)?
- Longer-term Impacts: What is the impact of Urban Assembly on longer-term outcomes, including: receipt of New York State diploma; enrollment, persistence, and completion of two-, four-, or six-year college degrees, or license/certificate programs; and postsecondary employment and earnings?
- Impact variation: Do Urban Assembly schools with specific features produce different effects?
- Cost analysis and effectiveness: How do the costs of Urban Assembly compare to business as usual costs?
Design, Sites, and Data Sources
The research team will take advantage of the rich array of data available.
Landscaping. The landscaping strand of work will explore the context for intervention, by capturing the New York City Department of Education’s system of school support organizations. In turn, these data will be used to document the comparable practices available to non-Urban Assembly schools and students.
Implementation. The implementation strand of work will document the implementation of the Urban Assembly model, assess the fidelity of model implementation, and compare the practices of Urban Assembly and its schools to those of other school support organizations and their schools.
Impact. The impact strand of work will be anchored in a high-powered, naturally occurring randomized controlled trial design executed to meet federal What Works Clearinghouse standards, complemented by a uniquely rigorous propensity score matching approach. This design serves to answer questions regarding treatment contrast, short-term impacts, long-term impacts, impact variation, and the cost-effectiveness for policymakers and practitioners who are supporting similar populations.
