An Experienced School Support Organization on a Large Scale: A Study of the Urban Assembly Network
Overview
Since 1997, Urban Assembly has worked with New York City schools with the mission of improving all students’ economic and social mobility, with a focus on “meeting students where they are,” even when they enter high school performing below grade level.
Under contract with the New York City Department of Education, in the 2019–2020 school year, Urban Assembly supported 23 “unscreened” schools serving roughly 9,000 middle and high school students in Manhattan, Brooklyn, and the Bronx. Urban Assembly provides its schools with professional development to school-based staff using two formats: professional learning communities and coaching. It offers schools five main program areas of support: academics, leadership, social-emotional learning, postsecondary readiness, and alumni success.
Although there has been significant monetary investment in school support organizations such as Urban Assembly, there is limited peer-reviewed research on their effectiveness. This study aims to fill that gap and yield lessons for other school support organizations on best practices. Additionally, this study will evaluate Urban Assembly's effects on the academic and social and emotional outcomes of students, applying the most rigorous approach to understanding what works to improve their outcomes.
Additional Project Details
Agenda, Scope, and Goals
The research team is conducting a comprehensive study of the landscape of school support organizations working with New York City Public Schools, the implementation of the Urban Assembly model, and the effects of the Urban Assembly model on students.
The landscape strand of work will explore the following research questions:
- What is the landscape of the larger system of high school support in New York City Public Schools?
- Does the implementation of the Urban Assembly model create differences from other school and student support practices in New York City Public Schools?
The implementation strand of work will explore the following questions:
- To what extent are the core program areas of Urban Assembly’s model being offered as intended? To what extent do schools participate in the core program areas offered by Urban Assembly as intended?
- What are the conditions that facilitate or hinder Urban Assembly’s ability to support its schools?
- Within Urban Assembly schools, how does the organizational structure of the school and its services support students?
- To what extent are the core program areas of Urban Assembly’s model (academics, leadership, social-emotional learning, postsecondary readiness, and alumni support) being offered as intended in schools? To what extent do students experience the core program areas of Urban Assembly’s model?
- What are the conditions that facilitate or challenge Urban Assembly schools’ ability to support their students?
The impact strand of work will explore the following questions:
- Service contrast: What is the impact of assignment to an Urban Assembly school on students’ experience of the features of the model's theory of change overall and of its core components?
- Short-term impacts: What is the impact of assignment to an Urban Assembly school on students’ outcomes during high school: measures of student social and behavioral competencies (including indicators of growth mindset, resilience, response to challenge, and intrinsic motivation) and progress towards high school graduation (including students being on track to graduation at the end of ninth grade, academic credit accumulation and grade point average, career and technical education credit accumulation, New York State Regents examinations, and attendance)?
- Longer-term impacts: What is the impact of Urban Assembly on longer-term outcomes: receipt of a New York State diploma; enrollment in two-, four-, or six-year college programs, as well as persistence in those colleges and completion of degrees or license/certificate programs; and employment and earnings?
- Impact variation: Do Urban Assembly schools with specific features produce different effects?
- Costs: How do the costs of Urban Assembly compare with business-as-usual costs?
Design, Sites, and Data Sources
The research team will take advantage of many data sources.
Landscape. The landscape strand of work will explore the context for intervention, describing New York City Public Schools' system of school support organizations. In turn, this information will be used to document the practices available to non–Urban Assembly schools and students.
Implementation. The implementation strand of work will document the implementation of the Urban Assembly model, assess the fidelity of model implementation, and compare the practices of Urban Assembly and its schools with those of other school-support organizations and their schools.
Impact. The impact strand of work will be anchored in a high-powered, naturally occurring randomized controlled trial design executed to meet federal What Works Clearinghouse standards, complemented by a uniquely rigorous propensity score matching approach. This design serves to answer questions regarding service contrast, short-term impacts, long-term impacts, impact variation, and the cost-effectiveness for policymakers and practitioners who are supporting similar populations.