PACE Center for Girls is a Florida-based nonprofit organization that serves girls of middle school and high school age who are at risk of involvement with the justice system. Girls who fit this profile often face challenges distinct from those faced by at-risk boys: They are more likely to have histories of abuse, trauma, truancy, substance abuse, and academic disruption. PACE aims to get these girls back on track by taking a gender-responsive approach, providing a blend of academic and social services that are shaped by an awareness of girls’ particular development and gender-specific needs.

Since its founding in 1985, PACE has been committed to being a learning organization: It has adjusted its services over time to align with emerging research, partnered with MDRC to evaluate the implementation and impact of its day program (illustrated on the next page), and consistently used the findings to launch improvements. Findings from the evaluation add to the body of knowledge about how gender-responsive principles can be implemented in different contexts to engage girls who are at risk. PACE’s partnership with MDRC and its improvement efforts also offer lessons for how researchers and practitioners can work together to produce rigorous and actionable evidence in the social sector and actively translate evidence into practice to refine and improve interventions.

Building an Effective Research-Practice Partnership

MDRC has worked very closely with PACE on every step of the evaluation, which was funded mainly through the Edna McConnell Clark Foundation’s Social Innovation Fund. The foundation was a key collaborator and supported the research-practice partnership throughout the evaluation.

During 2011 and 2012, MDRC worked with PACE to assess the feasibility of conducting a randomized controlled trial (RCT) across its program sites. These efforts included visits to several PACE centers for conversations with program leaders, staff members, and participants; training, technical assistance, and strategic planning for participant recruitment; and assistance with defining program goals and metrics. The two organizations worked collaboratively for two years to develop an evaluation plan, ensure the support of major stakeholders for the evaluation, assess funding and staffing needs, and examine the outreach and recruitment practices necessary to support an RCT. PACE staff members helped the MDRC team better understand the program, provided valuable guidance to develop the research questions, and helped incorporate study enrollment and data collection procedures into their regular work. They also participated in a training session on research procedures led by MDRC. The collaboration ensured that the research questions and evaluation design were grounded in the operational reality of program sites and what PACE hoped to learn from the research.

Between 2013 and 2015, PACE staff members played a critical role in helping to recruit and enroll participants into the evaluation, arranged site visits for the MDRC research team to observe program activities and speak with girls and local stakeholders, and participated in interviews and surveys about program implementation. Throughout the evaluation, the research team provided extensive verbal and written feedback that emerged from analyses of quantitative and qualitative data, allowing PACE to reflect on the findings and ask follow-up questions on interpretation, implications, further analysis, and dissemination before formal reports were shared with the public. As information became available, the PACE leadership worked actively to understand specific findings and use them for improvement efforts.

What the Evaluation Found

MDRC’s evaluation found that the PACE model positively affected girls in the short term, helping them to be more engaged in school and on track academically. Engagement in school is a protective factor against future justice system involvement, and the program’s academic impacts may lead to higher rates of high school graduation to help keep girls on a positive trajectory. Examining these potential benefits would require following the study sample for additional years. There was no real pattern of effects across self-reported measures of youth development and risky behavior in the 12 to 18 months following study enrollment.

The encouraging evidence on academic outcomes supports the findings from MDRC’s implementation analysis that while the content of academic classes at PACE was similar to what girls would learn in regular public schools, the holistic model reengages girls in school in a way that is different from a traditional school’s approach: It offers smaller classes, access to frequent individual academic advising, case management, counseling, and connections to other services in the community (like transportation or health care) in an environment that recognizes the strengths and individual needs of its girls. The organization successfully defines its model through both general program principles and a detailed policy and procedures manual and supports implementation through staff training and use of program data. The analysis found that while PACE’s model is implemented consistently across its locations, services varied somewhat in program areas where the model gave less specific guidance or because of differences in staffing across locations.

The MDRC evaluation has been a catalyst for change and reflection.
–Executive director of a PACE center

How PACE Used the Findings to Inform Continuous Improvement

In response to the implementation findings, PACE has launched efforts to (a) further codify the core services and principles that define its model and (b) strengthen organizational practices and systems for talent recruitment, learning and staff development, performance monitoring, and data collection (including a new data system that better integrates social service and academic service indicators and helps the staff access more comprehensive, holistic reports). These efforts include the following:

1. Defining its foundational principles and applying the concepts to staff roles. 
   
   The evaluation found that while PACE’s program culture was implemented uniformly, some staff members could not articulate the use of the principles in practice. Specifically, teachers with short tenures had a difficult time articulating gender-responsive approaches. PACE has further defined how staff members can use the foundational principles to support the program culture at its centers and has developed tools and training sessions to help program staff members in different roles more directly and intentionally apply the principles in their work. For instance, below is a sample of guidance from a new coaching tool that helps teachers apply the principles in their classrooms.
    

   Gender-responsive	Trauma-informed	Strengths-based
   Teacher communicates respect, compassion, trustworthiness, empathy, and active listening. Example: Facing a girl when she is speaking, and addressing concerns quietly without calling attention to individual girls.	Teacher uses de-escalation techniques rather than engaging in power struggles. Example: Lowering the voice rather than raising it during a conflict.	Teacher uses authentic and/or specific praise frequently. Example: Saying “that revision sounds much more descriptive” instead of “good job.”

2. Standardizing counseling services to better target girls’ risk factors. 
   
   The evaluation found no standard set of recommended counseling approaches to use in sessions with girls. In response, PACE has developed a guidance manual that identifies interventions and approaches and specifies whether they are for use by clinical or nonclinical staff members (for example, cognitive behavioral therapy for counselors and therapists). The manual also integrates guidance on the application of PACE’s three foundational principles in its delivery of social services. PACE is providing training to social service staff members on the practices in the manual and offering guidance to help managers connect girls to counselors with appropriate skills for their needs and to help managers provide counselors with ongoing learning and development opportunities.  
3. Standardizing instructional practices in classrooms.  
   
   Since girls enter PACE on a rolling basis with varying academic needs, teachers have to support girls in different grade levels and employ different curricula within the same class period, often relying more heavily on independent work than group work. Classroom observations for the implementation analysis found that many teachers offered limited academic support during independent work, and the program offered minimal guidance on instructional methods at the time of the visits. As a result of the findings, PACE has made efforts to incorporate standard practices across classrooms (such as requiring warm-up activities at the beginning of class to get students in the habit of being on task and help them recall prior lessons). PACE monitors lesson plans on a weekly basis and offers training and guidance on balancing teacher-led instruction and independent work. PACE is also building a digital platform to help teachers and academic managers share lesson plans across the organization.
4. Strengthening recruitment and retention of quality teachers. 
   
   The evaluation found that staff turnover affected consistency of implementation in some program areas. PACE managers reported significant turnover among teachers and faced competition from school districts in hiring and retaining them. Evaluation findings accelerated PACE’s efforts to strengthen its practices to recruit and train teachers and bolster incentives for retention. For example, the organization has increased teachers’ salary ranges and vacation time and now offers training to new teachers to help them obtain professional certification.

PACE’s commitment to learning and results, coupled with MDRC’s commitment to rigor, resulted in a partnership focused on building evidence to benefit the girls and communities we serve.
–Mary Marx, PACE President and CEO

Conclusion

Gender-responsive programs have been part of the landscape of services provided to girls and women in or at risk of entering the justice system for more than three decades. MDRC’s evaluation of PACE Center for Girls offers an opportunity to understand how a large and well-known gender-responsive program implements its model consistently in a diverse set of communities, and the encouraging short-term findings contribute to the growing literature on the effectiveness of such programs.

Moreover, PACE’s work to support the evaluations and its response to the findings exemplify how nonprofit social sector providers and researchers can effectively partner to generate actionable evidence that leads to continuous improvement. PACE is a growing and evolving organization, and it continues to assess the services provided and its implementation practices. The organization’s unique blend of services, its continuous learning agenda, and its program improvement efforts offer lessons for the field more broadly, especially for practitioners and policymakers looking for ways to better support at-risk young people.