Rural school districts in low-income communities face unique challenges in preparing and inspiring students to go to college. As part of a federal Investing in Innovation (i3) development grant, MDRC supported and evaluated an initiative that sought to improve college readiness in rural schools, in part by aligning teaching strategies across grade levels and between schools and the local college. Our implementation research allowed us to provide formative feedback that program leaders used to improve the process.

A collaborative effort to improve college readiness

We evaluated a collaborative initiative that involved

• three rural school districts (four middle schools and four high schools) in central Florida,

• a local college with satellite locations near each of the school districts, and

• a nonprofit organization whose goal is to close the opportunity gap for disadvantaged students — the AVID (Advancement Via Individual Determination) Center.

The initiative involved two main parts. First, it implemented the AVID College Readiness System (ACRS) — a model that focuses on ensuring that all students in a school are being prepared for success in college and careers, and which has three key components: (1) professional development for teachers, counselors, and administrators to support students’ critical thinking and study skills; (2) an AVID elective class that aims to help middle-achieving students who want to go to college but whose grades are falling short; and (3) school-based “site teams” of teachers and staff members to ensure successful implementation.

Second, AVID and its school partners developed alignment activities to build collaboration within and across grade levels, schools, and the college. These activities included (1) middle school and high school administrators working together to create smooth transitions for students; (2) teacher content collaboratives of secondary and postsecondary instructors in core subject areas meeting regularly to learn from each other and align their curricula; (3) school, district, and college administrators and lead teachers sharing best practices; and (4) college instructors participating in AVID training to learn the classroom strategies used by the middle school and high school teachers.

Implementation research leading to formative feedback

Early in our study, we worked with the AVID Center and its school partners to articulate the intervention’s theory of change in a logic model. This was an important part of our implementation research, because we needed to understand the process and what we should be measuring. At the same time, developing a logic model helped AVID and its partners articulate the key ingredients of their work together and the outcomes they were seeking. 

For three years, we measured the fidelity of each of the key AVID components to the model as designed, which allowed us to assess the growth in each component. We also conducted interviews and focus groups with staff members across the partnership, as well as with middle school and high school students at four of the eight participating schools. The perspectives of participants helped us learn about the obstacles to implementation as well as the factors that promoted it.

To support the development of the initiative, we shared information from these efforts — formative feedback — with the AVID Center after the first and second years of implementation so the partners could identify areas that needed improvement or expectations that needed adjustment. For example:

• We found that the middle schools and high schools were struggling to meet attendance goals for the teacher collaboratives. Teachers weren’t always able to attend the all-day collaborative meetings because schools found it difficult to find qualified substitute teachers in their rural communities. The AVID Center decided to reduce the number of teachers expected to attend the meetings and plan ways for the teachers who attended to share what they learned with their colleagues back at their schools.

• In focus groups, teachers said that the collaborative meetings lacked structure and a clear purpose. In response, the AVID Center clarified the purpose of each meeting, distributed agendas, and recommended that the same teachers from each school attend all the meetings so that each meeting could build on the content of the previous ones. School leaders also participated to ensure that the decisions were brought back to the schools and implemented. In later focus groups, teachers reported that the meetings had become more productive and useful, and the fidelity score for teacher collaboratives showed more successful implementation by the third year.

• Part of the plan for supporting middle-achieving students was for college students to tutor these students at the middle schools and high schools, but it was difficult to find and retain the tutors, especially given the long distances that some of the tutors had to travel. To ensure tutors were available, the AVID Center trained some junior and senior high school students to play this role.

The AVID Center and its school partners wasted no time in using such findings to make improvements. Over the three years, our findings suggest, most of the school partners were successful in implementing the ACRS component, though the alignment activities took more time to develop. The alignment activities — like the components of many new programs — faced unanticipated challenges. A core function of implementation research is to identify such challenges to give programs their best chance to succeed.