When schools or programs face challenges in delivering services — such as limited time — and researchers are not on-site to monitor their implementation, how can the researchers know what is happening and how it varies across sites? MDRC’s evaluation of the Response to Intervention reading framework highlights ways to document how schools use their time and which students receive services.

The Response to Intervention (RtI) framework has gained a foothold in a majority of schools and districts over the past decade.  The idea is to screen all students at the beginning of the school year and offer additional support, or interventions, to students who demonstrate difficulty reading. The framework typically specifies providing interventions only to struggling readers and in addition to the 90-minute reading block that all elementary school students in RtI schools are supposed to receive.

Our implementation study revealed variation in how schools delivered reading services and used their limited time during the school day. Subsequent studies also have found variation in fidelity and implementation.

In particular, we found that while some schools did implement the program as intended, others did not. These other schools, instead of adding time to the reading block for struggling readers, implemented the interventions during the core reading block itself. Further, schools were not targeting the interventions solely to struggling readers. Interventions in some cases were replacing, not supplementing, core instruction.

While the recruited schools met key standards associated with the RtI framework, the study’s measurement approach acknowledged that each school might implement the framework differently because of varying lengths of the school day, staffing arrangements, and  concentrations of student need. Our surveys asked how time and staff members were allocated for different intervention groups, recognizing that small groups were the relevant unit of service delivery. This allowed the team to explore important variation within and between schools.

Researchers also can use service logs and time diaries to reveal variation in implementation, especially in large multisite studies, when extensive interviews and focus groups are not feasible. 

Suggested citation for this post:

Balu, Rekha. 2017. “Anticipating Variation in Implementation: The Multistate Evaluation of Response to Intervention Practices.” Implementation Research Incubator (blog), May. http://www.mdrc.org/publication/anticipating-variation-implementation.