Introduction

The U.S. Department of Education’s Institute of Education Sciences has released the interim report from the Reading First Impact Study, a Congressionally-mandated evaluation of the federal government’s $1.0-billion-per-year initiative to help all children read at or above grade level by the end of third grade. The evaluation is being conducted by Abt Associates and MDRC with Westat, Rosenblum-Brigham Associates, RMC Research, Computer Technology Services, DataStar, Field Marketing Incorporated, and Westover Consulting.

Reading First is a curricular and instructional cornerstone of The No Child Left Behind Act of 2001 (PL 107-110, Title I, Part B, Subpart 1). It builds directly on recommendations from the National Reading Panel’s (NRP’s) review of scientifically-based reading research. The NRP identified five areas of instructional practice that were found to be essential to teaching children to read: phonemic awareness (manipulation of individual speech sounds), phonics (mappings between sounds and print), fluency (improved speed and accuracy in oral reading), vocabulary, and text comprehension. Reading First is the largest federal funding initiative ever undertaken with the explicit goal of increasing classroom teachers’ use of research-based reading instructional practices. It does so through support for teacher professional development, curricula, materials, coaching, assessments, and supplemental interventions.

Rather than an evaluation of the NRP’s conclusions, the Reading First Impact Study is an assessment of the extent to which this federal funding stream increases teachers’ use of the essential elements of reading instruction and improves students’ reading comprehension skills. The present report — the first of two from the study — examines the impact of Reading First funding in 2004-2005 and 2005-2006 in 17 school districts across 12 states and in one statewide program (for a total of 18 sites). In summary, the interim report’s findings are:

• On average, Reading First changed teachers’ instructional practice by increasing the time they spent on the five essential components of reading instruction that were the focus of the program.



• On average, across the 18 participating sites, Reading First did not improve students’ reading comprehension test scores.



• These overall averages may mask potentially important variation in the findings. Study sites that received their Reading First grants later in the federal funding process experienced positive and statistically significant impacts both on the time teachers spent on the five essential components of reading instruction and on first- and second-grade students’ reading comprehension. In contrast, the other sites experienced no statistically significant impacts on instruction in the five dimensions, and their estimated impacts on reading comprehension scores were negative and not statistically significant. The differences in impacts between the groups of sites should be interpreted cautiously, however, because some of the differences were not statistically significant and the analysis was not able to determine the specific sources of the difference in impacts.

The story of Reading First’s implementation and impact is still unfolding. The study’s final report will provide an additional year of follow-up data and will provide a more detailed assessment of the relationship between teachers’ use of scientifically-based reading instruction and students’ reading comprehension skills. The final report from the study is scheduled for release later in 2008. 

The Reading First Program

Reading First is a federal funding stream designed to support instructional practices that were validated by the National Reading Panel’s systematic review of rigorous reading research. The NRP’s Congressional mandate was to synthesize the extensive body of scientifically-based reading research on how critical reading skills are most effectively taught and what instructional materials and approaches are most beneficial for students of varying abilities. Based on its review of more than 100,000 studies, the NRP issued a final report, Teaching Children to Read: An Evidence-Based Assessment of the Scientific Research Literature on Reading and Its Implications for Reading Instruction, in 2000 that identified five areas that were found to be essential to teaching young children to read: phonemic awareness, phonics, fluency, vocabulary, and text comprehension.

The Reading First legislation defines its primary goal as supporting the five essential components of reading instruction that were identified by the NRP. The administrative guidelines for the Reading First Program provides further direction to prospective state, district, and school grantees about materials and activities for which funding can be used:

• Reading curricula and materials that focus on the five essential components of reading instruction as identified by the National Reading Panel and specified in the Reading First legislation;



• Professional development and coaching for teachers on how to use scientifically-based reading practices and how to work with struggling readers; and



• Diagnosis and prevention of early reading difficulties through student screening, interventions for struggling readers, and monitoring of student progress.

Reading First grants were made to states between July 2002 and September 2003. By April 2007, states had awarded subgrants to 1,809 school districts, which had provided funds to 5,880 schools. Districts and schools with the greatest demonstrated need, in terms of student reading proficiency and poverty status, were intended to have the highest funding priority. In addition to grants for individual schools, states and districts could reserve up to 20 percent of their Reading First funds to support staff development and reading assessments, among other activities, for all high-need schools. 

Impact Findings

The Reading First Impact Study is not an evaluation of the NRP findings, nor is it a test of whether the instructional practices identified in the NRP report produce improvements in students’ reading skills. Rather, the study is an assessment of a federal funding stream and whether that funding and the accompanying parameters for its use changed teachers’ classroom instructional practice and improved students’ reading achievement. It compares a group of 125 schools that were selected to receive a Reading First funding grant (referred to here as Reading First schools) with a group of 123 schools from the same sites that did not receive a grant (referred to here as non-Reading First schools). Reading First schools in the study sample received their grant awards between April 2003 and August 2004. The average grant across the study sites was $409 per pupil and ranged from $173 to $856 per pupil. The 248 study schools come from 17 schools districts and one state-wide Reading First program. Although the study sites are a not national probability sample, they share many characteristics with the national population of schools receiving Reading First grants.

The Reading First Impact Study employs regression discontinuity designs in 17 of the study sites and a randomized experiment in one site. Regression discontinuity is the strongest quasi-experimental method that exists for estimating program impacts. The application of the design for the Reading First Impact Study capitalizes directly on the systematic process used by a number of school districts to allocate their Reading First funds. Non-Reading First schools in regression discontinuity analyses play the same role as do control schools in a randomized experiment — they represent the best indication of what outcomes would have occurred for the treatment group (Reading First schools) in the absence of the program being evaluated.

Key findings from the study include:

• On average, Reading First increased the amount of time that teachers spent on the five essential components of reading instruction promoted by the program. In the absence of a Reading First grant, the study estimated that first-grade teachers in non-Reading First schools spent an average of approximately 51 minutes of their reading classes (typically about 90 minutes per day) focusing on the five essential components of reading instruction and second-grade teachers spent approximately 47 minutes per class on these components. Reading First increased this instructional time by approximately 8.6 minutes per class for first-grade teachers and 12 minutes per class for second-grade teachers. This reflects an additional 45 minutes per week for first-grade teachers and 60 minutes per week for second-grade teachers over and above the time they would otherwise have spent on these components of reading instruction.



• On average, Reading First did not improve students’ reading comprehension test scores. Across the 18 study sites and two years of data collection, 45 percent of first-grade students in the Reading First schools scored at or above grade level on the SAT-10 reading comprehension test. This represents a 3 percentage point increase (that is not statistically significant) over first-grade students in non-Reading First schools. Thirty-nine percent of second-grade students in both Reading First and non-Reading First schools scored at or above grade level, indicating no difference between the two groups of schools. Thirty-eight percent of third-grade students scored at or above grade level, reflecting a 2 percentage point decrease (that is not statistically significant), compared to third-grade students in non-Reading First schools.



• The impacts of Reading First on teachers’ instructional practice and students’ reading achievement are consistently positive for late award sites and mixed for early award sites. For grades one and two in sites that received their Reading First grants in 2004, the program produced positive and statistically significant increases in both teachers’ instruction in the five dimensions and in students’ reading comprehension. In early award sites that received their grants in 2003, program impacts were not statistically significant, with impacts on instruction being smaller in magnitude and impacts on student reading comprehension being negative.



• It is not possible to determine what characteristics of early award sites and late award sites may have caused observed differences in Reading First impacts. The average Reading First grant was higher in late award sites than early award sites ($574 versus $432 per student per year). It also appears that reading comprehension test scores were lower for non-Reading First schools in the late award sites compared to their counterparts in the early award sites, suggesting that these late award sites were serving more educationally disadvantaged students who were further behind. These differences, plus others not measured, could have produced the impact differences observed. 

Next Steps

The study shows that teachers in both Reading First schools and non-Reading First schools spent a considerable amount of class time focusing on the five essential elements of reading identified by the NRP. Nevertheless, Reading First increased the amount of time teachers focused on these elements over and above what would have occurred without the funding grants that were provided to the Reading First schools. At the same time, impacts on student reading comprehension test scores were not statistically detectable on average, and schools receiving Reading First grants were still well short of the program’s ultimate goal of ensuring that all students are reading at grade level by the end of third grade. On average, more than half of third-grade students in the study sample’s Reading First schools were performing below grade level three years into the initiative. Finally, the study’s interim report provides suggestive evidence that Reading First may improve reading achievement under circumstances where the program had larger effects on teachers’ instructional practice. This speculative evidence will be investigated in more detail in the study’s final report.

The Reading First Impact Study has completed its data collection, which will provide considerably more information for the final report, including three years of data on students’ reading comprehension and teachers’ classroom instruction. The final report will draw upon additional data collected in the 2006-2007 school year, including assessments of first-grade students’ decoding skills and surveys of educational personnel. The availability of these additional sources will allow the research team to answer questions about the impact of Reading First more definitively, to conduct a more detailed analysis of relationships between instruction and reading achievement, and to assess whether there are meaningful variations in impacts.