Collecting data on subjects at multiple points in time allows researchers to examine how and when outcomes change over time. By incorporating this type of longitudinal data into a random assignment study, researchers can test not only whether there is a difference between the outcomes of program and control group members at one point in time, but also whether there is a statistically significant difference in the rate of change between program and control group members’ outcomes.

For example, in a randomized trial of a preschool math curriculum versus business-as-usual preschool, collecting data on students’ math skills at multiple points in time — say in preschool, kindergarten, first grade, and third grade — would allow one to estimate the impact of the intervention on growth in math skills through the end of third grade and test whether the impact is statistically significant. This information is particularly useful because it can help pinpoint when interventions are most efficacious and how programs change trajectories, without it being necessary to run a separate test for each outcome at each point in time. It also allows the researcher to fit linear and nonlinear parameters, depending on the functional form of the data. Longitudinal data collected in descriptive studies are also valuable because they make it possible to examine individual research subjects’ trajectories over a given time period, helping both to identify informative patterns in the data and to illuminate developmental processes.  

Individual growth modeling is one analytic approach that can be used to model data where repeated measures are collected for subjects (for example, students, parents, families, schools, etc.). The most basic growth model is composed of the fixed and random effects that best capture the collection of individual trajectories over time. A fixed effect represents a single value that exists in the population and a random effect represents the random probability distribution around that fixed effect. In the growth model, the fixed effects represent average trajectories of all the individuals in the sample, and the random effects represent the variance of the individual trajectories around these group means. Taken together, the fixed and random effects capture the general characteristics of growth both for the sample as a whole and for the individuals within the sample.

To conduct this analysis, one needs data from a minimum of three points in time for each subject with a time-varying outcome variable, and a way to operationalize time (for example, child age at the time of assessment, or data-collection time point). The predictors can vary across time as well, but they need not. For example, if one had multiple test score data from multiple points in time in elementary school and time-invariant demographic information on children, one could estimate changes in test scores for lower-income versus higher-income students, or dual language learners versus native English speakers, and further test whether these trajectories were significantly different from one another.  

It is important to fit a longitudinal model that maps onto the research question of interest, particularly if one is interested in specific inflection time points or in considering nonlinearity in the data. For the last few years, our research team has partnered with the Boston Public Schools Department of Early Childhood, the University of Michigan, and the Harvard Graduate School of Education to collect longitudinal data on a sample of students enrolled in Boston’s prekindergarten program as they move through third grade. A central goal of this study was to examine how rates of growth in students’ academic skills varied during the prekindergarten year, over the summer after prekindergarten, and during the kindergarten year. Our hypothesis was that growth rates would be relatively similar in prekindergarten and kindergarten but decrease during the summer.

To test this hypothesis, we used data on students’ outcomes over time and fit a regression with students’ language and math skills as outcomes and individual indicator variables representing the spring of pre-K, the fall of kindergarten, and the spring of kindergarten as independent variables (with the fall of pre-K as the reference group), with random effects for students and schools. In some research areas, this is called an individual growth model. We used the coefficients on the fixed effects from our models to calculate growth rates during the prekindergarten year, the summer before kindergarten, and the kindergarten year, and tested whether those three growth rates were significantly different from one another. We then considered how growth rates varied for different subgroups of students: White versus non-White students, low-income versus higher-income students, and dual language learners versus native English speakers.

We found that on average, growth in children’s math and language skills slowed during summer, but the patterns varied by domain and group. For example, non-White and dual language learner students showed the largest drop-off in language skills during summer. In addition, lower-income students demonstrated slower summer growth in math skills than their higher-income peers. Subgroups of students had similar rates of growth in academic skills during the prekindergarten and kindergarten years, suggesting that the summer between prekindergarten and kindergarten may be an important time period contributing to disparities in early academic skills.   

Ongoing work will build on this research to continue examining students’ trajectories through third grade. Future descriptive analyses could use this longitudinal framework to explore mechanisms across time, for example by testing whether the quality of summer learning experiences throughout elementary school mediates associations between changes in children’s social-emotional skills and growth in academic skills through third grade. Future experimental research could use longitudinal data collection to test whether random assignment to a summer learning program had an impact on growth in academic skills during the summer after preschool and beyond. In designing any longitudinal study, researchers should be careful to consider the attrition of subjects over time, the need to collect the same measures to assess outcomes at each time point (to the extent possible), and the psychometric properties of the outcome measures and their data-collection method.

The full article summarizing the results from the analysis of growth in students’ academic skills over time appears in the American Education Research Journal.