Agenda, Scope, and Goals

Descriptive Study

CAPR is conducting a descriptive study to establish a better understanding of the approaches used by colleges and states in the assessment practices, instruction, and structure of their developmental education programs; of new reform strategies that are emerging; and of the extent to which colleges are adopting more ambitious reforms. This five-year study asks the following research questions:

1. What assessment tools and practices do community colleges and other open-access institutions currently use to assess the reading, writing, and mathematics skills of incoming students and determine their readiness for college-level courses?

2. What considerations or factors underlie institutions’ choices for particular practices?

3. What are the major strategies they use to teach developmental education?

4. Are there different approaches used for traditional students (coming straight from high school) than for nontraditional students (who may be older and coming from the workforce or from adult education programs)?

5. To what extent are developmental education assessment and instructional practices aligned with the college and career readiness criteria reflected in the Common Core State Standards?

6. To what extent are developmental education reforms affecting the traditional open-door or open-access mission of community colleges?

7. To what extent have colleges been able to engage a significant proportion of faculty in developmental education reform efforts?

8. What is the nature and extent of the use of technology, especially online technology, for developmental education instruction?

9. Do the delivery of remediation and the characteristics and experiences of developmental students differ between for-profit and public institutions?

To answer these questions, CAPR is conducting a descriptive study based on a nationally representative survey of two- and four-year colleges as well as on other sources of evidence, including qualitative interviews with institutional and state-level representatives.

Assessment

Assessment and placement practices are an instrumental component of developmental education because they determine the extent to which incoming college students participate in developmental programming. Most community colleges currently use short standardized tests to determine placement into developmental education, and research indicates that this approach results in large numbers of placement errors. CAPR’s assessment study addresses the following questions:

1. How can colleges improve upon current assessments or assessment practices to make sure students are placed into appropriate-level courses?

2. What are the effects of alternative assessment and placement strategies on students’ overall academic performance, persistence, and progress toward college degrees?

The five-year study is conducted in collaboration with community colleges from the State University of New York (SUNY) system that are interested in modifying their assessment procedures.

The New Mathways Project

Research suggests that accelerated developmental math, better alignment between student goals and course content, and student-centered pedagogical practices may improve students’ progress through remediation and better prepare them for college-level courses. The New Mathways Project (NMP), developed by the Charles A. Dana Center at the University of Texas, incorporates each of these attributes by providing accelerated developmental and college-level math pathways, which differentiate math content and instruction to fit students’ career needs.

CAPR’s study of the New Mathways Project (NMP) addresses the following questions:

1. Do NMP students have better academic outcomes than students in traditional developmental math programs? Are these outcomes mediated through changes in student engagement?

2. To what degree is there fidelity to the NMP model across colleges? What aspects of NMP are consistent across sites? What adaptations were made and why?

3. How do the curriculum and pedagogy in NMP courses differ from the colleges’ traditional developmental math courses?

4. Is NMP cost-effective relative to business as usual?

The five-year study is conducted in collaboration with the Charles A. Dana Center and several Texas community colleges currently implementing the NMP model.

Design, Sites, and Data Sources

Descriptive Study

This descriptive study is principally built around a nationally representative survey of nearly 1,700 institutions. Other important sources of evidence will include qualitative interviews with institutional and state-level representatives; information drawn from existing data and research; and a detailed analysis of developmental students at a large, multi-campus, for-profit institution.

Assessment

This random assignment study evaluates a “data analytics” placement method whereby colleges use multiple measures to predict student performance in college-level math and English courses. In addition to placement test scores, these predictive measures may include high school GPA, high school course-taking patterns, and noncognitive assessments. Data on these measures will be used in a predictive model, developed in collaboration with the colleges. A decision rule can then be used to assign students to college-level math and English courses or developmental courses in math, reading, and writing.

Students entering participating colleges in the fall of 2015 through the spring of 2016 will be randomly assigned to be placed either: (1) using the new decision rule or (2) according to existing placement practices. Differences in later outcomes can then be attributed to the way in which students were placed. Students will be tracked for up to four semesters following placement to learn about their subsequent performance in college. The outcomes of primary interest will be completion of the first college-level courses in the relevant areas and total college-level credits earned.

The New Mathways Project

This random assignment study will use a research sample composed of approximately 2,000 students at four to six colleges in Texas who are assessed as in need of one or two levels of developmental math. Students who agree to participate and meet the study criteria will be randomly assigned into a treatment group that enters the yearlong New Mathways Project (NMP) program or into a control group that enters the colleges’ traditional developmental and college-level math sequence. Differences in later outcomes can then be attributed to this difference in type of developmental math course sequence.

Students will be followed for at least four semesters after their developmental math sequence assignment to learn about their subsequent performance in college. The outcomes of primary interest are completion of developmental and college-level math courses, average number of math credits earned, transfer to a four-year institution, and overall academic progress and course completion. This wide array of measures will give a more complete picture of students’ success in math, rather than of success only in passing the NMP courses.