Intervention Timing and Treatment Effects Matter


By Lawrence F. Katz

In this Expert Commentary, the third in the series, Lawrence Katz addresses several key questions for future work raised by the findings from the BIAS project (discussed in Chapter 3 of the final report).

The BIAS team has done a remarkable job in a short period of designing and implementing 15 tests of low-cost behavioral interventions to improve the efficacy of key U.S. poverty alleviation policies using rigorous randomized controlled trials. I found several of the results of the behavioral interventions to be striking. The first is the larger impacts of outreach and reminders (in the BIAS interventions in Texas and Washington) for inducing one-time applications for child support order modifications among incarcerated noncustodial parents (that appear to be clearly in their interest) than for reminders trying to increase actual child support payments by those parents. It seems that behavioral nudges (defined as subtle and modest changes that help improve individual decision making), reminders, or information are not going to be sufficient to have large impacts on compliance when individuals don’t have the financial resources to comply or don’t view the required payments to be “legitimate.”

The second striking result is the importance of timing in behavioral information interventions that try to improve the active choices of program participants. In the child care provider choice intervention test for Indiana, an information intervention early in the process (before parents had already decided about child care providers) appears to have larger impacts in improving the selection of higher-quality-rated providers. In contrast, late information interventions seem to have limited impacts in other settings such as housing and neighborhood choices for Housing Choice Voucher recipients.[1] The same issues are likely important in school choice systems where simplified and personalized presentation of information about choices can have substantial impacts, but only when provided before choices are close to finalized.[2] And a third notable finding observed in the Los Angeles study that attempted to increase engagement in welfare-to-work activities is the power of messages that focus on loss as opposed to gain in exerting an impact on behavior. As the BIAS team notes, a sizable amount of evidence supports such approaches for prospect theory and loss aversion in earlier work in behavioral economics.[3]

The results from the BIAS project behavioral interventions raise several key questions for future work. The first is the extent to which behavioral nudges that increase program take-up or engagement also eventually improve important long-run outcomes such as child well-being and economic self-sufficiency of parents. Behavioral interventions with large impacts on program participation (such as the Texas and Washington interventions) to apply for child support order modifications could (for reasonable sample sizes) serve as first stages to learn about longer-run impacts on ultimate outcomes. Earlier behavioral interventions to help individuals apply for student financial aid for college by completing the Free Application for Federal Student Aid form have generated large enough impacts to analyze effects on actual college going and persistence.[4] Behavioral nudges with small (even if detectable) impacts on program participation are unlikely to be powerful enough to estimate impacts on economic and socioeconomic outcomes of ultimate interest.

A second key question is who are the “compliers” (marginal participants) who respond to the behavioral nudges but would not participate in the absence of such interventions? Are the marginal participants more disadvantaged than typical participants in the control groups? The behavioral view is that the most disadvantaged (and most needy) individuals face substantial “bandwidth” costs to dealing with complex program rules and compliance (“paperwork”) requirements. Behavioral interventions that help reduce these costs could expand participation on the margin for needier individuals.[5] The traditional view of neoclassical economics has been that hassle factors (or barriers to completing an action) lead to beneficial self-selection, with the neediest individuals having the greatest incentive to bear such costs and the less needy finding it not worth the effort and time.[6] An examination of the characteristics of the marginal compliers and the extent to which they are more or less advantaged and seem to make decisions by weighing the costs against the benefits in the 15 BIAS interventions could illuminate this debate.

A final issue for further research is how to combine low-cost behavioral nudges or information interventions as an initial approach to increase program compliance and then to move to more high-intensity efforts (such as personal counseling assistance) for remaining eligible individuals who don’t respond to the nudges. In other words, information interventions can help sort out those not needing more resource-intensive help to participate and engage in social programs.[7]

 

 

[1]Heather L. Schwartz, Kata Mihaly, and Breann Gala, “Encouraging Residential Moves to Opportunity Neighborhoods: An Experiment Testing Incentives Offered to Housing Voucher Recipients,” online document, Housing Policy Debate (2016): 1-31.

[2]Justine S. Hastings and Jeffrey M. Weinstein, “Information, School Choice, and Academic Achievement: Evidence from Two Experiments,” The Quarterly Journal of Economics 123, 4 (2016): 1373-1414.

[3]Stefano DellaVigna, “Psychology and Economics: Evidence from the Field,” Journal of Economic Literature 47, 2 (2009): 315-372.

[4]Eric P. Bettinger, Bridget Terry Long, Philip Oreopoulos, and Lisa Sanbonmatsu, “The Role of Application Assistance and Information in College Decisions: Results from the H&R Block FAFSA Experiment,” The Quarterly Journal of Economics 127, 3 (2012): 1205-1242.

[5]Marianne Bertrand, Sendhil Mullainathan, and Eldar Shafir, “A Behavioral-Economics View of Poverty,” American Economic Review 94, 2 (2004): 419-423.

[6]Albert L. Nichols and Richard J. Zechauser, “Targeting Transfers through Restrictions on Recipients.” The American Economic Review 72, 2 (1982): 372-377.

[7]An approach similar to this one is taken in the Paycheck Plus project described in Chapters 2 and 3 of the final BIAS report. See Lashawn Richburg-Hayes, Caitlin Anzelone, and Nadine Dechausay with Patrick Landers, Nudging Change in Human Services: Final Report of the Behavioral Interventions to Advance Self-Sufficiency (BIAS) Project, OPRE Report No. 2017-23 (Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, 2017), pp. 13-44.

Katz, Lawrence F.. 2017. “Intervention Timing and Treatment Effects Matter.” New York: MDRC.