This Expert Commentary by Crystal Hall, the second in the series, reflects on the critical “behavioral diagnosis and design” process that began the partnership with each agency (described in detail in Chapter 2 of the final report).

Project researchers engaged diligently in the process of problem definition and behavioral diagnosis (explained in Chapter 2 of the BIAS final report) before designing each BIAS intervention (illustrated in the accompanying figure).[1] This process is neither quick nor easy, but these steps are crucial prerequisites for an effective study design. More important, a rigorous practice of the diagnosis method could inform the generalizability and scalability of different intervention components studied in a particular context.

Yet there are tradeoffs to be considered when deciding when to engage in a full diagnosis and design process. In an ideal situation, diagnosis and design would be a seamless stage of the research process. Striving to understand the mechanics of a program or service from the perspective of the intended client may seem obvious, but it is often overlooked by researchers, front-line workers, or advocates. This lapse can cause inaccurate assumptions about the constraints and motivations of a subpopulation and ultimately lead to ineffective program design. The process of behavioral diagnosis and design is an excellent way to avoid such an outcome. As noted in Chapter 2 of the BIAS final report, this process is more powerful when coupled with insights gained from administrative data.

On the other hand, diagnosis and design can be time- and labor-intensive. Furthermore, it can be difficult to disentangle the relative impacts of various types of behavioral “bottlenecks.” If a client appears to be flustered by a complex form, avoids making a decision because of countless options, and feels that a process is unfair, it could be hard to tell which techniques to use. It can also be difficult to know what specific bottlenecks are exhibiting the greatest influence on the observed behavior.

Moving forward, I offer three suggestions as researchers and practitioners continue to expand the lessons learned from applied behavioral science and the technique of behavioral diagnosis and design. First, the process of diagnosis and design could be used more extensively as a tool to consider the scale and generalizability of interventions. When coupled with good administrative data, there is great potential to make educated comparisons from one context to another. For example, comparing the application to a benefits process in one state with that of a similar state could be more effective when using this process. Considering questions of scalability during the diagnosis process could also help determine the prioritization of scarce time and resources to explore various research questions.

Second, it could be powerful to shift the focus of behavioral diagnosis and design to the environment encountered by the staff that implement programs and interact with clients. In most cases, the process of diagnosis and design focuses on the client perspective — with good reason. However, designing an intervention for staff could have several benefits. This type of intervention could be easier to implement, in some cases, when training and subsequent tracking of behavior are easier. In addition, one intervention that touches a single case worker who interacts with dozens of clients could have a relatively large impact, under the right circumstances. There are concerns to be considered with respect to the rigor and statistical power of this type of design, but it is nonetheless compelling to consider — especially in the realm of the social policies and programs in the BIAS portfolio.

Finally, researchers and practitioners should work harder to integrate the use of administrative data in the process of behavioral diagnosis and design. As explained in Chapter 2 of the BIAS final report, data can play an invaluable role in this process, and there is still untapped potential in the way that data can be leveraged in the process of intervention design. The role of data will be even more useful to consider as researchers and practitioners work to address questions and behaviors that span government agencies and different decision domains.

The rigorous process of behavioral diagnosis and design should not be overlooked when assessing the broader contributions of the BIAS project. The lessons learned from this technique should continue to evolve and grow, so that we can most effectively leverage the tools offered by behavioral science.