As the only nonacademic presenting a paper at this conference, I see it as my charge to focus on the challenge of implementing random assignment in the field. I will not spend time arguing for the methodological strengths of social experiments or advocating for more such field trials. Others have done so eloquently. But I will make my biases clear. For 25 years, I and many of my MDRC colleagues have fought to implement random assignment in diverse arenas and to show that this approach is feasible, ethical, uniquely convincing, and superior for answering certain questions. Our organization is widely credited with being one of the pioneers of this approach, and through its use producing results that are trusted across the political spectrum and that have made a powerful difference in social policy and research practice. So, I am a believer, but not, I hope, a blind one. I do not think that random assignment is a panacea or that it can address all the critical policy questions, or substitute for other types of analysis, or is always appropriate. But I do believe that it offers unique power in answering the “Does it make a difference?” question. With random assignment, you can know something with much greater certainty and, as a result, can more confidently separate fact from advocacy.

This paper focuses on implementing experiments. In laying out the ingredients of success, I argue that creative and flexible research design skills are essential, but that just as important are operational and political skills, applied both to marketing the experiment in the first place and to helping interpret and promote its findings down the line. Conducting a successful random assignment experiment in a complex, real-world context requires a continuing balancing of research ambition against operational realism. Lest this sound smug — because I am talking from an institutional track record of success in doing this — let me add that success remains an uphill battle. No one has ever welcomed random assignment. Moreover, this challenge has recently become more acute as research questions and programs become more complex, and the political and funding terrain more hostile.

My background theme is that this is a battle worth fighting. People who are active in public policy debates and who fund this type of research know the political and financial costs of evaluations that end in methodological disputes. Henry Aaron put this well in his influential book Politics and the Professors, which describes the relationship between scholarship and policy during the Great Society era and its aftermath. Pointing to the conservative effect on policymakers of disputes among experts, he asked: “What is an ordinary member of the tribe [that is, the public] to do when the witch doctors [the scientists and scholars] disagree?” He went further, arguing that such conflict not only paralyzes policy but also undercuts the “simple faiths” that often make action possible.

Random assignment, because of its unique methodological strengths, can help avoid this kind of conflict — what Aaron called “self-canceling research.” But random assignment studies must be used judiciously and interpreted carefully to assure that they meet ethical norms and that their findings are correctly understood. It is also important that researchers not oversell this technique. Random assignment can answer the important “Does it make a difference?” and “For whom?” questions, but it must be combined with other approaches to get answers to the critical question of “Why?” and “Under what conditions?”