Randomized experiments have become part of the standard toolkit for policy evaluation, and are usually designed to give precise estimates of causal effects. But, in practice, their actual goal is to pick good policies. These two goals are not the same.
Is this the best way to go about things? Can we maybe make better policy choices, with smaller experimental budgets, by doing things a little differently? This is the question that Anja Sautmann and I address in our new work on “Adaptive experiments for policy choice.” If we wish to pick good policies, we should run experiments adaptively, shifting toward better policies over time. This gives us the highest chance to pick the best policy after the experiment has concluded.