A computational (AI) agent is autonomous to the extent it makes decisions in its encountered circumstances without direct intervention by human designers. By this definition, autonomous agents are here today: trading in financial markets, controlling vehicles, and performing a variety of other specific functions. Their scope is likely to be increased significantly in the near future, so it is imperative to develop a better understanding of the risks and benefits, and most importantly: new techniques for predicting and controlling their behavior. How can we make the world safe for autonomous agents? Our work in this area includes case studies on autonomous agents in the financial domain, as well as general techniques for specifying preferences and behaviors of AI agents.