AUTHOR=de Melo Celso M. , Marsella Stacy , Gratch Jonathan 
  
TITLE=Risk of Injury in Moral Dilemmas With Autonomous Vehicles
  
JOURNAL=Frontiers in Robotics and AI
  
VOLUME=Volume 7 - 2020
  
YEAR=2021
  
URL=https://www.frontiersin.org/journals/robotics-and-ai/articles/10.3389/frobt.2020.572529
  
DOI=10.3389/frobt.2020.572529
  
ISSN=2296-9144
  
ABSTRACT=As autonomous machines, such as automated vehicles (AVs) and robots, become pervasive in society, they will inevitably face moral dilemmas where they must make decisions that risk injuring humans. However, prior research has framed these dilemmas in starkly simple terms: i.e., framing decisions as life and death, neglecting the influence of risk of injury to involved parties on outcome. Here we focus on this gap and present experimental work that systematically studies the effect of risk of injury on the decisions people make in these dilemmas. In four experiments, participants were asked to program their AVs to either save five pedestrians -- which we refer to as the utilitarian choice -- or save the driver -- which we refer to as the non-utilitarian choice. The results indicate that most participants made the utilitarian choice, but that this choice was moderated by perceived risk to the driver and risk to the pedestrians. As a second contribution, we demonstrate the value of formulating AV moral dilemmas in a game-theoretic framework that considers the possible influence of others' behavior. In the fourth experiment, we show that participants were more (less) likely to make the utilitarian choice, the more utilitarian (non-utilitarian) others behaved; furthermore, unlike the game-theoretic prediction of non-utilitarianism, we found significant evidence of utilitarianism. We discuss theoretical implications for understanding of human decision making in moral dilemmas and practical guidelines for the design of autonomous machines that solve these dilemmas while, at the same time, being likely to be adopted in practice.