Behavioral Cues of Humanness in Complex Environments: How People Engage With Human and Artificially Intelligent Agents in a Multiplayer Videogame - Tulk Jesso, Stephanie; Kennedy, William G.; Wiese, Eva (Frontiers in Robotics and AI, 2020)
The development of AI that can socially engage with humans is exciting to imagine, but such ad-vanced algorithms might prove harmful if people are no longer able to detect when they are inter-acting with non-humans in online environments. Because we cannot fully predict how socially intelligent AI will be applied, it is important to conduct research into how sensitive humans are to behaviors of humans compared to those produced by AI. This paper presents results from a behavioral Turing Test, in which participants interacted with a human, or a simple or “social” AI within a complex videogame environment. Participants (66 total) played an open world, interactive video-game with one of these co-players and were instructed that they could interact non-verbally however they desired for 30 minutes, after which time they would indicate their beliefs about the agent, including three Likert measures of how much participants trusted and liked the co-player, the extent to which they perceived them as a “real person”, and an interview about the overall perception and what cues participants used to determine humanness. T-tests, Analysis of Variance and Tukey’s HSD was used to analyze quantitative data, and Cohen’s Kappa and χ² was used to analyze inter-view data. Our results indicate suggest that it was difficult for participants to distinguish between humans and the social AI on the basis of behavior. An analysis of in-game behaviors, survey data and qualitative responses indicate suggest that participants associated engagement in social interactions with humanness within the game.