Limits of Perceived Audio-Visual Spatial Coherence as Defined by Reaction Time Measurements - Stenzel, Hanne; Francombe, Jon; Jackson, Philip J. B. (Frontiers in Neuroscience, 2019)
The ventriloquism effect describes the phenomenon of audio and visual signals with common features, such as a voice and a talking face merging perceptually into one percept even if they are spatially misaligned. The boundaries of the fusion of spatially misaligned stimuli are of interest for the design of multimedia products to ensure a perceptually satisfactory product. They have mainly been studied using continuous judgment scales and forced-choice measurement methods. These results vary greatly between different studies. The current experiment aims to evaluate audiovisual fusion using RT measurements as an indirect method of measurement to overcome these great variances. A two-alternative-forced choice (2AFC) word recognition test was designed and tested with noise and multi-talker-speech background distractors. Visual signals were presented centrally and audio signals were presented between 0° and 31° audio-visual offset in azimuth. RT data were analyzed separately for the underlying Simon effect and attentional effects. In the case of the attentional effects, three models were identified but no single model could explain the observed RTs for all participants so that data were grouped and analyzed accordingly. The results show that significant differences in RTs are measured from 5° to 10° onwards for the Simon effect. The attentional effect varied at the same audio-visual offset for two out of the three defined participant groups. In contrast with the current research, these results suggest that, even for speech signals, small audio-visual offsets influence spatial integration subconsciously.