Front. Hum. Neurosci.Frontiers in Human NeuroscienceFront. Hum. Neurosci.1662-5161Frontiers Media S.A.10.3389/fnhum.2024.1495592Human NeuroscienceOriginal ResearchBaseline dependent differences in the perception of changes in visuomotor delayVighGesche12*LimanowskiJakub231Faculty of Psychology, Technical University of Dresden, Dresden, Germany2Center for Tactile Internet With Human-in-the-Loop, Technical University of Dresden, Dresden, Germany3Institute of Psychology, University of Greifswald, Greifswald, Germany
Edited by: Hai-Ning Liang, The Hong Kong University of Science and Technology (Guangzhou), China
Reviewed by: Olga Mikhailovna Bazanova, Federal Research Center of Fundamental and Translational Medicine, Russia
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
Introduction
The detection of, and adaptation to delayed visual movement feedback has been extensively studied. One important open question is whether the Weber-Fechner Laws hold in the domain of visuomotor delay; i.e., whether the perception of changes in visuomotor delay depends on the amount of delay already present during movement.
Methods
To address this, we developed a virtual reality based, continuous hand movement task, during which participants had to detect changes in visuomotor mapping (delay): Participants (N = 40) performed continuous, auditory-paced grasping movements, which were measured with a data glove and transmitted to a virtual hand model. The movements of the virtual hand were delayed between 0 and 700 ms with the delay changing repeatedly in a roving oddball design. Participants had to indicate any perceived delay changes by key press. This design allowed us to investigate detection accuracy and speed related to the magnitude of the delay change, and to the “baseline” delay present during movement, respectively.
Results
As expected, larger delay changes were detected more accurately than smaller ones. Surprisingly, delay changes were detected more accurately and faster when participants moved under large > small delays.
Discussion
These results suggest that visual movement feedback delay indeed affects the detection of changes in visuomotor delay, but not as predicted by the Weber-Fechner Laws. Instead, bodily action under small delays may have entailed a larger tolerance for delay changes due to embodiment-related intersensory conflict attenuation; whereas better change detection at large delays may have resulted from their (visual) saliency due to a strong violation of visuomotor predictions.
actiondelayvirtual realityvisuomotor mappingWeber-Fechner-lawEXC 2050/1390696704AZ 97-932German Research Foundation (DFG, Deutsche Forschungsgemeinschaft)Cluster of Excellence “Centre for Tactile Internet with Human-in-the-Loop”VolkswagenStiftung10.13039/501100001663section-at-acceptanceCognitive NeuroscienceIntroduction
For much of our usual sensory input, like sound, light and weight, perceptibility of a change is proportional to the stimuli perceived before the change. Thus, the larger an initial stimulus is, the larger the change needs to be for the resulting stimulus to be perceived as different. This relationship is described by the Weber-Fechner Laws (Fechner et al., 1966), which have been confirmed across different sensory modalities (Baird, 1997; Gescheider, 1997). Recent empirical evidence suggests that the Weber-Fechner Laws may also apply to more abstract domains like time and reward perception (Brannon et al., 2008; Namboodiri et al., 2014; Takahashi, 2005, 2006); Nieder and Miller (2003).
The Weber-Fechner-Laws have also been evaluated in the domain of bodily perception. Interestingly, studies investigating self-motion perception found that the relationship between stimulus intensity and change perception was better described by power laws than the Weber-Fechner-Laws (Mallery et al., 2010; Nesti et al., 2015; Nouri and Karmali, 2018). Carriot et al. (2021) found improved discrimination performance at higher stimulus levels (of self-motion), rather than nonlinearities predicted by the Weber-Fechner-Laws. Furthermore, Ganel et al. (2008) showed that just noticeable differences depended on object size only in perceptual tasks, not in visuall guided grasping tasks—concluding that the visual coding for action does not follow the Weber-Fechner-Laws. Thus, it is possible that the processing of stimuli related to bodily action and self-motion may follow different perceptual laws than the perception of external stimuli (Carriot et al., 2021). One key question that, to our knowledge, has not yet been addressed is whether the Weber-Fechner Laws hold for the perception of visuomotor delays.
The flexible control of bodily action by the brain relies on the capacity to adapt internal models for motor control to novel sensory movement feedback (Limanowski, 2022; Shadmehr et al., 2010; Wolpert and Kawato, 1998). Sensorimotor adaptation is necessary in cases where, e.g., visual movement feedback is delayed, i.e., a temporal incongruence is introduced between movements and their visual feedback (Foulkes and Miall, 2000; Rohde and Ernst, 2016). Sensorimotor delays result ‘naturally’ from conduction delays within our nervous system, thus posing a problem for the brain, particularly regarding the on-line control of action using (delayed) sensory feedback. It is generally agreed upon that the brain overcomes such delays through internal forward modeling of the sensory movement consequences (Miall and Wolpert, 1996; Todorov and Jordan, 2002). But sensorimotor delays are also increasingly encountered in new technologies enabling cyber-physical interactions, such as virtual reality or robotics (Peters et al., 2018; Schulze et al., 2024). In most of these scenarios, sensorimotor delays are unavoidable (e.g., due to limitations in bandwidth or computing power) and, more importantly, they can vary unpredictably (Planthaber et al., 2018).
Many studies have investigated the perceptual thresholds for detecting visuomotor delays, the capacity to adapt to them, and the neuronal underpinnings of these processes (Farrer et al., 2008; Foulkes and Miall, 2000; Krugwasser et al., 2019; Leube et al., 2003; Limanowski et al., 2017; Miall et al., 1985; Miall and Jackson, 2006; Parvin et al., 2024; Rohde and Ernst, 2016; Tsakiris et al., 2006). However, to our knowledge, it has not yet been addressed is whether the perceptual sensitivity to visuomotor delay changes depends on the amount of delay present during movement, as predicted by the Weber-Fechner-Laws.
Therefore, we addressed this question through a virtual reality based hand movement task: Participants performed continuous right-hand grasping movements, paced by an auditory rhythm. Their movements were fed to a photorealistic virtual hand model presented on screen, with an experimentally added delay that changed in a roving oddball fashion (Figure 1). Participants had to report any perceived delay changes (i.e., changes in the mapping of their real hand movements to those executed by the virtual hand) by key presses, as fast as possible throughout the movement task. We expected larger delay changes to be detected more accurately and faster. Furthermore, following the Weber-Fechner Laws, we expected that participants would detect delays better and quicker if they were moving under relatively smaller delays.
Experimental task and design. Participants controlled the movements of a photorealistic virtual hand presented on screen (A) via their hand movements measured by a data glove worn on their right hand (occluded from view, B). (C) Participants had to execute continuous grasping (open-and-close) movements paced by an auditory cue, a tone changing volume with a frequency of 0.5 Hz (i.e., periodically growing louder and quieter). Flexion data from the glove were fed to the virtual hand model, which displayed the executed finger movements. During the task, we added various amounts of delay to the virtual hand movements. This altered the visuomotor mapping; i.e., the temporal congruence between the participants’ executed hand movements and the movements of the virtual hand (i.e., the visual movement feedback). The schematic plot shows the real hand closing and opening following the auditory cue’s volume, with a delay added to the virtual hand movements. (D) During the continuous open-and-close movement task, we repeatedly changed the amount of visual feedback delay in a roving oddball design. This plot shows an example sequence of delays between 0 and 700 ms as used in the experiment (i.e., one run of 4.9 min = 147 open-and-close movement cycles à 2 s each). Participants had to report any perceived changes in the visuomotor delay by pressing a button with their left hand as fast as possible (see B).
MethodsParticipants
We recruited 40 healthy, right-handed participants, with normal or corrected-to-normal vision (15 female, mean age = 28.1, range = 22–38). We chose our sample size roughly following previous work investigating sensorimotor conflicts with continuous movement paradigms (Charalampaki et al., 2024; Dewey, 2023; Limanowski and Friston, 2020; Haering and Kiesel, 2016; Asai, 2015; Salomon et al., 2013). Participants were recruited through undergraduate psychology lectures and seminars and social media. We excluded three participants (one because the data glove could not be adequately fitted and calibrated; one participant was not able to perceive the changes in volume of the tracking sound; and one did not perceive any delay of the virtual hand, even for a delay of 700 ms), leaving us with 37 participants whose data could be analyzed. The experiment was approved by the ethics committee of the Technische Universität Dresden and conducted in accordance with this approval.
Experimental procedure
First, participants were briefed about the experimental procedure and completed a demographic questionnaire. Then they put on a sanitary glove (for hygienic reasons) and the data glove on their right hand (5Dt Data Glove Ultra, 14 sensors, 10 bit flexure resolution per sensor, 60 Hz sampling rate, communication with the PC via USB). The data glove allowed our participants to control a photorealistic virtual hand (Figure 1B; modelled in blender 2.79, https://www.blender.org/), by measuring the flexion of the participants’ fingers and transmitting these values to the virtual hand model’s fingers (Figures 1A–C). The virtual hand was shown on a monitor in front of the participant (1,920 × 1,080 pixels resolution, screen distance to eye about 90 cm, 18.92° of visual angle, 13 ms Motion Picture Response Time). Participants placed their right hand on their leg, with the palm facing upward, in a position that allowed for comfortable opening and closing of the hand (see Figure 1A). Their right hand was occluded from view by a barber coat. The data glove was then calibrated; i.e., the virtual fingers’ movement range was dynamically adjusted to the sensor values until it plausibly displayed the movements of the real hand.
Participants were instructed to fixate a dot presented centrally on screen, and visible at all times, for the entirety of the experiment (Figure 1B). The movement task was to execute simple, continuous grasping movements: Participants opened and closed their right hand (starting from a closed position) paced by a rhythmic auditory volume cue; i.e., a 250 Hz tone that grew louder and quieter following a sine wave function with 0.5 Hz frequency. When the tone was loudest, the physical hand of the participant was supposed to be open, when it was quietest the hand was supposed to be closed (Figure 1C).
Before the actual experiment, participants practiced to align their real hand grasping movement with the auditory rhythm (without added delay to the virtual hand) until they felt confident that they could follow the auditory cue with their hand movements continually throughout the main experiment.
In the first part of the experiment, we tested for potential differences in the perceived difficulty of the hand movement task itself across the different delay levels used in the main experiment. Participants had to perform 10 auditory paced grasping movements (20 s) while fixating the dot in front of the virtual hand, at each delay level used in the main experiment—and rate the task difficulty (see below). I.e., the virtual hand moved either synchronously (0 ms delay) or was delayed by 100, 200, 300, 400, 500, 600, or 700 ms. We chose relatively long delays for our experiment compared with previous single-trial delay detection tasks (Leube et al., 2003; Limanowski et al., 2017; Limanowski et al., 2020), because our pilot experiments indicated that delay perception was more difficult with our task setup, likely because it required simultaneously maintaining a continuous synchronization of the movements with the pacing rhythm. Each delay level was presented three times, in randomized order, resulting in 24 trials in total. After each trial, we asked our participants to indicate how difficult they felt it was to follow the auditory cue, in comparison to the synchronous practice trial, on a 5-point Likert scale from “very easy” to “very difficult.”
In the second, main part of the experiment, participants also executed continuous, paced grasping movements—now with an added delay to the virtual hand movements that changed repeatedly, following a roving oddball paradigm (Figure 1D). Participants put their left index finger on the space bar of a computer keyboard, and were instructed to press it as fast as possible every time they detected a change in the visual movement delay (i.e., a change in the mapping of the virtual hand movements to their actually executed movements). Participants were specifically instructed and trained to maintain a stable grasping rhythm with their real hand; i.e., to not alter their movements in response to the delays or delay changes.
During the experiment, each of 8 “baseline” delay levels (ranging from 0 ms, i.e., synchronous to 700 ms in steps of 100 ms, see above) was presented for 3 to 6 movement cycles (i.e., 6–12 s), before changing to a larger or smaller delay. The delays changed between ±100 and 400 ms (in steps of 100 ms), whereby the changes were introduced gradually; i.e., over the course of 0.5 s to prevent sudden jumps in the visual movement. The delays were presented in runs of 147 movements (4.9 min), each of which contained a predefined, pseudorandomized sequence of 32 delay changes. Thus, we ensured that every run contained 8 delays with each length (3, 4, 5, and 6 movement cycles); and that, within each run, each baseline delay level was presented once in combination with each absolute delay change (i.e., ±100–400 ms; 8 delays × 4 changes = 32 delay changes). We focused our analysis on the absolute magnitude of delay changes (±100–400 ms) because we could not systematically compare relative, i.e., directional delay changes (in- vs. decreases) at all levels. In other words, due to the nature of the design, it was not possible to include all combinations of baseline delay and delay changes in- and decreases. For instance, at 700 ms, the delay could only decrease, not increase (i.e., there was no change of 700 + 100 ms, only 700–100 ms). As a control analysis, we tested for potential differences between delay in- vs. decreases with a repeated measures ANOVA with the factors delay change (100–400) and direction of change (in- vs. decrease) on the detection scores; which showed that detection was not significantly different between delay in- vs. decreases [F(3,108) = 0.09, p = .77]. An example of a roving oddball delay sequence can be seen in Figure 1D. Overall, eight such sequences were generated, and presented to each participant in randomized order; resulting in 256 delay changes per participant in total. After each run, participants could take a break until they felt ready for the next one.
Data analysis
To analyze delay change detection performance, we defined a correct delay detection (“hit”) as a button press within two movement cycles (4 s) of a delay change. We chose to include the second movement cycle, because each delay change was gradually introduced (see above), leaving only a partial movement cycle for “detection” otherwise. For consistency, we also tested whether our results could be replicated with detection windows of one and three movement cycles. Button presses outside of this window were categorized as false alarms. If multiple button presses were recorded within 250 ms of a hit or a false alarm [which we (Jain et al., 2015)], they were counted as a single button press.
To quantify the overall delay change detection accuracy, we first calculated the False Alarm Ratio as the number of false alarms per number of total alarms (i.e., button presses; Barnes et al., 2007). Note that this is different from the often used False Alarm Rate (also called False Positive Rate), which is defined as the number of false alarms per total number of “non-events.” In our case, we could not clearly define “non-events,” as this would apply to any time point not corresponding to a delay change, i.e., to most of the actual movement period.
Then, we calculated a “Detection Score” as a more conservative index of detection performance, punishing the percentage of hits by the False Alarm Ratio. Thus, participants who were more hesitant and thus more accurate, but had lower hits overall, would still receive an adequate score as compared to participants that hit the button at a near random rate. We calculated the detection score as follows:
detectionscore=PercentageofHits1+FAR
Detection scores were calculated for each of the 32 delay changes (see above). We then inserted the detection scores into a 4 × 8 repeated measures ANOVA, with the factors Delay Change (100, 200, 300, and 400 ms) and Baseline Delay (0, 100, 200, 300, 400, 500, 600, 700, and 800 ms). To test for non-normal properties, we used the Shapiro test. Since our data showed non-normal properties, we used an aligned ranks transformation ANOVA for our analysis. The degrees of freedom were calculated using the Kenward-Roger approximation. We also tested for outliers, defined as values of more than three times the interquartile range above the third quartile or below the first quartile; no such extreme values were found. Furthermore, we tested for sphericity using Mauchly’s Test. Since sphericity may be violated for delays, we used Greenhouse–Geisser corrections for our results. When determining the significance of our results, we corrected for multiple comparisons using the Holm-Bonferroni method. Post-hoc, we used a linear regression model (method of least squares) to further evaluate the main effects; i.e., in terms of consistency and directionality.
As an index of tracking performance, we calculated the auditory cue tracking error as the averaged absolute phase shift in time between the normalized mean movement of the participant and the target movement. To test whether tracking performance (i.e., how well participants were able to maintain an accurate and stable grasping rhythm) was influenced by detection performance, we calculated Spearman’s rank correlation coefficients for tracking errors vs. the detection scores and vs. the reaction times, respectively, averaged over delay levels.
Results
A repeated measures aligned ranks transformation ANOVA on the difficulty ratings given in the first part of the experiment (i.e., the reported difficulty of tracking the auditory target phase with the grasping movements) showed no significant effect of delay on tracking difficulty [F(7,885) = 0.64, p = .72, for the average difficulty ratings of each delay level see Table 1]. The difficulty ratings for all delay levels were unaffected by order effects (paired t-tests on ratings given in the first vs. last third of the rating blocks, all n.s.). Together, this means that participants found the paced movement task itself comparably difficult across all delay levels.
Means and standard deviations for difficulty ratings, tracking error, detection score and reaction times, for all delay levels.
Delay level
0 ms
100 ms
200 ms
300 ms
400 ms
500 ms
600 ms
700 ms
Difficulty rating [1–5, a.u.]
2.58 ± 1.15
2.51 ± 1.20
2.44 ± 1.08
2.51 ± 1.17
2.53 ± 1.01
2.33 ± 1.04
2.50 ± 1.17
2.59 ± 1.12
Tracking error [s]
0.23 ± 0.17
0.22 ± 0.18
0.24 ± 0.18
0.23 ± 0.18
0.23 ± 0.18
0.27 ± 0.18
0.27 ± 0.18
0.24 ± 0.18
Detection score [a.u.]
0.08 ± 0.04
0.1 ± 0.03
0.12 ± 0.04
0.12 ± 0.06
0.17 ± 0.08
0.14 ± 0.02
0.16 ± 0.05
0.17 ± 0.04
Reaction time [s]
2.14 ± 0.18
2.04 ± 0.19
2.11 ± 0.27
2.06 ± 0.20
2.02 ± 0.14
1.97 ± 0.30
1.82 ± 0.20
1.85 ± 0.30
In the main experiment, participants were able to maintain regular grasping and tracked the auditory cue well and comparably across delay levels (Figure 2). On average, they slightly lead the phase of the auditory cue, with an average absolute tracking error of 0.24 ± 0.15 s. Importantly, however, tracking error did not significantly differ between delay levels [F(7,295) = 0.23, p = .98, see Table 1]. Together, these results suggest that our participants were well able to comply with the task instructions, and that the amount of delay of the virtual hand itself did not significantly influence tracking (grasping) performance, nor the perceived task difficulty.
Participants’ average hand movements per delay level (with standard deviations in gray shading), relative to the instructed rhythm prescribed by the auditory cue. Participants tracked the auditory target rhythm comparably well across all delay levels, albeit overall leading slightly (see Results for details). Thus, visual delay per se did not bias grasping or tracking performance.
To validate the overall delay change detection accuracy of our participants, we first tested it against the chance level. The average False Alarm Ratio of our participants was 0.45 ± 0.10, which was significantly lower than the ratio that would have resulted from random button pressing (i.e., >0.67; t(36) = 13.13, p < .001). This suggests that, overall, the participants were able to notice the delay changes and pressed the button accordingly.
An aligned ranks transformation ANOVA on the detection scores showed a significant main effect of Delay Change; i.e., the magnitude of the change in the delay of the visual hand movement (F(3,1,116) = 19.48, p < .001, ƞ2 = .05). A post-hoc linear regression showed that these effects could be linearly approximated (Figure 3A, R2 = .07, Slope = 2.58∙10−4, df = 35, p < .05; for average detection scores see Table 2). However, there was no significant effect of Delay Change on the corresponding reaction times (Figure 3B, for means and standard deviations see table 2), while the interaction effect was significant [F(21,1,116) = 3.505, p < .001, ƞ2 = .06]. In sum, as expected, larger delay changes were consistently detected better (albeit not significantly faster) than smaller ones.
Accuracy and speed of detecting changes in visual movement feedback delay during continuous grasping. The left plots show the average detection scores (A) and reaction times (B) for each of the 32 combinations of delay change magnitude and baseline delay. Detection accuracy as quantified by the Detection Score was significantly affected by the amount of delay present during movement (main effect Baseline Delay, p < .001) and the magnitude of the change in delay (main effect Delay Change, p < .001). Reaction times were significantly affected by Baseline Delay (main effect, p < .001). The right plots show post-hoc linear regression analyses (see Results for details). Overall, these analyses showed that larger delay changes were detected significantly more accurately (and slightly but not significantly faster) than smaller changes. Furthermore, change detection was significantly better and faster when participants were moving at larger, compared to smaller, delays.
Means and standard deviations for detection scores and reaction times, for the four different absolute magnitudes of delay change, respectively.
Delay change (absolute)
100 ms
200 ms
300 ms
400 ms
Detection score [a.u.]
0.10 ± 0.03
0.11 ± 0.03
0.15 ± 0.06
0.16 ± 0.04
Reaction time [s]
2.07 ± 1.23
1.81 ± 0.84
2.04 ± 0.76
2.01 ± 0.77
Secondly, there was a significant main effect of Baseline Delay on detection scores [F(7,1,116) = 11.76, p < .001, ƞ2 = .07] and reaction times [F(7,1,116) = 6.56, p < .001, ƞ2 = .04]; as well as a significant interaction between Baseline Delay and Delay Change [F(21,1,116) = 2.09, p < .05, ƞ2 = .04 Figure 3B]. Post-hoc linear regression analyses showed, again, that these effects could be linearly approximated (detection scores, Figure 3A: R2 = .06, Slope = 1.18∙10−4, df = 35, p < .001; reaction times, Figure 3B: R2 = .02, Slope = −4.05∙10−4, df = 35, p < .05, for means see Table 1). In sum, delay change detection was overall better and faster at larger > smaller delays; i.e., the more the visual movement feedback was delayed during movement when the change occurred.
The above significant main and interaction effects on detection performance were also significant when limiting the analysis to a detection window of only the first movement cycle (all Fs > 2.2, all ps < 0.01) or the first three movement cycles (all Fs > 4.3, all ps < 0.001). The significant main effect of Baseline Delay on the reaction times, and the interaction effect, were also replicated when limiting the analysis to the first movement cycle (all Fs > 3.54, all ps < 0.001) or the first three movement cycles (all Fs > 1.55, albeit not reaching statistical significance, ps < 0.06). The slopes of the respective linear regressions confirmed the directionality of the effects as in our main analysis. In short, the width of our detection window did not bias our results.
Finally, we tested whether the participants’ average detection performance covaried with how well they were able to maintain stable tracking. With this, we aimed to detect a possible relationship between delay detection and tracking error despite the fact that tracking performance on average was comparable across delay levels (cf. Figure 2). This analysis revealed a significant negative correlation between detection accuracy and tracking performance: Participants who tracked the target rhythm overall better (i.e., had a smaller average tracking error) detected less of the delay changes (ρ = .40, p < .05, Figure 4). The correlation between reaction times and tracking error showed a similar directionality, but was not significant (smaller tracking error ~ longer reaction times to delay changes; ρ = −.05, p = .78).
Spearman correlation between detection score and tracking error of the individual participants. Participants with better detection scores had a larger tracking error; i.e., they followed the auditory target cue worse with their grasps.
Discussion
Using a virtual reality based continuous hand movement task, we tested whether the detection of changes in visuomotor delay depended on the amount of delay currently present during movement. As expected, we found that larger delay changes were detected significantly better than smaller ones. Somewhat surprisingly, however, delay changes were detected significantly better and faster when participants moved under larger > smaller delays. In other words, our results suggest a higher sensitivity to changes in visual movement feedback delay when there already is a substantial amount of delay present during movement. This seems to indicate that the Weber-Fechner Laws do not apply to the perception of (temporal) visuomotor mapping; as they would predict the opposite; i.e., better sensitivity to delay changes under smaller delays.
Results inconsistent with the Weber-Fechner Laws have been reported in other domains before, where they have been interpreted as indicating, for instance, saturation effects (Augustin, 2008; Augustin and Tanja, 2008; Carriot et al., 2021; Doble et al., 2003). In our case, we propose two complementary explanations for our results:
Firstly, it is well established that increasing visuomotor delay reliably reduces the self-attribution of the observed movements (also called a loss of control or “agency”) over those movements (Krugwasser et al., 2019; Wen and Imamizu, 2022) and the subjective embodiment of the seen moving body part (also called a loss of “ownership”; see (Dummer et al., 2009; Farrer et al., 2008; Leube et al., 2003; Tsakiris et al., 2006). Thus, it is reasonable to assume that, in our experiment, participants experienced a relatively stronger subjective embodiment of the virtual hand under small delays. After our experiment, seven of our participants spontaneously reported having perceived up to half of all virtual movements as synchronous to their own movement (i.e., much more than actually were synchronous)—several of those participants reported a feeling of ownership over the virtual hand in those synchronously perceived periods. While these were only spontaneous post-hoc reports, they tentatively support the idea of embodiment biasing delay detection in our task.
Experienced embodiment can substantially affect attentional control and task performance in virtual reality based movement tasks, as shown, e.g., by Iwasaki et al. (2022). In our case, one such effect could be intersensory conflict attenuation. As demonstrated in the ‘rubber hand illusion’ (Botvinick and Cohen, 1998), the embodiment of an alternative (fake or virtual) limb biases the subjective perception of visuoproprioceptive mismatches between real and fake limbs, attenuating intersensory conflict through updates to the corresponding neuronal representations (Limanowski, 2022). Since visuomotor delays are, effectively, also visuoproprioceptive mismatches, we speculate that a similar intersensory conflict attenuation could occur at smaller, compared with larger delays. In other words, an increased embodiment of the virtual hand at smaller delay levels could have increased the tolerance for visuomotor mismatches, and thus, impaired delay detection performance. An increased embodiment of the virtual hand under small > large delays could also have distorted the perception of movement timing more generally (Desantis et al., 2016; Haering and Kiesel, 2015; Rohde and Ernst, 2016).
A complementary influence on detection performance could have resulted from larger visuomotor delays being more salient stimuli per se. This saliency effect could stem from the fact that these delays constitute ‘non-standard’ visuomotor mappings; i.e., mappings that violate the life-long learned associations (of congruence) between motor commands and visual movement feedback (Quirmbach and Limanowski, 2024; Yon et al., 2018). Thus, larger delays could have captured more attention—onto the visuomotor relationship, or more generally onto the visual movement feedback—than smaller delays. This would likely have improved delay detection performance at larger delay levels. Note that this interpretation complements, rather than contradicting, the idea of an embodiment-related increased tolerance for changes at lower delay levels. However, as we did not explicitly assess subjective embodiment or attentional allocation, both of the above interpretations have to be explicitly tested in future experiments.
Finally, note that our task design implied maintaining focus on the real hand movements, to track the auditory rhythm, while the virtual hand moved incongruently. Visual body movements that are incongruent with one’s actual movements and movement goals can capture attention and lead to behavioral deficits—this is sometimes also referred to as visuomotor interference (Blakemore and Frith, 2005; Brass et al., 2001; Kilner et al., 2003; Limanowski and Friston, 2020; Metral and Guerraz, 2019). The distracting effect of incongruent visual stimuli may be particularly strong when they depict one’s own, delayed movements (Salomon et al., 2013). The positive correlation between detection performance and tracking error suggests that participants who used more cognitive-attentional resources to maintain stable tracking lacked those resources for the detection task. The ‘dual task’ nature of our design could also explain the relatively low overall detection performance we observed, in contrast to other studies using single-movement designs (Farrer et al., 2008; Leube et al., 2003). Interestingly, tracking performance did not differ between delay levels. This could suggest that our participants resisted or counter-acted any potential biasing effect of visual feedback delay on movement execution (cf. Limanowski and Friston, 2020). An interesting question (which we, however, cannot answer) is whether an increased embodiment of the virtual hand would lead to stronger visuomotor interference effects (hence expected at smaller delays); or whether larger delays per se elicit stronger visuomotor interference effects.
Our study raised further questions that should be followed up by future work. Due to the nature of our design, we could not systematically compare detection and reaction times to delay in- vs. decreases. Although in a control analysis, we did not find any significant differences in detection scores between delay in- vs. decreases (see Methods), this needs to be addressed in detail by future work. Furthermore, it has been shown that, when moving under visuomotor delay, visual movement feedback is processed differently depending on whether vision is currently task-relevant or a distractor (Limanowski et al., 2020; Limanowski and Friston, 2020). An open question is whether visuomotor delay changes are processed differently depending on whether they are task-relevant or -irrelevant. Furthermore, we observed substantial performance differences (in tracking and detection) between participants; future work should look into the possible causes for such inter- but also intrapersonal variability. For instance, one factor influencing overall performance could have been prior experience with VR scenarios, which we did not assess. Visuomotor temporal incongruence can in principle also be implemented by leading, not lagging, vision; in the context of self-initiated hand movements, however, this is associated with a number of problems related to the predictability of the visual trajectories before actual movement. Finally, visual feedback delay is only one kind of visuomotor incongruence; future work should test whether other manipulations, such as spatial offset or gain scaling, produce similar results.
To conclude, we have shown that, during bodily action in virtual reality, the current degree of visuomotor delay can bias the perception of changes in the magnitude of this delay. This bias does not seem to follow the Weber-Fechner-Laws. Instead, bodily action under relatively small delays may entail some degree of tolerance for delay changes resulting from intersensory conflict attenuation due to a stronger embodiment of the (relatively more congruently moving) virtual hand; whereas large delays may capture (visual) attention due to their violation of visuomotor predictions. Thus, our results highlight the importance of cognitive-attentional factors in visuomotor processing; and the need to consider these factors when designing cyber-physical interactions with variable delays.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving humans were approved by ethics committee of the Technische Universität Dresden (SR-EK-174032021). The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.
The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This work was funded by the German Research Foundation (DFG, Deutsche Forschungsgemeinschaft) as part of Germany’s Excellence Strategy—EXC 2050/1—Project ID 390696704—Cluster of Excellence “Centre for Tactile Internet with Human-in-the-Loop” (CeTI) of Technische Universität Dresden. JL was supported by a Freigeist Fellowship of the VolkswagenStiftung (AZ 97-932).
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
ReferencesAsaiT. (2015). Feedback control of one’s own action: self-other sensory attribution in motor control. Conscious. Cogn.38, 118–129. doi: 10.1016/j.concog.2015.11.002, PMID: 26587957AugustinT. (2008). The parameters in the near-miss-to-Weber’s law. J. Math. Psychol.52, 37–47. doi: 10.1016/j.jmp.2007.11.001AugustinT.TanjaR. (2008). Empirical evaluation of the near-miss-to-Weber’s law: a visual discrimination experiment. Psychology Science Quarterly, 50.BairdJ. C. (1997). Sensation and judgment: Complementarity theory of psychophysics. Lawrence Erlbaum Associates. Available at: https://search.ebscohost.com/login.aspx?direct=true&scope=site&db=nlebk&db=nlabk&AN=707812 (accessed April 29, 2024).BarnesL. R.GruntfestE. C.HaydenM. H.SchultzD. M.BenightC. (2007). False alarms and close calls: a conceptual model of warning accuracy. Weather Forecast.22, 1140–1147. doi: 10.1175/WAF1031.1BlakemoreS.-J.FrithC. (2005). The role of motor contagion in the prediction of action. Neuropsychologia43, 260–267. doi: 10.1016/j.neuropsychologia.2004.11.012, PMID: 15707910BotvinickM.CohenJ. (1998). Rubber hands ‘feel’ touch that eyes see. Nature391:756. doi: 10.1038/35784, PMID: 9486643BrannonE. M.LibertusM. E.MeckW. H.WoldorffM. G. (2008). Electrophysiological measures of time processing in infant and adult brains: Weber’s law holds. J. Cogn. Neurosci.20, 193–203. doi: 10.1162/jocn.2008.20016, PMID: 18275328BrassM.BekkeringH.PrinzW. (2001). Movement observation affects movement execution in a simple response task. Acta Psychol.106, 3–22. doi: 10.1016/S0001-6918(00)00024-X, PMID: 11256338CarriotJ.CullenK.ChacronM. (2021). The neural basis for violations of Weber’s law in self-motion perception. Proc. Natl. Acad. Sci.118:e2025061118. doi: 10.1073/pnas.2025061118, PMID: 34475203CharalampakiA.CistonA. B.FilevichE. (2024). Contributions of tactile information to the sense of agency and its metacognitive representations. J. Exp. Psychol. Gen.153, 2427–2440. doi: 10.1037/xge0001634, PMID: 39115900DesantisA.WaszakF.MoutsopoulouK.HaggardP. (2016). How action structures time: about the perceived temporal order of action and predicted outcomes. Cognition146, 100–109. doi: 10.1016/j.cognition.2015.08.011, PMID: 26409246DeweyJ. A. (2023). Cognitive load decreases the sense of agency during continuous action. Acta Psychol.233:103824. doi: 10.1016/j.actpsy.2022.103824, PMID: 36623472DobleC. W.FalmagneJ.-C.BergB. G. (2003). Recasting (the near-miss to) Weber’s law. Psychol. Rev.110, 365–375. doi: 10.1037/0033-295X.110.2.365, PMID: 12747528DummerT.Picot-AnnandA.NealT.MooreC. (2009). Movement and the rubber hand illusion. Perception38, 271–280. doi: 10.1068/p5921, PMID: 19400435FarrerC.FreyS. H.Van HornJ. D.TunikE.TurkD.InatiS.. (2008). The angular gyrus computes action awareness representations. Cereb. Cortex18, 254–261. doi: 10.1093/cercor/bhm050, PMID: 17490989FechnerG. T.BoringE. G.HowesD. H.AdlerH. E. (1966). Elements of psychophysics. Volume 1/Gustav Fechner; translated by Helmut E. Adler; edited by David H. Howes, Edwin G. Boring; with an introduction by Edwin G. Boring. Holt, Rinehart and Winston, Inc.FoulkesA. J.MiallR. C. (2000). Adaptation to visual feedback delays in a human manual tracking task. Exp. Brain Res.131, 101–110. doi: 10.1007/s002219900286, PMID: 10759175GanelT.ChajutE.AlgomD. (2008). Visual coding for action violates fundamental psychophysical principles. Curr. Biol.18, R599–R601. doi: 10.1016/j.cub.2008.04.052, PMID: 18644333GescheiderG. A. (1997) in Psychophysics: The fundamentals. ed. MahwahN. J. (New Jersey, USA: L. Erlbaum Associates).HaeringC.KieselA. (2015). Was it me when it happened too early? Experience of delayed effects shapes sense of agency. Cognition136, 38–42. doi: 10.1016/j.cognition.2014.11.012, PMID: 25490127HaeringC.KieselA. (2016). Time perception and the experience of agency. Psychol. Res.80, 286–297. doi: 10.1007/s00426-015-0654-0, PMID: 25749800IwasakiY.NavarroB.IwataH.GaneshG. (2022). Embodiment modifies attention allotment for the benefit of dual task performance. Commun. Biol.5, 701–707. doi: 10.1038/s42003-022-03603-6, PMID: 35835983JainA.BansalR.KumarA.SinghK. D. (2015). A comparative study of visual and auditory reaction times on the basis of gender and physical activity levels of medical first year students. Int. J. Appl. Basic Med. Res.5, 124–127. doi: 10.4103/2229-516X.157168, PMID: 26097821KilnerJ. M.PaulignanY.BlakemoreS. J. (2003). An interference effect of observed biological movement on action. Curr. Biol.13, 522–525. doi: 10.1016/S0960-9822(03)00165-9, PMID: 12646137KrugwasserA. R.HarelE. V.SalomonR. (2019). The boundaries of the self: the sense of agency across different sensorimotor aspects. J. Vis.19:14. doi: 10.1167/19.4.14, PMID: 30952165LeubeD. T.KnoblichG.ErbM.KircherT. T. J. (2003). Observing one’s hand become anarchic: an fMRI study of action identification. Conscious. Cogn.12, 597–608. doi: 10.1016/S1053-8100(03)00079-5, PMID: 14656503LimanowskiJ.KirilinaE.BlankenburgF. (2017). Neuronal correlates of continuous manual tracking under varying visual movement feedback in a virtual reality environment. NeuroImage. 146, 81–89.LimanowskiJ. (2022). Precision control for a flexible body representation. Neurosci. Biobehav. Rev.134:104401. doi: 10.1016/j.neubiorev.2021.10.023, PMID: 34736884LimanowskiJ.FristonK. (2020). Attentional modulation of vision versus proprioception during action. Cereb. Cortex30, 1637–1648. doi: 10.1093/cercor/bhz192, PMID: 31670769LimanowskiJ.LitvakV.FristonK. (2020). Cortical beta oscillations reflect the contextual gating of visual action feedback. NeuroImage222:117267. doi: 10.1016/j.neuroimage.2020.117267, PMID: 32818621MalleryR. M.OlomuO. U.UchanskiR. M.MilitchinV. A.HullarT. E. (2010). Human discrimination of rotational velocities. Exp. Brain Res.204, 11–20. doi: 10.1007/s00221-010-2288-1, PMID: 20526711MetralM.GuerrazM. (2019). Fake hand in movement: visual motion cues from the rubber hand are processed for kinesthesia. Conscious. Cogn.73:102761. doi: 10.1016/j.concog.2019.05.009, PMID: 31200242MiallR. C.JacksonJ. K. (2006). Adaptation to visual feedback delays in manual tracking: evidence against the Smith predictor model of human visually guided action. Exp. Brain Res.172, 77–84. doi: 10.1007/s00221-005-0306-5, PMID: 16424978MiallR. C.WeirD. J.SteinJ. F. (1985). Visuomotor tracking with delayed visual feedback. Neuroscience16, 511–520. doi: 10.1016/0306-4522(85)90189-7, PMID: 4094689MiallR. C.WolpertD. M. (1996). Forward models for physiological motor control. Neural Netw.9, 1265–1279. doi: 10.1016/S0893-6080(96)00035-4, PMID: 12662535NamboodiriV. M. K.MihalasS.Hussain ShulerM. G. (2014). A temporal basis for Weber’s law in value perception. Front. Integr. Neurosci.8. doi: 10.3389/fnint.2014.00079, PMID: 25352791NestiA.BeykirchK. A.PrettoP.BülthoffH. H. (2015). Human discrimination of head-centred visual–inertial yaw rotations. Exp. Brain Res.233, 3553–3564. doi: 10.1007/s00221-015-4426-2, PMID: 26319547NiederA.MillerE. K. (2003). Coding of cognitive magnitude: compressed scaling of numerical information in the primate prefrontal cortex. Neuron37, 149–157. doi: 10.1016/S0896-6273(02)01144-3, PMID: 12526780NouriS.KarmaliF. (2018). Variability in the Vestibulo-ocular reflex and vestibular perception. Neuroscience393, 350–365. doi: 10.1016/j.neuroscience.2018.08.025, PMID: 30189227ParvinD. E.TsayJ.DangK. V.StoverA. R.IvryR. B.MoreheadJ. R. (2024). Implicit adaptation is modulated by the relevance of feedback. bioRxiv:2022.01.19.476924. doi: 10.1101/2022.01.19.476924PetersB. S.ArmijoP. R.KrauseC.ChoudhuryS. A.OleynikovD. (2018). Review of emerging surgical robotic technology. Surg. Endosc.32, 1636–1655. doi: 10.1007/s00464-018-6079-2, PMID: 29442240PlanthaberS.MallwitzM.KirchnerE. A. (2018). Immersive robot control in virtual reality to command robots in space missions. J. Softw. Eng. Appl.11, 341–347. doi: 10.4236/jsea.2018.117021QuirmbachF.LimanowskiJ. (2024). Visuomotor prediction during action planning in the human frontoparietal cortex and cerebellum. Cereb. Cortex34. doi: 10.1093/cercor/bhae382, PMID: 39325000RohdeM.ErnstM. O. (2016). Time, agency, and sensory feedback delays during action. Curr. Opin. Behav. Sci.8, 193–199. doi: 10.1016/j.cobeha.2016.02.029SalomonR.LimM.KannapeO.LloberaJ.BlankeO. (2013). “Self pop-out”: agency enhances self-recognition in visual search. Exp. Brain Res.228, 173–181. doi: 10.1007/s00221-013-3549-6, PMID: 23665753SchulzeA.BodenstedtS.DistlerM.WeitzJ.SpeidelS.WagnerM. (2024). Künstliche Intelligenz in der onkologischen Chirurgie. Die Onkol.30, 372–379. doi: 10.1007/s00761-024-01486-2ShadmehrR.SmithM. A.KrakauerJ. W. (2010). Error correction, sensory prediction, and adaptation in motor control. Annu. Rev. Neurosci.33, 89–108. doi: 10.1146/annurev-neuro-060909-153135, PMID: 20367317TakahashiT. (2005). Loss of self-control in intertemporal choice may be attributable to logarithmic time-perception. Med. Hypotheses65, 691–693. doi: 10.1016/j.mehy.2005.04.040, PMID: 15990243TakahashiT. (2006). Time-estimation error following weber-Fechner law may explain subadditive time-discounting. Med. Hypotheses67, 1372–1374. doi: 10.1016/j.mehy.2006.05.056, PMID: 16872753TodorovE.JordanM. I. (2002). Optimal feedback control as a theory of motor coordination. Nat. Neurosci.5, 1226–1235. doi: 10.1038/nn963, PMID: 12404008TsakirisM.PrabhuG.HaggardP. (2006). Having a body versus moving your body: how agency structures body-ownership. Conscious. Cogn.15, 423–432. doi: 10.1016/j.concog.2005.09.004, PMID: 16343947WenW.ImamizuH. (2022). The sense of agency in perception, behaviour and human–machine interactions. Nat. Rev. Psychol.1, 211–222. doi: 10.1038/s44159-022-00030-6WolpertD. M.KawatoM. (1998). Multiple paired forward and inverse models for motor control. Neural Netw.11, 1317–1329. doi: 10.1016/S0893-6080(98)00066-5, PMID: 12662752YonD.GilbertS. J.de LangeF. P.PressC. (2018). Action sharpens sensory representations of expected outcomes. Nat. Commun.9:4288. doi: 10.1038/s41467-018-06752-7, PMID: 30327503