Head-mounted displays augment what users can see by adding visual contents into their field of view, enabling them to access additional information while engaging in tasks such as productivity or casual interactions Cho et al. (2024). Given that users have limited amount of attentional resources, providing them with the right information at the right time is a crucial area of research. Within our work, we aim to provide AR contents that are beneficial but have minimal negative impact on users’ current task.

A number of techniques aim to minimize the impact of AR information on users’ current activities by adapting how to display AR content to users’ cognitive states. Prior studies minimize workloads required to access information, or reduce awareness of non-essential data. Mise-Unseen, for example, leverages moments when users turn their heads away, hiding changes in the visual scene during these transitions (Marwecki et al., 2019). Other works modify the information itself by varying the placement, level of detail (Lindlbauer et al., 2019) or the amount of information (Billinghurst and Starner, 1999; Rhodes, 1998) of AR content. Building on these adaptive approaches to minimize interruptions during tasks that users perform, our work extends to walking scenarios. Users have to also constantly observe the surroundings when viewing AR contents. We aim to deliver AR visual information that is adaptive to user’s walking activity.

During walking, we constantly redirect our visual focus to navigate through complex surroundings (Logan et al., 2010). In scenarios when another task is present, walking behavior is affected negatively, including reduced speed, cadence, stride time, and even risk of falling (Kao et al., 2023; Al-Yahya et al., 2011; Lim et al., 2017). Speed is the most common reported outcome measure of such interference (Al-Yahya et al., 2011; Grießbach et al., 2024). When presenting information in user’s field of view in mobile AR scenarios, it is important to examine and control how much attention contents draw, in order to minimize the impact of concurrent tasks on walking performance.

A moving object produces visual flows that tells our eyes about its motion (Barlow and Foldiak, 1989; Durgin et al., 2005). However, these signals can be distorted if we are moving (Durgin, 2009). Barlow’s theory suggests that the perceived object motion is influenced by both the object’s and our own motion (Barlow and Foldiak, 1989). When we are moving, object’s motion appears less apparent if our movement is in the same direction as the moving objects. At the same time, stationary or counter-directional objects’ motion stand out more prominently to the viewer. In AR displays, integrating visual information into natural head movement patterns, such as head rotation, has the potential to align contents into the direction of visual attention, and thereby reducing distraction during walking activities.

Our goal is to subtly introduce non-urgent AR information to not disrupt users when they are walking. We draw inspirations from a research on ambient and secondary displays in Virtual and Augmented Reality. Unlike urgent notifications demanding immediate action, ambient and secondary displays aim to keep users aware of information that is not their primary focus, without causing abrupt stops (Matthews, 2006).

In information placement and retrieval for ambient display, peripheral interfaces present visual information outside user’s central field of view (beyond 30°) to avoid obstructing their main focus (Weiser and Brown, 1996; Matthews, 2006; Cadiz et al., 2001). Along the line, Lages and Bowman propose adapting virtual content to position them close to nearby walls during walking without blocking central vision (Lages and Bowman, 2019a; Lages and Bowman, 2019b). Han et al. embed contents directly into existing physical objects to minimize distraction (Han et al., 2023).

Ambient AR displays aim to optimize information through various retrieval methods and placement. Glanceable interfaces, for example, enable users to access information from the periphery using head cursors or gaze summoning (Lu et al., 2020). Prior study results have indicated a trade-off: suddenly displaying visuals into central-field displays are intrusive, while peripheral placements often go unnoticed. In ambient and secondary display, achieving a balance between intrusiveness and noticeability is crucial (McCrickard et al., 2003). Placement is also investigated in mobile scenarios, such as comparing information placed in the central visual field using display-fixed and body-fixed systems. Users perceived body-fixed systems as less urgent, with both systems having similar perceived task loads (Lee and Woo, 2023). However, these work often overlook a critical initial step: how AR visuals capture user attention at the first place.

Visual information appears in the user’s field of view through various forms of motion, each capturing attention differently. Fade-in and Pop-up techniques are often compared, with Pop-up generally regarded as more distracting, while Fade-in gradually draws attention and is preferred (Janaka et al., 2023). However, these techniques often happen in static scenarios where there is no motion in the user’s field of view. As discussed above, walking presents a different context in terms of visual attention, which requires a distinct approach towards minimizing disruption. Moreover, behavioral measures for assessing task interference during walking remain underrepresented in prior research.

Inspired by prior studies, we aim to use a motion-aware technique to adapt AR information delivery to user movement in walking scenarios, with the goal to minimize disruptions to users’ walking behavior.

Our method takes an AR message that should be delivered to users as input, as well as information about users’ position and direction. The visual information onsets at the start of a head turn, and moves to its final location by the time the head rotation is complete, as shown in Figure 2. In our current implementation and evaluation, we do not rely on turn detection, but leverage fixed points in the evaluation environment where users have to turn (e.g., when walking towards a wall). We chose this implementation, as we were interested in the “ideal” conditions, i.e., when we have ground-truth of whether users will turn. In future work, we plan to replace this implementation with a simple rotational velocity threshold: once users’ turn with a certain speed, we deliver a message if it was queued by the system.

To move in AR messages in the moments of head rotation, the visuals’ velocity is set proportional to the user’s head rotation velocity. Based on pre-tests and pilot studies, we set the velocity to be 50% higher than users’ head rotation velocity. This threshold balances the requirements that the notification appears at the end position quickly, but the movement is still perceived as in sync with users’ actual rotation. The starting position of the visual information is outside the mid-peripheral field ( 60 ° , i.e., outside of the headset’s display range), with the ending position at approximately 15 ° , which lies within the central visual field ( ± 30 ° ) . For our evaluation, the visual information remain visible until manually dismissed by users.

We use a within-subject design with two independent variables, Animation Technique with four levels (TurnAware, Fade-in, Pop-Up, None) and Task with two levels (Object swapping, Number reading). This design yielded a total of eight conditions, i.e., the combination of one animation technique with a primary task. Each participant completed all eight conditions. As dependent variables, we measured primary and secondary task performance (time, errors), walking behavior (speed changes, resumption lag), and subjective ratings, described below.

The study was situated in 6.8 × 12.6   m 2 office space. The study was conducted using a Meta Quest Pro headset with passthrough enabled. Participants wore the headset during all conditions, including during the None condition. The AR environment was developed using Unity 2022.3.11f1. We placed virtual decorations around the environment using models from Unity Asset Store to create a visual balance between physical and virtual contents in this AR space. In our pre-tests without additional virtual decorations, the AR messages were overly salient because the passthrough view showed the physical world with lower visual quality compared to the virtual objects, as shown in Figure 4. We believe that future headsets will be able to deliver a passthrough experience where the visual discrepancy between the physical world and virtual contents will be significantly lower. Thus, to equalize the visual quality between the space users see and the AR messages, we introduce the additional virtual elements. In other words, we decrease the visual saliency of the AR messages compared to users’ surrounding world. We plan to replicate our experiment with more advanced headsets in the future.

Participants walked on a predefined path, illustrated as dotted path in AR to participants, shown in Figure 4B. Along the path, there were various sending points, i.e., possible locations on which AR message might occur. We pre-selected those locations to increase the consistency across participants. Our experimental system then randomly selected where to display the specific AR messages during each condition. Participants used Oculus Touch controllers in all conditions to perform the primary task, such as selecting Yes/No questions in Number Reading Task, and grabbing virtual objects in Object Swapping Task. The two task’s apparatus is shown in Figure 5. Participants were monitored by the experimenter to ensure that they complete the tasks while adhering to the predefined walking path. We used the headset’s built-in tracking system to record participants’ movements and interactions for later analysis.

We recruited 16 participants (age: M = 23.87 years, S D = 3.53 ; 10 female, 6 male; all students and staff from a local University). All participants had normal or corrected-to-normal visual acuity and no (known) color deficiencies, based on self-reports. Participants self-identified as having prior Augmented Reality experience of M = 2.26 ( S D = 1.01 ) on a scale from 1 (None) to 5 (Expert). All participants were naive with respect to the purpose of the experiment. Participants were compensated with a $ 20 gift card. The study was approved by the local Institutional Review Board.

The study lasts around 70 min. After signing an informed consent form and completing the demographic questionnaire, participants were introduced to the study task, and went through a guided practice round involving walking two cycles along the path without AR visual messages. Then, they proceed with the full experiment. During the study, each participants went through all four techniques, once per task, for a total of 8 study conditions. Participants first completed all conditions for one primary task (four times, once per animation technique), and then for the other primary task. The order of animation technique was counterbalanced using a Latin square, the order of tasks was alternated. Among study conditions that includes AR information, participants receive a total of 6 or 7 messages in the 3 cycles of walking. After completing each study condition, participants took off the headset and completed the post-condition questionnaire.

We measured participants’ primary and secondary task performance, as well as their walking behavior, resumption lag in walking. We draw inspiration from McCrickard’s evaluation of human information processing for notification systems (McCrickard et al., 2003), where participants’ walking patterns indicates interruptions, and their reactions to AR visuals indicates noticeability.

During the study, we observed that one participant did not follow the designated walking path, which caused the system to lose tracking. This participant received too few visual messages (1 or 2) to make a meaningful comparison. As a result, we count this as an outlier and exclude their data from subsequent analysis, and analyzed the remaining 15 participants.

We employed a series of Friedman tests to analyze walking behavior data across. We first conducted normality testing using Shapiro-Wilk test (Razali and Wah, 2011) for the measured walking behavior data. Results indicated that our the dependent variables did not follow the normality assumption (all p < 0.05 ). To address the non-normality, we use a series of non-parametric Friedman tests (Zimmerman and Zumbo, 1993) to analyze the behavioral data (walking speed, resumption lag, and reaction time) across different techniques.

If the Friedman tests indicated a main effect, we analyzed the difference between each technique in each of the two task using pairwise Wilcoxon signed-rank tests (Wilcoxon et al., 1970) with Bonferroni adjustment as post-hoc tests. We used the same method for self-reported measures of perceived workload and subjective ratings. The statistical tests were performed using the statsmodel library version 1.14 in Python 3 (Seabold and Perktold, 2010).

The results for largest difference between walking speed is shown in Figure 7, while the average difference is shown in Figure 8. Below, we report on the data from the largest difference in walking speed, as we believe it shows the highest possible disruption that participants might experience. The statistical results between largest and average walking speed, however, yielded similar main effects and pairwise differences. TurnAware achieves the least disruption in walking speed compared to the two baseline techniques, as shown in Figure 7. A Friedman Test on the combined task data showed a significant main effect of technique towards the walking speed difference ( χ 2 ( 2 ) = 108.64 , p < 0.001 ). Pairwise Wilcoxon signed-rank tests indicated significance in two task pairs: TurnAware-Fade ( p < 0.001 ) , and TurnAware-Pop ( p < 0.001 ) , but not in Fade-Pop ( p = 0.234 ) . Our result shows that TurnAware outperforms the two baselines in having the least reduction of walking speed when delivering AR Messages to participants.

In the number reading task, TurnAware techniques’ walking speed difference compare to None condition is − 15.30 % (SD = 32.33), with − 24.74 % ( S D = 44.23 ) for Fade-in and − 24.65 % ( S D = 43.25 ) for Pop-in. In object swapping task, TurnAware’s techniques’ speed difference is − 19.46 % ( S D = 38.19 ) , with − 29.86 % ( S D = 49.86 ) for Fade-in and − 28.42 % ( S D = 48.53 ) for Pop-in. Under varying path conditions, the TurnAware technique consistently led to lower speed differences compared to other two techniques.

Using our technique, participants took the least time to resume to their previous walking speed, as evidenced by the shortest resumption lag among all three techniques (Figure 9). The Friedman Test result indicates a significant main effect ( χ 2 ( 2 ) = 11.87 , p < 0.005 ). Posthoc pairwise Wilcoxon signed-rank tests show a significant difference among all pairs (all p < 0.001 ) when looking at the combined results for both primary tasks. We focus on these results in the subsequent reporting For the individual primary tasks, only the pairwise differences between Fade-In and TurnAware, and Pop-up and TurnAware yielded statistical differences.

On average, the resumption lag was 1.27 s ( S D = 1.80 ) , which is less than Fade-in ( M = 2.09 ; S D = 2.82 ) and Pop-up ( M = 1.62 ; S D = 2.10 ). These results show that participants using the TurnAware technique can resume their walking speed faster compared to the two baseline techniques.

Beyond less interruptions on walking, participants were able to more quickly react to the AR messages. The Friedman test showed a main effect of the techniques on reaction time ( χ 2 ( 2 ) = 96.58 , p < 0.001 ). Post-hoc pairwise Wilcoxon signed-rank tests shows significance among all pairs ( p < 0.001 ) . Our reaction time result indicates that participants in TurnAware technique dismiss the AR message faster than the two baseline conditions.

Fade-in has the longest reaction time ( M = 4.63 ; S D = 1.35 ), with Pop-in in the middle ( M = 3.98 ; S D = 1.27 ), as shown in Figure 10. The TurnAware technique achieves the lowest reaction time (seconds) among the three conditions ( M = 3.31 sec; S D = 1.51 ).

Participants’ perceived workload scores from the NASA-TLX subscale are shown in Figure 11. We employ Friedman tests for main effects, and pairwise Wilcoxon signed-rank tests as posthoc tests. For clarity, we only report the statistically significant effects ( p < . 05 ) .

In the object swapping task, TurnAware achieves a lower perceived mental work load (1: very low, 7: very high) ( M = 1.88 , S D = 0.22 ) compared to Fade-in technique ( M = 2.41 , S D = 0.30 ). This indicates that participants perceived the TurnAware technique as less mentally demanding than Fade-in technique. In the number reading task, participants reported a statistically significantly better perceived performance (1: perfect, 7: failure) using TurnAware ( M = 1.71 , S D = 0.24 ) compared to Pop-up ( M = 2.47 , S D = 0.37 ). This indicates that participants felt more successful and confident in their performance when using the TurnAware technique than Pop-up technique, and participants felt more effective in achieving the task goals.

None of the other questions on noticeability, distraction, etc. yielded any statistically relevant differences. It is worthy to note that more than half of the participants reported difficulty to discern or perceive the differences between the techniques presented. We discuss the discrepancy between these subjective responses and walking behavior below.

Unlike static information placement, such as (Lee and Woo, 2023; Lu et al., 2020), in which no significant main effect were found among placement directions (Lee and Woo, 2023), our technique adapts dynamically to user’s motion. Instead of assuming a fixed placement position, our approach leverages the natural changes in the user’s field of view induced by their movements. This allows visual content to be seamlessly moved into the user’s field of view as they turn. TurnAware dynamically adjust onset movement of visuals to match user’s movement direction and rotational speed. This alignment means that users do not have to make additional, abrupt shifts in attention to perceive the information.

Our results indicate that TurnAware helps users balance between walking (indicated by walking speed change and resumption lag) and secondary tasks (indicated by reaction time). Our approach makes the presentation of visual information more accessible and less intrusive. TurnAware is particularly suitable for delivering secondary information, such as weather updates or fitness accomplishments, due to its ability to let user access information with less disruption. Instead of having to deliberately check messages or being startled by sudden notifications, users can receive messages like weather updates and fitness accomplishments directly in their field of view in a seamless way. We believe this enhances the safety and convenience of AR.

Users’ visual field naturally changes while they are turning, i.e., there is motion resulting in optical flow. Integrating AR content while users are turning reduces the startling effects when being presented with new visual information, making it less disruptive, particularly while walking. From subjective feedback, some participants also noted that they were able to anticipate the movement of the visuals from the peripheral to the center if presented with a motion-aware message. We believe that this also enabled them to react faster compared to other techniques.

Our approach of incorporating new information based on motion is related to works that introduce information without users’ awareness, like Mise-Unseen, which hides virtual scene changes by leveraging gaze data (Marwecki et al., 2019). In contrast, our works aims to not decrease noticeability, but to balance disruptiveness and noticeability by aligning visual information to users’ movement. We believe that this balance can be particularly beneficial in multi-tasking AR scenarios, such as viewing daily news or online chats while walking, where the ability to maintain attention and mobility is crucial.

Our results indicate discrepancies between the behavioral data and subjective feedback. Specifically, while participants were less disrupted in their walking behavior with TurnAware, questionnaire data did not indicate differences in perceived load or preference.

We believe this can be attributed to various factors. First, participants are more familiar with the baseline techniques, Fade-in and Pop-ups, as they are already employed in current application such as system notifications, and messaging app notifications. Therefore, while participants change their walking behavior, they might not feel disrupted. Second, we saw that participants exhibited habituation effects, which is also reflected in their comments. A few participants expressed that at first, they were startled by the Pop-ups, but anticipated them in later trials. We believe that in a real-world setting, this habituation effect might not occur since users would not be able to anticipate the messages, thus would likely be disrupted. Lastly, around half of the participants reported difficulty distinguishing between the techniques during post-experiment interviews. This indicates that even though differences were subtle, TurnAware yielded less disruption to walking behavior.

Overall, we believe that AR messages delivered with our motion-aware technique balance noticeability and disruption in scenarios where users are on-the-go.

Our main result shows that TurnAware minimizes disruption on walking behavior compared to other baselines. We hope to explore different environments and scenarios in the future, as our experiment is conducted in relatively controlled space. Our method relies on the assumption that users frequently turn their heads, for example when walking around corners in buildings, or to look at shop windows while browsing through a street. In real-world scenarios, however, the frequency varies, depending on factors such as walking path, traffic, etc. This means that if a message is urgent, i.e., should be displayed immediately, and users do not turn their head, other delivery mechanisms might be preferable. We thus see TurnAware as complementary to other deliver mechanisms, and particularity well suited for non-urgent messages. Investigating more dynamic environments, such as crowded streets, helps explore how environmental distraction might impact the effectiveness of our technique, and potential improvements to enhance notification visibility without increasing intrusiveness.

In our experiment, the turning points were pre-defined and users performed a targeted walking task. For use in free-walking scenarios, TurnAware could be enhanced by integrating a turning point detection system by leverage body tracking models (Zheng et al., 2023; Ponton et al., 2022; Armani et al., 2024), for example. This would allow our approach to be used in a free-walking scenario, where the walking path would be less predictable.

We used two different primary tasks (number reading and object swapping) in our experiment, neither of which was very cognitively demanding. Future research should explore the effects of TurnAware with tasks of higher cognitive load, including wayfinding or multi-user scenarios. We believe that interaction techniques such as TurnAware that balance disruptiveness and noticeability will be beneficial for users’ ability to handle AR messages.

Finally, our current technique focused on delivery of visual information without active retrieval. Future studies could combine our technique with active retrieval methods (Lu et al., 2020). Giving users the freedom to look up information in combination with targeted notification delivery will provide them with the benefits of both approaches. We hope to explore how to refine AR information delivery through automatic and manual deliver in the future.