In the context of CMC, researchers have explored a wide range of the social effects of communication between people using network-connected digital devices to exchange messages (e.g., text messaging, social network site interactions, videoconferencing) (Thurlow et al., 2004). Among the CMC channels, text messaging has been majorly studied because of its unique features attributed to the limited non-verbal information for exchange and asynchronicity (Walther, 1996; Joinson, 2001). As important implications on the use of text messaging, researchers identified the users’ hostile, aggressive, and uninhibited communicative behaviors, called flaming, increased compared to those of face-to-face and videoconferencing (e.g., Lea et al., 1992; Kiesler et al., 1985; Castellá et al., 2000). One explanation on the presence of these behaviors has been given based on deindividuation. Kiesler et al. (1984) discussed the phenomenon of the deindividuation in CMC as a situation where “social standards will be less important and communication will be more impersonal and more free because the rapid exchange of text, the lack of social feedback, and the absence of norms governing the social interaction redirect attention away from others and toward the message itself.” Such situations reduce users’ public self-awareness (Matheson and Zanna, 1988), and more impulsive and assertive behavior not considering the recipients’ feelings can be promoted (Castellá et al., 2000).

To overcome this issue while ensuring the benefit of using text messaging, we consider an approach that involves an intelligent supportive agent, such as a dialogue robot for text messaging. By providing interventions from the robot, such as showing empathy to the users, the quality of messages can be influenced. As an example, from a case study conducted to encourage elderly self-disclosure to their remote family members by applying a social mediator robot, the property of its expressions (i.e., personality traits and quantity of social behaviors) significantly influenced the depth of the elderly self-disclosure (Noguchi et al., 2020). The key finding of the study is that the information disclosed by users depends on the behaviors of the mediator robot locally interacting with them. Furthermore, recently, Hancock et al. (2020) conceptualized artificial intelligence-mediated communication (AI-MC) as a modern extension of CMC and envisioned it as a possible future communication style for humans: defined as “interpersonal communication in which an intelligent agent operates on behalf of a communicator by modifying, augmenting, or generating messages to accomplish communication goals.” Some of our devices have been embedded with some agents to mediate our messaging (e.g., Gmail smart reply). Therefore, if more and more robots and AI agents mediate our communication, they can be applied as a new standard in the future.

Expression modalities of social robots have been widely explored in human-robot interaction (HRI). Many social robots have been designed to convey their internal states by speech dialogues and non-verbal signals. For example, facial expressions (Todo, 2018), body movements (Nakata et al., 1998), sounds (Jee et al., 2009), sweats (Guo and Tanaka, 2020), colors (Terada et al., 2012), vibrations (Song and Yamada, 2017), and their combinations (Löffler et al., 2018; Song and Yamada, 2017) have been used as non-verbal expression modalities for social robots. Recently, dynamic haptic parameters, such as changes in skin temperature (Peña and Tanaka, 2020; Nie et al., 2012; Block and Kuchenbecker, 2018), texture (Hu and Hoffman, 2019), and SwarmHaptics (Kim and Follmer, 2019) have been discussed in the literature. Interactions using these haptic modalities are particularly useful in complementing the expressiveness of robots designed with few anthropomorphic features and configurations. However, besides (Noguchi and Tanaka, 2020), no study on robotics research has investigated the use of internal weight movements (weight shifts) in robot expression.

From the technical perspective of weight shifts, handheld devices with weight-shifting functions have been mainly studied in the field of virtual reality (VR). Since the relationship between the presented torque and the human perception of heaviness was found by Woodruff and Helson (1965), researchers have designed devices based on the principle. For example, TorqueBAR, Shifty, and Transcalibar are haptic feedback devices equipped with 1D or 2D translational weight-shifting mechanisms. Changing the distribution of the internal weight enhanced the users’ perception of virtual objects (Swindells et al., 2003; Zenner and Krüger, 2017; Shigeyama et al., 2019). SWISH is a human-computer interface capable of simulating dynamic fluid behavior in VR or augmented reality (Sagheb et al., 2019). It mimics the shifts of the center of gravity of a fluid vessel with a responsive motor actuation integrated with virtual fluid simulation. From other perspectives on weight shift use, haptic displays for mobile phones have been discussed. Hemmert et al. (2010) proposed Weight-Shifting Mobile, a mass-shift based system, for haptic actuation in mobile phones. They suggested that changing of the gravitational properties of the device provides beneficial applications, including augmenting digital content with physical mass, ambient displays, and haptically augmented wayfinding. Compared with previous works, studying weight shifts as an expression modality of social robots is novel. Additionally, we focus on the combining speech dialogues and discuss its effects on human perception, an unexplored area in both HRI and VR.

From the embodied perspective on cognition, the weight-based interaction channel of handheld devices can influence users’ judgements and decisions. Jostmann et al. (2009) suggested that the abstract concept of importance is grounded in bodily experiences. Some important results were obtained from the studies conducted: holding a heavy clipboard increased judgments on the monetary value and made participants consider fair decision-making procedures as more important. Similarly, Ackerman et al. (2010) explored how incidental heaviness sensation affects human decisions. As a result, heavy objects made job candidates appear more important: participants using heavy clipboards rated the candidate as displaying a more serious interest in the position. These findings suggest that, in principle, communicating through weight conveys the seriousness of the robot. Therefore, if an appropriate weight shift is expressed along with robot speech, the user may listen to the speech more seriously than the case no weight shift is expressed.

To control participants’ mental states to be frustrated in a messaging context, the following scenario and the accompanying message were given to each participant by a paper.

Context scenarios that make participants frustrated due to the tardiness of others is often used in psychological assessment (e.g., Rosenzweig, 1978). In designing our scenario, we modified and applied a scenario used in a Japanese study involving 183 graduate and undergraduate students (Kikuchi et al., 2008) In this scenario, the part “in the shade under the scorching sunshine” is used to control the physical load associated with each participant. To minimize both the ceiling and floor effects, a moderate level of the physical load was presented in our context scenario ² .

In the message sent by friend A, there was a statement explaining the reason for the tardiness. To identify what reasonable statement to be included, a preliminary survey was conducted. From the survey involving 17 participants, it was revealed that recipients felt strong anger when the sender explicitly informed them of forgetting the appointment. Thus, we included the phrase in the message.

Ninety-four participants (51 males; 43 females; age: 18–41, M = 21.6, SD = 3.19) were recruited at the University of Tsukuba. They were all Japanese speakers, and every experimental material (e.g., instructions, questionnaires, and speech dialogues of the robot) were provided in Japanese. An experimental booth was set up in a cafeteria of the university, and the recruitment was conducted for passers-by. Posters and banners were placed near the booth to guide interested people in our study. Each participant got a sticky note with a print picture of OMOY on the front cover as a souvenir. The total duration required for each participant was 10–15 min. The study protocol was approved by the research ethics committee of the Faculty of Engineering, Information, and Systems of the University of Tsukuba (2019R317-2), and all participants provided informed consent.

During the recruitment of participants, we placed a public notice about our study titled “An experiment of a dialogue handheld robot.” at the cafeteria where the experimental booth was set up. If a person approached the booth and showed interest in this study, we encouraged them to take a sit on a chair placed in the booth. The study was conducted under face-to-face conditions between an experimenter and each participant. First, the participant completed a survey on their age, gender, on a form displayed on the iPad. Next, we introduced OMOY to the participant as: “Here is a handheld communication robot, named OMOY. This robot is equipped with a movable weight inside the body. One of its purpose is to mediate human messaging. Today, you will experience an example in which the robot mediates a message.”

To provide a better understanding of how the interaction with OMOY is like (e.g., about the vocal features and weight shifts), each participant was asked to listen to the self-introduction by OMOY. We instructed them to hold OMOY from the back with their non-dominant hand (similar to holding a smartphone) and to not tilt it by more than 45° to prevent it from lying down. Additionally, from pilot studies, noise from the actuators was an important consideration. Thus, participants wore headphones with a noise cancelling function (SONY WH-1000XM4) to listen to the robot. In the self-introduction, OMOY said: “Nice to meet you. Thank you for your cooperation in the test today. My name is OMOY. I will help you with your daily messaging. I read out messages received from someone and can give you my opinions when you are about to make a reply. Additionally, I can do weight shifts like this way. Best regards.” At the underlined part, OMOY performed an weight shift pattern including translational and rotational weight movements that were completely different movement patterns used in the main study.

Subsequently, the experimenter showed the context scenario paper (described in Section 4.1) to each participant while asking them to imagine their “normal friend” as friend A in the context scenario. Because it was known that people easily provide forgiveness to their intimate persons (McCullough et al., 1998), we instructed the participants that friend A is neither their best friend, boyfriend/girlfriend, nor a person with whom they want to be close. After the context scenario was given, the participants were asked to hold OMOY and wear the headphone. By sending a command to the robot through a laptop, the experimenter produced each stimulus: first, the robot auditory reported the message sent by friend A, then said its opinion with or without weight shifts. At the end of each experimental session, the experimenter passed an iPad to the participant who answered questions regarding their anger state, interpersonal motivation, and seriousness recognition.

Since the impressions of the robot’s intervention were expected to be different depending on the user’s affective state, we divided the participants into two groups based on high/low pre-anger using the mean value (with boundary value at 1.85). In the group of participants with low pre-anger, 52 participants were allocated, whereas 42 participants were in the group with higher pre-anger. The t-test ensured that the participants involved in the higher pre-anger group showed stronger pre-anger (M = 2.28, SD = 0.30) than the participants in the group with lower pre-anger (M = 1.50, SD = 0.21; t (70.7) = 14.3, p < 0.001, d = 0.26). To discuss the extent participants’ anger and interpersonal motivation changed before and after the robot intervention, we applied the rate of change (ROC) as dependent variables. Each value of ROC was given by the following equations: R O C A n g e r = p o s t ‐ a n g e r − p r e ‐ a n g e r p r e ‐ a n g e r (1) R O C R e v e n g e = p o s t ‐ r e v e n g e − p r e ‐ r e v e n g e p r e ‐ r e v e n g e (2) R O C A v o i d = p o s t ‐ a v o i d − p r e ‐ a v o i d p r e ‐ a v o i d (3)

We analyzed the effects of independent factors such as the participants’ pre-anger (high/low), opinion type, and the presence of weight shifts, on each dependent variable: seriousness recognition, post-anger, ROC _Anger , ROC _Revenge , and ROC _Avoid , by performing a 2 × 3 × 2 ANOVA or ANCOVA. Suppose a significant regression coefficient of the gender factor of participants was found, we put this variable into the statistical model as covariance and performed an ANCOVA (otherwise, we used ANOVA).

We examined if participants recognized the serious intention of the robot which exhibited weight shifts. A gender difference was detected, thus we performed an ANCOVA. Results showed a significant main effect of weight shifts (F (1, 81) = 16.1, p < 0.001, η p 2 = . 17 ) indicating that participants agreed that the robot with weight shifts was more serious (M = 6.10, SD = 2.58) than the case with no weight shift (M = 4.58, SD = 2.62). Therefore, we see that the weight shift pattern used in this study was successfully designed. Also, there was a significant main effect of the opinion type (F (2, 81) = 17.7, p < 0.001, η p 2 = . 30 ). Then, we performed the Bonferroni test as a post hoc comparison. Results indicated that participants evaluated the robot with no comment (opinion type 3) as less serious, compared with the robots on the other two opinion types (p < 0.001).

First, we performed ANOVA on the post-anger variable to compare the intensity of anger that the participants held after the robot intervention. The results showed a significant interaction between the factors of the participants’ pre-anger and the presence of weight shifts (F (1, 82) = 9.87, p = 0.002, η p 2 = . 11 ). Also, post hoc comparisons using t-tests showed a significant difference in the presence of weight shifts for participants with high pre-anger. Their post-anger was significantly smaller if the robot with weight shifts was used (M = 1.77, SD = 0.37), compared with the cases where the robot did not exhibit any weight shift (M = 2.17, SD = 0.53; t (40) = 2.82, p = 0.007, d = 0.46) (Figure 4). However, no significant difference was found in the participant group with low pre-anger. Also, participants having high pre-anger reported significantly larger post-anger than the cases with having low pre-anger: with weight shifts (t (45) = 4.29, p < 0.001, d = 0.33); without weight shifts (t (30.2) = 7.22, p < 0.001, d = 0.41) (Figure 4).

Next, we performed ANOVA on the dependent variable ROC _Anger , to compare how participants’ anger level had changed over the robot intervention. The results showed a significant interaction between the factors of pre-anger and the presence of weight shifts (F (1, 82) = 18.5, p < 0.001, η p 2 = . 18 ). Post hoc comparisons using t-tests showed significant differences in the presence of weight shifts for both groups, but the opposite effects were observed (Figure 5). In the high pre-anger group, participants’ anger was more suppressed in the condition where the robot exhibited weight shifts (M = − 0.23, SD = 0.18) than the case it exhibited no weight shift (M = − 0.035, SD = 0.15; t (40) = 3.86, p < 0.001, d = 0.17). However, in the low pre-anger group, participants’ anger was more suppressed in the condition where the robot exhibited no weight shift (M = − 0.15, SD = 0.14) than the case it exhibited weight shifts (M = − 0.068, SD = 0.16; t (50) = 2.07, p = 0.043, d = 0.15). In cases applying weight shifts, while 23% of anger was suppressed in the high pre-anger group, only 6.8% was suppressed in the low pre-anger group. The difference was statistically significant (t (45) = 3.28, p = 0.002, d = 0.17). In cases no weight shift is applied, 3.5% of anger was suppressed in the high pre-anger group, and 15% of anger was suppressed in the low pre-anger group. The difference was also statistically significant (t (45) = 2.85, p = 0.007, d = 0.14). No significant effect on the opinion type of the robot was observed.

We examined whether the robot’s seriousness recognized by the participants affected their ROC _Anger , ROC _Revenge , and ROC _Avoid by the regression analysis. In the high pre-anger group, a significant regression was found for each dependent variable (ROC _Anger R ² = 0.12, F (1, 40) = 5.20, p = 0.028; ROC _Revenge R ² = 0.12, F (1, 40) = 5.21, p = 0.028; ROC _Avoid R ² = 0.26, F (1, 40) = 13.8, p = 0.001). Regression coefficients (ROC _Anger β = − 0.028, p = 0.028; ROC _Revenge β = − 0.028, p = 0.028; ROC _Avoid β = − 0.038, p = 0.001) show that the seriousness recognition was negatively correlated with the participants’ anger, revenge motivation, and avoidance motivation. However, in the low pre-anger group, no significant regression was found. The results suggest that the seriousness displayed by the robot had effects only on the participants having high pre-anger.