In terms of algorithms for LfD, there are generally two types, offline and online learning. Learning from demonstration through teleoperation could provide solutions to both types, offline learning and online learning. In Peternel et al. (2016), the authors proposed a human-in-the-loop paradigm to teleoperate and demonstrate a complex task to a robot in real-time. However, this work did not consider the compliant manipulation skills. Online LfD has some advantages over offline learning from demonstration. First, online LfD could form the skill model gradually during the demonstrations. Second, the transition between the teleoperated mode, semi-autonomous and autonomous mode is straightforward. Third, online learning also could provide real-time feedback on the performance of the model, which is like the iterative learning control. The demonstrator could get real-time feedback on the performance of the learned skills. In addition, the learned skill could execute the task online and directly on the sensorimotor level. Finally, because the skill is encoded in the end-effector, it is straightforward to transfer among different robot platforms.

In Latifee et al. (2020), incremental learning from the demonstration method was proposed based on the kinaesthetic demonstration to update the current learned skill model. But the kinaesthetic teaching method lacks immersive, especially involved contact-rich task with tactile sensing. Also, it is hard to demonstrate the impedance skill simultaneously. During the human-in-the-loop demonstration, there is a requirement on the mechanism of control allocation and adaptation between the human demonstration and the autonomous execution by the robot.

In Rigter et al. (2020), the authors integrated shared autonomy, LfD, and RL, which reduced the human effort in teleoperation and demonstration time. The controller can switch between autonomous mode and teleoperation mode, enabling controller learning online. The human-in-the-loop provides a solution for imitation learning to exploit human intervention, which can train the policy iteratively online (Mandlekar et al., 2020). Shared control is an approach enabling robots and human operators to collaboratively efficiently. In addition, shared control integrated with LfD can further increase the autonomy of the robotic system, which enables efficient task executions (Abi-Farraj et al., 2017). Human-in-the-loop and learning from the demonstration were used to transfer part assembly skills from humans to robots (Peternel et al., 2015). An approach combining operator's input and learned model online was developed for remotely operated vehicles (ROVs) to reduce human effort and teleoperation time. In addition, intelligent control methods were employed in the teleoperation to improve the trajectory tracking accuracy, which can ensure the stability of the human-robot skill transfer system (Yang et al., 2019, 2021a). A comprehensive review on human-robot skill transfer can refer to Si et al. (2021c).

In the past years, many researchers from the motor control and neurobiology field tried to answer how biological systems execute complex motion in versatile and creative manners. The motion primitives theory was proposed to answer this question, which means our humans can generate a smooth and complex trajectory out of multiple motion primitives. DMPs are an effective model to encode the motion primitives for robots. Therefore, how to generate a smooth and complex trajectory based on a library of DMPs, has gained attention in the robot skill learning communities, and several approaches have been developed to merge the DMPs sequences. In Saveriano et al. (2019), the authors proposed a method to merge motion primitives in Cartesian space, including position and orientation parts. The convergence of the merging strategy has proved theoretically, and experimental evaluation was performed as well.

Additionally, the motion primitives theory and knowledge-based framework were integrated and employed in the surgery, which can be generalized to different tasks and environments (Ginesi et al., 2019, 2020). Furthermore, the sequence of DMPs was employed to encode cooperative manipulation for mobile dual-arm (Zhao et al., 2018). The authors proposed to build a library of motion primitive through LfD, and the library, including the translation and orientation, can be generalized to different tasks and novel situations (Pastor et al., 2009; Manschitz et al., 2014). Also, a novel movement primitive representation, named Mixture of attractors, was proposed to encode complex object-relative movements (Manschitz et al., 2018). In our previous study, we proposed a method to merge the motion primitive based on the execution error and real-time feedback to improve the generalization capability and robustness (Si et al., 2021b).

The general form of dynamics of the n-DOF serial manipulator robot can be modeled as Santos and Cortesão (2018),

where the D(q) is the inertia matrix, the C(q,q˙), and G(q) represent the Coriolis and centrifugal, respectively. q and q˙ are the joint position and velocity in the joint space, respectively. τ_c is the actuator torque and τ^ext represents torque generated by the end-effector interacting with environments.

Based on Equations (1) and (2), the designed control variable can be described as the following,

where τ_cmp represents a new control variable in the joint space. In order to facilitate the following analysis, we design the controller in Cartesian space; hence, Equation (3) is rewritten in Cartesian space as the follows,

where m_p(q) represents the inertial matrix in Cartesian space, f_tol is the control force in Cartesian space, and J(q) is the Jacobian matrix. The task space velocity can be described as,

The ẋ_p is the velocity in Cartesian space. Additionally, the control torque τ_cmp can be described as,

For the redundant manipulator, the null space can be used to execute second priority tasks, such as obstacle avoidance, tracking orientation, and pose optimization. The property of the null space has a lot of benefits, such as the control torque will not influence the main task. In this study, we optimize the robot pose to keep the joint close to the middle of the range of the joint. The total torque employed in the joint can be described as,

where τ_null is the optimization torque in the null space, and τ_m is the torque for the main task. The τ_null can be represented as,

where τ_n represents the optimization torque, and NproT(q) is the null space projector. Because the τ_null is executed in the null-space, the optimization task will not affect the main task. The null space projector NproT(q) can be described as,

where I is a identity matrix, and the J⁺(q) is the inverse of J(q), which can be described as,

where m_p(q) represents the inertial matrix in Cartesian space, J(q) is the Jacobian matrix. D(q) is the inertia matrix. The optimization torque can be computed as,

K_o is a gain matrix, which needs to be designed based on the requirement on the pose optimization and main tasks. Q(q) is the cost function, which tries to control the joint as close as the middle of the joint angle range.

where q_i is the current angle of the ith joint, q_id is the middle angle of ith joint. q_imax and q_imin are the maximum and the minimum angle of ith joint.

Dynamic movement primitives is an effective model for encoding motion skills via a second-order dynamical system with a nonlinear forcing term. The core idea of robots skills based on DMPs is to model the forcing term in such a way, allowing to generalize the learned skills to a new start and goal position while maintaining the shape of the learned trajectory. DMPs can be used to model both periodic and discrete motion skills. Currently, most research on DMPs mainly focuses on the position and orientation DMPs and their modifications (Lu et al., 2021a; Si et al., 2021a), which can be used to represent arbitrary movements for robots in Cartesian or joint space by adding a nonlinear term to adjust the shape of trajectory. Also, the DMPs can be used to model the force profiles (Zhang et al., 2021) and stiffness profiles (Zeng et al., 2018). For one degree of multiple dimensional dynamical systems, the transformation system of position DMP can be modeled as follows (Ijspeert et al., 2013),

where the p_g is the desired position, p is the current position; v is the scaled velocity, τ_s is the temporal scaling parameter, which can be used to modify the velocity. α_z, β_z are the design parameters, generally, α_z = 4β_z. F_p(x) is the nonlinear forcing term responsible for tuning the shape of the trajectory. The F_p(x) can be approximated by a set of radial basic functions,

where ψ_i(x) is a Gaussian radial basis function with the center c_i and width h_i; p₀ is the initial position, w_i is the weight LfD. The phase variable x is determined by the canonical system, which can be represented as follows,

where α_x is a positive gain coefficient, τ_s is the temporal scaling parameter, and the x₀ = 1 is the initial value of x, which can converge to zero exponentially. For the multiple Degree of Freedom (DoF) dynamic system, each dimension can be modeled by a transformation system, but they share a common canonical system to synchronize them.

As shown in Figure 2, the proposed framework includes teleoperation, perception, skill modeling and robot control. This section derives the controller in Cartesian space, and the whole control structure can be found in Figure 3. It will be convenient to design the controller in the task space because the teleoperation control would be intuitive and straightforward in the task space. For the human-robot collaboration task, the safety of the human is significant; hence, compliant control has been employed for the collaborative robots. In 1985, impedance control was first proposed by Hogan (1985), and since then, a lot of exciting work has been done. A number of work on the impedance control was done for the manipulator control. Additionally, in order to achieve human-like manipulation, the variable impedance control was employed in the human-robot interaction. The core idea of impedance control models the dynamic behavior of robots under disturbance from the environment. In Ochoa and Cortesao (2021), the authors proposed a similar impedance controller for the polishing task. The impedance controller can be described as the following,

where A is the mass matrix, D is the damping matrix, and K is the stiffness matrix. x_d represents the equilibrium point and x_p is the current position of the robot end-effector. f_imp is the interaction force between the robot end-effector and the environment. For the dynamic equation of manipulator in the task space, Equation (4), the total force f_tol, exerting on the robot, can be calculated as follows:

We define a new control variable ftol*, and the f_c is represented as follows,

The Equation (4) can be represented as follows,

Therefore, the control law of ftol* is,

In this article, the mass matrix is approximated by Ochoa and Cortesao (2021),

So, the control law becomes,

The joint torque for the main task can be given by,

Because the J˙(q) has a small influence on the system, the -mp(q)J˙(q)q˙ can be ignored. Therefore, the control law for the main task in the Cartesian space can be written as follows,

where ftol* can be rewritten as,

where f is the force vector for translation control in Cartesian space, and u is the torque vector for orientation control. The translation controller in discrete form can be described as the following,

where D_p is the damping matrix, K_p is the stiffness matrix, and I_p is the integral matrix. i_p[t #x02212; 1] is the integral error in the position at time [t−1]. p_d[t] and p_c[t] are the desired position and current position at time t, respectively. Similarly, the orientation controller in discrete form can be represented as,

where D_o is the damping matrix, K_o is the stiffness matrix, and I_o is the integral matrix. w_c is the angular velocity, Δo_cd is the orientation error, and i_o is the integral error in orientation. Finally, based on Equations (2), (8), and (27), the total torque command can be described as the following,

The desired trajectory, including the translation and orientation, is generated from the input device based on displacement commands in the teleoperation mode. The human operator is provided with a GUI interface and a 3D mouse to monitor and control the system. The 3D mouse has two buttons and a six-DoF motion sensor. The two buttons are used to switch control modes, such as teleoperation, autonomous, and collaboration. The six-DoF motion axis of the 3D mouse is employed to generate the reference trajectory for the impedance controller in Cartesian space.

where ΔP and ΔR represent the translational and rotational displacements, respectively. ΔZ is then converted to the desired motion in the robot's base frame.

In the autonomous mode, the desired trajectory in the robot's base frame, including the translation and orientation, is generated based on the learned DMPs.

where P_d and R_d represent the desired trajectory in translational and rotational directions, respectively. Z_d is then converted to the desired motion in the robot's base frame.

A collaborative robot, Franka Emika Panda, was used to conduct experiments, as shown in Figure 4. An external force/torque sensor is equipped in the wrist to sense the interaction force and torque between the end-effector tool and the environment. As shown in Figure 4, we designed the fixtures (3D printing) to connect the layup tool (roller), force/torque sensor, and the robot end-effector. A RealSense depth camera D435 is used to observe the working scenario of the robot, and the visual feedback is transmitted to the computer on the leader side for the human operator to monitor the remote scenario.

A 3D mouse from the 3DConnexion company is more suitable for teleoperation, and it can output linear and angular components of the joystick's position and the status of the two buttons. The twist command of the 3D mouse, consisting of the linear and angular components, is used to map the translation and orientation of the end-effector. The buttons states as an event-trigger signal were employed to switch control modes. There is a control interface of the robot manipulator provided by the Franka control interface (FCI) on the robot side, which provides the control interface and a fast and direct low-level bidirectional connection to the robot arm. In the leader site, there is a laptop to execute the control and learning algorithm. The generated command is transferred to the Franka control board. Linux Operating system is run on the laptop, and Robot Operating System (ROS) is used to communicate among the different modules.

As shown in Figure 5, the control system parameters can be displayed and modified by the human operators online. The human operators can change the control modes (switching between teleoperation mode and collaboration mode) via adjusting these parameters, including stiffness K_p and K_o, damping D_p and D_o, integral I_p and I_o, and the null-space optimization gain matrix K_n. These parameters can be modified online based on the different task requirements, such as more stiff in one degree or more compliant in another degree. The interaction force/torque between the end-effector and the environment can also be displayed on a monitor, which provides more knowledge on the interaction process.

In terms of the controller design and the human-robot skill transfer, defining the proper coordinate frame is necessary. The proper frame could reduce the cognitive workload during teleoperation and human-in-the-loop interaction and collaboration. In this study, we defined three frames on the robot sides, as shown in Figure 6. The impedance controller is defined in the cartesian space, and the desired command is based on the based frame of the robot. The task description is based on the component frame, and the transformation from the based frame of the robot to the component frame is fixed. The impedance gain defined by the user is based on the end-effector frame, therefore, the parameters of the controller need to be transformed into the based frame.

To enable the human-robot composite layup skill transfer, the human operator needs to composite layup through teleoperation. The motion primitives of composite layup were recorded, as shown in Figure 7. During the demonstration, the human operator demonstrated the layup for a flat component. From the results, the roller moves forward and back in the X-Y plane, which is the primitive motion skill for the composite layup. We modeled this motion primitive by DMPs, which can be generalized to different locations. Regarding the parameters of DMPs, please refer to Table 1. For the Z-axis, there is a small motion, which can be used to generate the contact force along Z-axis in the end-effector frame. The stiffness parameters for the X and Y are the same, K = 3,000, while for the Z-axis is large, K = 6,000, which can be guaranteed to generate contact force along Z-axis. The small impedance along X and Y, which can feature a compliant manner.

In this case study, we would like to evaluate the generalization to a novel and large component, which is necessary for the composite layup in the real industry plant. The automation of composite layup only relies on the motion primitive and a little bit of information on the component. Generally, it is straightforward to attain the geometry of the component based on the CAD model of the part. In this case, we assumed that the vertex coordinates of the part are known. For example, given the four vertex coordinates, we can get the region where the parts need to be manufactured for a plane. For the area that needs to be processed, we divided several sub-areas. Each sub-area can be modeled with a task variable, including the start and end coordinates. For the DMPs based skill model, only the task variable is needed to reproduce the motion skills.

The number of motion primitive is dependent on the size of the component. For the X direction, as the generalization of DMP, the number can be random. In the Y direction, the number is the length of the workpiece, divided by the width of the roller, which ensures the roller can cover the whole workpiece. As shown in Figure 8, there are 16 motion primitives for the big plane. Each motion primitive is similar to the human demonstration motion. Between two motion primitives, we used a motion planning algorithm to generate a transition trajectory. The first row, along X-axis, shows that the coordinates of the roller decrease from 0.6 to 0.34 m. The middle row, along the Y-axis, shows that the coordinates of the roller change between −0.17 m and 0.1 m. The real trajectory of robots can cover the whole workpiece.

The Figure 9 shows the tracking error between the command from the DMP model and the actual trajectory of the roller. The tracking error is less than 0.005 m in the X, Y, and Z-axis. During the composite layup, the orientation of the end-effector is fixed, and the tracking error is less than 0.02 rad. The control accuracy is enough for the composite layup, which proves the performance of the impedance controller for the composite layup.

In this case study, we would like to evaluate the generalization to a novel and small component, which is similar to the previous experiment case. The main difference is that the four vertex coordinates are changed. The motion primitive is the same as the previous experiment, and we evaluate that the learned skill can be generalized to a novel component with different sizes.

As shown in Figure 10, for the small component, only eight motion primitives are needed to cover the whole plane. The proposed framework could generalize to different sizes only based on the vertex coordinates for a plane part. From Figure 11, the tracking errors are less than 0.005 m, and the orientation tracking errors are less than 0.02 rad. The control accuracy is enough for the composite layup.

This experiment case aim at evaluating the generalization to an inclined plane. In this case, the key points for this task are P₁(0.36, - 0.14, 0.38), P₂(0.37, 0.01, 0.42), P₃(0.50, 0, 0.42), and P₄(0.50, - 0.18, 0.38). We assume that we know the CAD model of the component or the orientation of the model can be measured based on sensing technology, such as machine vision.

From Figure 12, there are eight motion primitives to cover the whole inclined plane. The main differences between the inclined plane and the horizontal plane are the motion along the Z-axis and the orientation. From the third row, the motion range along the Z-axis is from 0.41 to 0.38 m in Figure 13. Additionally, the orientation is different, which needs to keep the roller perpendicular to the plane. The tracking errors are less than 0.005 m, and the orientation tracking errors are less than 0.02 rad, which demonstrates the generalization of the learned skill and the performance of the impedance-based controller for a composite layup in the inclined plane.

This experiment aims at evaluating the collaboration performance of the impedance control-based teleoperation system. Existing research focuses mostly on physical human-robot collaboration, with less work on collaboration between teleoperation, in-site humans and robots. In this experiment, we evaluated that teleoperation and in-site human can collaborate smoothly through modifying the parameters of the impedance controller. We make use of the character of the torque-computed control based on impedance control. For example, we set the stiffness of the impedance controller to a specific degree; the control along this degree becomes a free-motion mode, which can be kinesthetic teaching or adjusting the robots by an in-site human.

From Figure 14A is the control command from the teleoperation, and (Figure 14B) is the command by in-site human operation. The orientation control is autonomous by the robot. Figure 15 is the trajectory of the end-effector in the hybrid control mode. The results show that the control system can integrate teleoperation and kinesthetic demonstration and autonomous. The transition among the three modes can be smooth.