The mapping of finger motions on multi-articulated robot hands is nowadays an open problem in the robotics community (Shahbazi et al., 2018; Carfì et al., 2021). Indeed, a general mapping solution for the replication of human primary hand (PH) motions onto a robotic target hand (TH), ensuring ease of use and good motion reproducibility, is a very challenging target to be achieved. The presence, in real situations, of unavoidable kinematic dissimilarities between PH and TH does not allow to precisely replicate PH finger motions onto TH fingers with a direct identity mapping, and some adaptation and/or interpretation must be adopted (Gioioso et al., 2012). Therefore, despite several advances during the last years, it results from evidence that developing TH to RH mapping strategies is a non-trivial and both conceptual and analytical problem (Meeker et al., 2018).

The two principal fields in which human-to-robot hand mapping methods used are teleoperation and learning by demonstration (Dillmann et al., 2000). In teleoperation applications, data measured from the operator’s human hand are used to control in real-time motion of a robot hand. Differently, in learning by demonstration applications, motion measurements from the human hand are exploited as a source of human skill information to improve the dexterity and the behavior of autonomous robot hands. In the literature, several works have attempted to design mapping solutions such that to obtain correct and predictable behaviors on the TH. Two fundamental sub-problems have been considered for human-to-robot hand mapping: 1) the measurement of human PH motions using appropriate kinematic modeling and sensor equipment and 2) the design of a mapping algorithm to obtain a proper imitation of PH motions on the robot TH. In this study, we will not focus on the former sub-problem, which has been extensively addressed using sensorized gloves (Cobos et al., 2010; Geng et al., 2011; Bianchi et al., 2013), vision systems (Gupta and Ma, 2001; Infantino et al., 2005; Wachs et al., 2005), hand exoskeletons (Bergamasco et al., 2007), and advanced calibration procedures (Rohling and Hollerbach, 1993). We will instead focus on the second sub-problem, for the reason that it is still lacking a general solution (Colasanto et al., 2012).

Among the many approaches that have been presented in the literature for PH to TH motion mapping, the simplest realization is the direct joint mapping (Speeter, 1992), in which joint angles of the PH are directly imposed onto the robot TH. Direct joint mapping presents the advantage of being rapidly implementable, while preserving PH finger shapes on anthropomorphic TH, allowing better reproduction of gestures and power grasps (Cerulo et al., 2017). Additionally, it generally permits an increased predictability of TH behaviors, allowing the human operator to more easily compensate for inaccurate behaviors by moving ones hand (Speeter, 1992). The second simplest mapping approach is the direct Cartesian mapping (Rohling and Hollerbach, 1993), which consists in computing Cartesian positions and orientations of the PH fingertips (i.e., the fingertip poses) by means of forward kinematics, and directly imposing them on the TH fingertips, followed by the computation of the TH joint angles via inverse kinematics. Direct Cartesian mapping is more appropriate for the execution of precision grasps and in-hand manipulation since it preserves the PH fingertip positions on the TH. However, on the other hand, it does not guarantee, in general, the preservation of the PH finger shapes, making difficult the correct execution of power grasps, non-prehensile manipulation, and gestures (Chattaraj et al., 2014). More advanced mapping approaches are based on attenuating the problem of PH-TH kinematic dissimilarities by using a suitable task-oriented description of the finger motions. This is the case of the object-based mapping method (Gioioso et al., 2013), in which PH finger motion information is encapsulated within a virtual object defined in the PH workspace, which is then reported in the TH workspace in order to impose coordinated finger motions. This mapping presents the advantage of being implementable for non-anthropomorphic TH, in which case direct joint and direct Cartesian mappings are particularly unsatisfactory. However, when the precision of single fingertips is required, or purposes different from grasping objects are desired, the object-oriented mapping can produce highly unintuitive behaviors of the TH (Meeker et al., 2018) critically affecting its applicability, even when the object-oriented mapping algorithm is abstracted from defining a specific virtual object shape (Salvietti et al., 2014). In an interesting more recent mapping approach denoted as DexPilot (Handa et al., 2020), PH fingertip poses are obtained via a marker-less vision, and, thereafter, a cost function based on the distance and orientation among fingertips in the TH workspace is designed in order to optimize the final TH finger configurations. Other types of mapping are based on recognizing the PH posture in order to reproduce it on the TH (Ekvall and Kragic, 2004; Pedro et al., 2012; Ficuciello et al., 2021). In this kind of approach, the functions of the TH are limited to a discrete set of predefined grasps/motions and, thereafter, based on the output of the PH posture recognition process (mostly based on machine learning techniques (Dillmann et al., 2000; Yoshimura and Ozawa, 2012)), one of the available TH motions is selected and executed. In this case, the operator can easily learn how to configure ones own hand in order to activate specific actions of the robot hand, whereas the lack of continuous control, and the fact that the number of predefined TH grasps/motions increases with the complexity of the applications, de facto make this mapping method difficult to be applied for several applications (e.g., precision grasps and gestures/grasps not included in the predefined TH motions.) Furthermore, types of PH-to-TH mapping approaches can be found in the literature, for example, based on dimensionality reduction of the TH input space (Ciocarlie et al., 2007; Salvietti, 2018), deep learning architectures (Gomez-Donoso et al., 2019; Li et al., 2019), or shared control (Song et al., 1999; Hu et al., 2005). However, in general, state-of-the-art mapping solutions do not take into account the necessity of preserving on the TH, within a single method, both PH finger shapes and fingertip Cartesian positions. Instead, these aspects are essential to ensure acceptable levels of intuitiveness and predictability of TH motions during a hand motion mapping (Fischer et al., 1998), and we therefore claim that they should be carefully considered for novel advances in the research problem of mapping motions on robot hands.

In the light of these concepts, in this study, we introduce a hybrid approach combining joint and Cartesian mappings in a single solution. Specifically, the proposed hybrid joint–Cartesian mapping exploits specific spatial information that we denote as “available in-hand,” which means that is available a priori just once certain hand kinematics is given because they can be systematically derived from geometrical considerations and forward/inverse kinematics computations. We therefore exploit spatial in-hand information to enforce a smooth and continuous transition between joint and Cartesian mappings. In particular, in this study, we present and evaluate two specific realizations of this mapping paradigm. The first one is denoted as distance-based hybrid mapping, in which the spatial in-hand information related to the distance between the fingertips of the thumb and opposite fingers is exploited to realize the transition between joint and Cartesian mappings. In the second realization, denoted as workspace-based hybrid mapping, the spatial in-hand information used to enforce the transition between joint and Cartesian mappings is related to the areas of the workspace in which the thumb and opposite fingertips can get in contact. In the article, these hybrid mapping realizations are thoroughly presented, and simulation experiments are conducted on a series of well-known multi-articulated robot hands, both anthropomorphic and non-anthropomorphic, in order to evaluate and compare the proposed mapping paradigm.

For the evaluation of the proposed PH-to-TH mapping strategies, we performed a series of simulation experiments using both non-anthropomorphic and anthropomorphic robot hands. As PH, we used the paradigmatic hand model, available from the open-source SynGrasp MATLAB toolbox (Malvezzi et al., 2015) (see Figure 4.) As TH, we simulated, via SynGrasp, four types of well-known multi-articulated robot hands: a 3-fingered hand inspired to the Barrett hand (Townsend, 2000), the 4-fingered anthropomorphic Allegro hand (WonikRobotics, 2015), the anthropomorphic DLR-Hit II hand (Butterfaß et al., 2001), and the anthropomorphic University of Bologna Hand IV (UBHand) (Melchiorri et al., 2013) (Figure 4).

Specifically, for the first simulation experiment, a particular motion was generated for the PH, referred to as tip-to-tip motion, in which the fingertips of the thumb and opposite fingers get in contact in a sequence, as shown in Figure 4. Furthermore, Figure 4 also reports the results of the distance-based (Figure 4A) and workspace-based mappings (Figure 4B) on the different simulated robot hands, along with the relative behavior of the gains for the transition between joint and Cartesian mappings (k _I , k _M , k _R , and k _P for the index, middle, ring, and pinkie fingers, respectively.) Note that for the 3-fingered hand we mapped only the PH thumb, index, and middle fingers, whereas for the Allegro hand we mapped only the PH thumb, index, middle, and ring fingers. From Figure 4, first of all, it is possible to see how both the distance-based and workspace-based mappings allow the TH to conserve the shape of the PH fingers (i.e., their joint configuration) when the fingers are not close to the thumb, and, at the same time, to perform a correct thumb–fingers tip-to-tip contact, thanks to the switching to the Cartesian mapping according to the related transition gains.

In Figure 6A we reported, for each different TH, the mean joint error—that is the mean of the norm of the error between the joint angle vectors of the PH and the TH—during the enforcing of the Cartesian mapping (top graph, for both distance-based and workspace-based approaches), and the mean thumb–finger fingertip Cartesian position error during the enforcing of the joint mapping (bottom graph, for both distance-based and workspace-based approaches). Note that the latter Cartesian error was normalized by the value of the distance between thumb and index fingertips in the open hand configuration, for each respective robot hand. This made the Cartesian error between the different robot hands comparable, taking into account the different TH sizes. Looking at the top graph of Figure 6A, it is possible to observe that for the workspace-based mapping the Allegro hand presented a clearly higher joint error when the workspace-based mapping was used, due to the fact that the Allegro hand presents a workspace geometrically less compatible with the one of the PHs (the paradigmatic hand.) Also, it is possible to see how 5-fingered anthropomorphic robot hands showed a lower joint error. Looking at the bottom graph of Figure 6A, it is worth highlighting that, using the workspace-based approach, the 3-fingered hand and the Allegro hand showed a substantially higher thumb–finger Cartesian error. This was due to the fact that, since these two TH present more pronounced kinematic differences with respect to the PH anthropomorphic structure and therefore more different workspaces, the distance-based mapping enforces the Cartesian mapping for a larger part of the motion, reducing the reported Cartesian error (which is reported when the joint mapping is enforced). Differently, the workspace-based approach required that both the thumb and opposite finger enter the convex hull region which enables the transition to the Cartesian mapping.

The second simulation experiment, visible in Figure 5, consisted in performing the grasp of a sphere requiring to use the joint mapping for the preservation of the PH finger shapes and a thin cylinder requiring precise positioning of the fingertip in order to preserve the fingertips–object contact point locations. For the grasping of the sphere, the PH fingers were closed uniformly along all joints producing the finger flexions. Then, the motion was mapped on the different simulated robot hands (distance-based and workspace-based mappings coincide, in this case, because only joint mapping was enforced in both mappings), and each PH finger was stopped when the relative TH finger got in contact with the object surface. The result of the sphere grasping can be seen in the left part of Figure 5 for the different robot hands, and the Cartesian error is reported in the top graph of Figure 6B. In the latter graph, it can be observed how the 3-fingered hand showed the higher Cartesian error, due to the fact that it presents the less anthropomorphic-like kinematic structure. For the grasping of the thin cylinder, the tip-to-tip motion only for the thumb and index finger was performed on the PH and then mapped with the distance-based and workspace-based approaches on the TH. The PH thumb and index were stopped when the TH counterpart got in contact with the object's surface. As can be seen in the right part of Figure 5, both proposed mapping methods were able to map the precision grasp on the different robot hands (the position of the cylinder was adjusted based on the specific TH because we reasonably assumed that, during the thumb–index approaching motion, the object location could be adjusted with respect to the robot hand accordingly). The bottom graph of Figure 6B reports the joint error when the cylinder is grasped. It is possible to see that, for the different robot hands, the distance-based and workspace-based mappings showed comparable joint errors, with the workspace-based mapping presenting lower error values for the 3-fingered, DLR, and UBHand robot hands. This latter aspect can also be seen in the right part of Figure 5, where the workspace-based approach was able to put the TH fingers in contact with the object with a more similar shape with respect to the PH. A video of the simulation results reported in Figures 4, 5 is available at the link: https://bit.ly/3OUcqaa.

Finally, in order to contextualize the proposed hybrid mapping approaches with respect to other mapping methods presented in the literature, we report a qualitative comparison. Specifically, in Table 1, we report a comparison between the proposed mapping, standard joint/Cartesian mappings, and two recent advanced mappings. The considered mappings are the following: a standard direct joint mapping in accordance with Speeter (1992), a standard direct Cartesian mapping as presented by Rohling and Hollerbach, (1993), a virtual-object-based mapping Gioioso et al. (2013), and the DexPilot mapping (Handa et al., 2020). Specifically, the qualitative comparison was made with respect to the following features: 1) PH-to-TH preservation of the finger shapes, 2) PH-to-TH preservation of the distance between thumb and opposite fingertips, 3) applicability to anthropomorphic TH, 4) applicability to non-anthropomorphic TH, 5) evaluation on multiple hands (with different kinematic structures and number of fingers), and 6) evaluation metrics (see Table 1). Looking at the table, it can be observed how only the proposed mappings and the DexPilot mapping (Handa et al., 2020) have the capability to preserve both PH finger shapes and fingertip positions on the TH. In our approaches, different from the others, in Table 1, we also evaluated these capabilities on different robot hand kinematic structures, both anthropomorphic and non-anthropomorphic, by means of the simulation study presented in this section.

Two realizations of a joint–Cartesian hybrid mapping for robot hands, based on spatial in-hand information, have been presented in this article: the distance-based and workspace-based approaches. By performing several simulation experiments of PH-to-TH mapping, we evaluated the proposed approaches on four well-known robot hands: a 3-fingered hand inspired by Barrett hand, the 4-fingered anthropomorphic Allegro hand, the anthropomorphic DLR-Hit II hand, and the anthropomorphic UBHand. We evaluated both a free motion of the PH including thumb–fingers tip-to-tip contacts and the grasping of a sphere (requiring finger shape preservation) and a thin cylinder (requiring fingertip position preservation.) The results show that both approaches were capable of successfully performing the mapping of PH finger shapes and TH fingertip positions, on both anthropomorphic and non-anthropomorphic robot hands, and the differences between the distance-based and workspace-based approaches have been discussed. Future developments will be devoted to performing experiments with real robot hands, focusing the study on the investigation of specific telemanipulation performance using the proposed mapping, instead of a simulative evaluation of multiple different robot hands.