The jamming based morphological filter is placed over a circular capacitive sensor disk which has 50 “taxels” providing high sensitivity and spatial distribution over the surface of the sensor. The sensor provides measurement with a resolution of 16 bits corresponding to a variation of capacitance proportional to the pressure acting on top of the sensor. Details of the specific sensor and its fabrication have been previous reported (Maiolino et al., 2013; Scimeca et al., 2018). The jamming filter is constructed from a thin flat latex layer, a plastic ring and an deformable silicone skin, which have been sealed to create an airtight system. This is filled with jamming particles, coffee, with a tube connected which allows a vacuum to be applied. This enables the stiffness of the filter to be varied online, transforming it from a low stiffness deformable filter, to a high stiffness, hard filter (Figure 1).

To demonstrate the advantages that can be achieved with this jamming filter we perform some simulated experiments using FEA. To do this, we explore the pressure patterns which are exerted on the surface of the capacitive sensors when different modalities of adaption of the sensor are performed. This is performed using FEA to first determine the shape of the sensors when it is unjammed by modelling the filter as a elastomeric material being brought into contact with the object with a fixed displacement (3 mm). From this process we extract the mesh, or resultant morphology of the filter from the simulation. To then simulate the sensor response, we take this mesh filter shape, and change the material stiffness to represent the jammed filter - using the Young’s Modulus of a hard plastic material (50 MPa). We then obtain a representative sensor response by visualizing the pressure distribution of the back side of the filter when the jammed filter in brought into into contact with an object with a 1N distributed load. This is equivalent to the conditions in which the filter is used on the UR5 robot arm.

Using this FEA analysis we can demonstrate two mechanisms by which this stiffness change offers key advantages. Firstly, it offers a compromise between soft and rigid sensors. It provides compliance and physical adaptation as seen in soft sensors, increase the contact area between the filter and the surface. However, when jammed, we do not see some of the more negative reduction in spatial and force resolution that can be seen in with soft sensors. As shown in Figure 2A we explore this by simulating the response from the fixed and adaptive filter for similar objects but with different, lower surface detail. In all cases we see that the Adaptive Filter leads to force transmission between the object and filter for the surface detail, so more surface information is captured. With the rigid sensor, the filter structure is not in contact with the surface detail of the object so gets no response from this detailing on the object. We also show these results quantitatively in Figure 2C left, were we consider the percentage difference in pixels of the FEA images for these different objects when seeing the fixed and adaptive filters. These results show that when used the fixed sensor, the objects have a low difference, appearing to be similar, however, the adaptive filter leads to far greater percentage differences in the responses, aiding classification or detection.

The second advantages of the filter is that it is possible to change the morphology of the filter online. This is achieved by “shaping” the filter when unjammed by forming the filter around a template and then jamming the filter to hold on to the shape. The morphology or structure of the filter can thus be chosen to aid in the task by inducing desired differences in the sensor response. For example, to offer improved discrimination between two objects that otherwise may appear similar to the sensor. Take two cubes, one slightly bigger than the other, which induce a very similar sensor response. By shaping the filter, we can introduce morphology into the filter which results in the two objects leading to a significantly different response when tested. This is highlighted in Figure 2B where example force profiles are demonstrated for a fixed sensor structure and a shaped filter. In Figure 2C right we also show these results quantitatively, by again considering the difference between the pixels of the FEA analysis. We see for the fixed filter the images have a low difference, however, for the shaped filter the difference is much higher, at over 60%.

In summary, changing the shape of the filter can induce the sensor response to be significantly different for the two objects in comparison to using only a fixed sensor structure. This could be viewed as the filter providing a physical memory, which is then amplified when it stiffens through the process of jamming.

The design of the template objects should seek to utilize these mechanisms to allow for optimal discrimination between objects. In this work we consider a small set of filters to demonstrate and explore the effectiveness of this approach. To explore the morphological adaptation we have a fully soft, hard flat and hard ridged filters. In addition, we have an “adaptive filter” where the template molds are the objects being tested themselves. This should aid discrimination as should act like a key and lock mechanism, where the adaptive filter for a specific object should “fit” best for the object on which it was made. These filter have been selected to highlight and explore the different mechanisms and advantages of this sensor filter, as indicated above.

To demonstrate the physical adaptation that can be achieved using the adaptive filter Figure 3 shows the deformed filter for various template objects. As shown in Figure 3 the filter takes on the shape of the surroundings, allowing for the texture and 3D shape of the filter to be adapted online.

We perform controlled experiments with a 6-DoF UR5 industrial robot arm. We manually teach the robot the location of the objects to palpate, and set the starting position of each experiment with the robot’s end-effector aligned normally to the surface of the object to touch. We control the arm in Cartesian coordinates at a approximately 125 Hz per second. Each experiment consists of two phases, i.e., sensor molding, and object touch.

The object molding phase is used to automatically induce a particular morphology onto the surface of the sensor filter. The filter is adapted online by pressing the sensor filter’s surface into the template object, to deform the filter, after which a vacuum is applied to the filter, activating the jamming mechanism, and the “memory” shape of the mold is maintained onto the filter’s surface.

The object touch phase consists of a controlled 5-s interaction between the sensorized end-effector and the object under touch. For each object to touch, the end-effector is controlled to move normally downward until a touch event is detected by the capacitive tactile sensor at its extremity. The touch event consists of a raise, in any of the 50 taxels, by more than 5% of their reading range. After contact is detected, the robot proceeds to perform a “rubbing motion”, by moving diagonally in the x-y plane of motion (Figure 4) for 5 s. From the detection of touch to the end of the 5-s motion, the tactile sensor is sampled at 50Hz. We thus retrieve a total of 250 tactile images for each experiment, each containing responses from 50 different taxels. This brings the dimensionality of each tactile experiment to a 12,500 dimensional vector.

The following section details the data processing and handling techniques which are used to analyze and reduce the sensory information gained during experiments.

We perform further experiments on the quality of the tactile information by training and comparing several machine learning algorithms on the data retrieved by the various filter modes. We pick 9 standard machine learning classifiers to perform this classification: Nearest Neighbors (Goldberger et al., 2004), Linear SVM (Mayoraz and Alpaydin, 1999), RBF SVM (Yan et al., 2006), Gaussian Process (Williams and Rasmussen, 2006), Decision Tree (Steinberg and Colla, 2009), Random Forest (Cutler et al., 2012), Feed Forward Neural Network (Rumelhart et al., 1986), Naive Baise (Schütze et al., 2008) and QDA (Hootman, 1992). All the models were implemented with the Scikit-learn library (Pedregosa et al., 2011). Given the purpose of this comparison, we fit each model with its default hyper-parameters from (Pedregosa et al., 2011), and do not perform any particular hyper-parameter tuning. To compare the results, we randomly select 70% of the lower-dimensional PCA projections of each tactile image data, and leave the remaining 30% for testing. This is performed upstream, thus all models will be compared on the same data splits. As no ad-hoc hyper-parameter fitting procedure was performed in each classifier, it was not necessary to select separate data-points for validation.

We explore a number of different classifiers to allow us to observe how each algorithm performs when given the data generated by each filter. Here we hypothesize that “higher quality” data should allow several standard algorithms to accurate classify each object under touch, while “low quality” data would make object discrimination harder, or impossible. This should be even more visible in lower dimensional projections.

In these first set of experiments we consider a set of similar sided “test” objects which have some variation of feature (stiffness, roughness, edges), to explore how the adaptive filter can assist in identifying specific objects. The analysis was performed on a set of 640 touch experiments, where 4 filters were used to touch 8 objects from the top, and each touch experiments was repeated 20 times. Moreover, the tactile sensor is sampled at 50 Hz over 5 s, where each sample is a 50-dimensional array of taxel values, bringing the dimensionality of each of the 640 data-points to 12,500.

To analytically investigate the results in Section 4.1.1, we can consider the Silhouette score; a measure of information structure directly dependent on the separation of object clusters in sensor space. We compute the Silhouette score across a varying number of PCA projected dimensions, so as not to show bias by our choice of dimensionality reduction. Figure 7A shows the Silhouette score of the sensor data as a function of the dimensionality reduction in PCA. First, we again notice that different filters provide a very different amounts of structure. As expected, both the Adaptive and Hard Ridged filters achieve the highest score across dimensions, with the Hard Ridged filter inducing more structure than the Adaptive Filter in two dimensions, but less so in all others.

This picture is also confirmed in Figure 7B, which depicts the explained variance for each principal component with varying filters. In the figure, we can see that by using the Hard ridged or Adaptive Filter we can retain over 60% of the variance with only one projected dimension, while we can explain over 90% of the total variance in the data with 4 or 5 dimensions in total.

Finally, we wish to verify if the observed influence of the filter on classification performance. We do this by comparing 9 standard classifiers with the data obtained by all filter shapes. The data is fit to each classifier as described in Section 2.4. Moreover, each classifier is trained separately with respect to the data generated by each type of filter, in turn. The random test and training splits for the data generated by a particular filter are kept consistent across all classifiers. Figure 8 shows the classification accuracy achieved with the data from each filter, as a function of the number of projected dimensions in PCA. As before, we do so to ensure there is no dependency between performance and dimensionality reduction.

We observe that the predictions loosely follow the estimations of information structure by the Silhouette score. The data generated by the Adaptive and the ridged filter can achieve state of the art results in 6 out of 9 classifiers, with the purely soft filter generally under-performing in all scenarios. It can be also noticed that the relative ordering loosely respects the silhouette predictions, where for a 2-dimensional projection the Stiff Ridged filters outperforms or equates the Adaptive filter with 8 out of 9 classifiers, while generally under-performing on other dimensions.

Table 1, illustrates the average accuracy performance of each of the 9 classifiers considered across the first 50 projected PCA dimensions. The first important consideration is that, on average the Adaptive Filter achieve the highest performance overall (99.49%) as well as the highest number of best performing classifiers overall (4/9). Moreover, depending on the classifier, it is possible to achieve up to a 24.37% increase in classification accuracy when considering an appropriate morphology. This validates the need of adaptive morphologies in classification tasks.

To demonstrate the generality of our approach, we perform test experiments on a set of 17 objects, which vary in shape, dimension, softness, texture and more (Figure 4C, Object Set 2). We pick the highest performing filters from previous experiments: the stiff Filter, the Hard Ridged Filter and the Adaptive Filter. The analysis was performed on a set of 510 touch experiments, where 3 filters were used to touch 17 objects from the top, and each touch experiments was repeated 10 times. Moreover, like before, each experiment is a 12,500 dimensional array of taxel values over a 5 s period.