Increasingly, there is demand for robotic systems to operate within variable unstructured domains, such as in autonomous exploration or alongside humans (Zou et al., 2017). This requires continually gathering detailed information about the local environment to facilitate executing the task in a safe manner. Tactile sensing at the external interface of robotic end-effectors enables a direct feedback loop to modulate applied force during interactions (Zou et al., 2017). The adaptability of soft tactile sensors enables the contact surface to conform to the shape of objects and distribute any handling forces over a larger contact area (He et al., 2020a).

In addition to the importance of tactile information when interacting within unknown environments, physical touch becomes highly important when other sensory information is unavailable (Tiwana et al., 2012). Despite research interest, commercial application of soft tactile sensors has been extremely low (Tiwana et al., 2012; Zou et al., 2017). Zou et al. (2017) highlight the high cost, low modularity and the advancements in micro manufacturing and software processing necessary to implement research sensors practically as key challenges to their adoption. By tackling these problems and working toward more generally applicable soft tactile sensor technology, we hope to broaden the range of applications for which robotic systems can be utilized and enable tasks, previously only possible with partial or total human intervention, to be completed autonomously.

A wide range of tactile sensing technologies has been developed with successful designs coming from both close biomimicry as well as deviating from nature to incorporate additional capabilities (Zou et al., 2017; Chi et al., 2018). One such deviation proposed in literature comprises adjusting the effective stiffness of the soft tactile membrane via pneumatic actuation in order to allow for the sensitivity and measurement range of the sensor to be adjusted, leading to a more generically applicable sensor (He et al., 2020a; Jenkinson et al., 2020; Zhang et al., 2021). Previous studies exploring pneumatic actuation within soft tactile sensors have applied this to reactive grasping (McInroe et al., 2018), shape identification (Huang et al., 2019; Xiang et al., 2019), estimating tissue elastic modulus (Gubenko et al., 2017) and an explorative capsule capable of self-locomotion (Hinitt et al., 2015). Variable effective stiffness tactile sensors have also been applied to emulate nodules inside phantom organs to assist with medical diagnosis training (He et al., 2020b).

A key challenge when incorporating sensors within soft robotics systems is to not alter the overall compliance of the robot’s external interfaces (Shintake et al., 2018). Optics-based tactile sensing is common across soft actuated sensors, as all electrical components can be physically separated from the tactile membrane (Shimonomura, 2019). Among the pneumatically actuated tactile sensors referenced above, only He et al. (2020a) and He et al. (2020b) use pneumatic variations as the primary form of sensing; all other studies chose to implement optics-based tactile sensing. These studies have focused on detecting surface or bulk features of the stimulus that they are sensing, where heterogeneity in the depth of the stimulus remains under-exploited as a tactile cue.

In this work, we explore the potential of pneumatically actuated soft tactile sensors though the development and experimental characterization of a novel sensor. We investigate the usefulness of varying the effective stiffness of the sensor’s tactile membrane when identifying nodules embedded in a soft medium, coarsely imitating the manual palpation tasks required for clinical breast examinations (CBEs).

CBEs serve a vital role in monitoring breast health and reducing the number of patient referrals to the more costly procedures of MRI or X-ray mammography, where many users report an unpleasant and painful experience (Bickley et al., 2013; Whelehan et al., 2017). These procedures are the current gold standard for medical diagnosis but are uncomfortable and their relatively high rate of malignant diagnosis of benign tissue (false positives), especially in younger women, can lead to patient anxiety and further unnecessary invasive treatment (Bickley et al., 2013). If data-led CBEs were to become a frequent and automated process, monitoring of breast health over time could be streamlined and anomalies identified through tactile sensing and machine learning. However, an automated breast examination system would require tactile sensitivity at least comparable to that of a trained healthcare professional performing CBE in order to achieve this (Tiwana et al., 2012).

Figure 1 depicts the proposed sensor along with its internal structure. The soft external membrane of the sensor deforms in response to tactile stimuli, as the internal pressure and motion of embedded tracking pins are monitored and analyzed. The tactile information is represented with the use of Voronoi diagrams as per the work of Cramphorn et al. (2018). We show that this approach is capable of resolving a broad range of tactile stimuli. We validate the sensor’s ability to distinguish surface lump of varying size and then characterize the sensor’s performance at detecting lumps embedded within a silicone matrix as the effective stiffness of the tactile membrane is adjusted.

The following sections focus on the design and fabrication of the compact, low-cost soft tactile sensor incorporating pneumatic actuation (Sections 2.1 and 2.2) and its characterization (Section 2.3). Section 3.1 presents a machine learning approach for identifying the presence of lumps utilizing features derived from Voronoi tessellation. The results demonstrate that the sensitivity of the sensor to identify lumps in a CBE-inspired task can be improved by pneumatically tuning the effective stiffness of the tactile membrane (Section 3.2). Section 4 discusses the results and makes recommendations for future work.

Incorporating pneumatic actuation within an optical based tactile sensor poses restrictions on the form the device may take: the sensor’s cavity needs to be pressurized and the image sensor must have unobstructed view to the tactile membrane. McInroe et al. (2018) tackled these same restrictions in the design of their sensor. Figure 2 shows a high level system diagram of such a system.

CBEs aim to identify anomalous hard lumps within a search domain of breast tissue. In the next two sections, we characterize the relationship between the presence of an anomalous lump and the area change and vector displacement of the 51 regions of the Voronoi diagram, in order to assess the performance of the presented sensor for lump detection. First, we consider the problem of detecting surface lumps, followed by considering embedded lumps, in which case this mapping becomes increasingly non-linear, due to the complex dynamics of elastic deformation.

Our proposed tactile sensor outputs a Voronoi representation of the contacted stimuli, which has a wide range of downstream applications. In this work, we used features extracted from the Voronoi diagram as inputs to a statistical machine learning model, trained for the task of lump detection. Experimental results demonstrated the sensor’s ability to successfully identify anomalous lumps located both on a flat surface and embedded within a silicone matrix. These results highlight the potential for this method of tactile sensing to be applied in medical diagnosis and surgery, where versatile soft sensing and actuation is required and use of harsher (e.g., x-ray) technology is not permitted. More widely, there is scope for application in other robotic tasks involving dexterous manipulation of delicate objects.

The sensor’s ability to resolve surface and embedded lumps was investigated on lumps ranging from 1 to 5 m m in diameter and 1 to 5 m m depth in a soft silicone matrix. The sensor was capable of both detecting and evaluating the size and depth of lumps across the full range of lumps tested. The ability of the sensor to detect surface lumps was reasonably uniform across the range of lump sizes, which suggests that satisfactory performance could also be obtained outside of the tested range. Future work will investigate the upper and lower bound of the sensor’s ability to resolve surface detail. Lumps embedded within a silicone matrix proved more difficult for the sensor to accurately identify. Lumps located closer to the surface and with a smaller diameter were easier for the sensor to distinguish. With a thinner layer of silicone between the firm lump and the tactile membrane, lumps embedded nearer to the surface caused a greater change in the detectable deformation of the membrane. When palpating smaller lumps, the change in tactile membrane deformation was more localized, and therefore easier to distinguish from the large scale deformation due to the soft matrix. The observed reduction in detection performance for larger diameter lumps is assumed to be due to constraining the experiments to central indentation. If a more complex surface scanning methodology were adopted, the edges of larger lumps could be detected as the surface is traversed. The accuracy of identifying both surface and embedded lumps increased with the normal indentation force exerted into the test slab. This effect was more pronounced in the variable depth trials than the variable diameter trials. This matches the assumption that a greater force would be required to compress the soft silicone above a deeper lump before its geometry is clearly identifiable.

In our experiments, pressure data were synchronized with image data and used to identify the frame corresponding to the point of maximum membrane indentation. However, this is far from the extent of the utility of the pneumatic subsystem within the sensor. Our proposed sensor can actuate its contact surface by varying its internal pressure, thereby changing the effective stiffness. Experiments were carried out with the internal pressure ranging from 0 to 200 m b a r . At 200 m b a r , the volume of the tactile membrane increased by 60 % . This potential for actuation of the contact surface may facilitate the use of other medical diagnosis techniques complementary to an automated CBE, such as nipple aspirate fluid collection (Shaheed et al., 2017). When detecting surface lumps, performance at different membrane stiffnesses was a function of indentation force. With light indentation force, a low-stiffness membrane, able to readily deform to the lump, performed significantly better than a high-stiffness membrane. This disparity in sensing performance across the stiffness range decreased as the indentation force was increased. At the highest indentation forces, where a certain internal pressure is required to resist larger scale deformation and remain conformed to the shape of the lump, the lowest membrane stiffnesses performed worse. The membrane stiffness also affected the sensor’s ability to detect embedded lumps. Noteworthy is the presence of minima in the prediction error against internal pressure curves for both the depth and diameter sensing tasks. This suggests that depending on the sensing task and the nature of the stimuli to be detected, different membrane effective stiffnesses can be considered optimal. With knowledge of the stimuli of interest, tuning the stiffness of tactile sensors could lead to improved performance or in cases where the stimuli to be detected is unknown, a methodology sweeping across effective stiffnesses while repeating palpations may be favourable. In future work, we will carry out further experimental investigation and look to characterize how the effective stiffness of the tactile membrane impacts sensing performance across a much larger and more varied sample size of trials.

The primary motivating factor behind averaging features axisymmetrically and constraining experiments to central indentation was the limited experimental sample size of approximately 4,000 observations for each experiment, which was small for a complex machine learning task. It also informed our choice of simple linear regression and Gaussian classifier models, which are suitable for identifying broad trends in small data sets while limiting overfitting. By increasing the size of the data set and training a more complex model, capable of learning the many non-linear behaviors inherent in soft body interactions, the performance of the sensor would likely improve, with no changes to the hardware. With more data and a more complex model, the number of training features could also be increased to remove the axisymmetric assumption. The additional spatial dimension would reveal information about the position and orientation of stimuli, and enable the modeling of more complex tactile interactions. Deep convolutional neural networks have shown promising performance in combination with raw optical tactile data (Lepora et al., 2018) and may provide a research avenue for improving sensor versatility in future work.

To the best our knowledge, our prototype sensor’s novel design combines optical marker-based tactile sensing with variable pressure in a significantly smaller footprint than any other device to achieve this in the literature. The sensor uses low cost manufacturing techniques and allows for the tactile membrane to be removed and replaced as a modular component. This addresses common concerns surrounding tactile sensors of high cost, low modularity, fragility and incompatibility with medical applications that require disposable contact surfaces (Tiwana et al., 2012; Zou et al., 2017). As the tactile membrane is modular and machine learning algorithms can be trained to new stimuli, future work will retain the proposed sensor design while investigating how changes to the thickness and material properties of the tactile membrane influence performance across different ranges of force applicable for clinical use.

Where previous studies have used the combination of tactile sensing and variable pressure in reactive grasping, shape identification and stiffness evaluation tasks, we utilize our control over internal cavity pressure to investigate the potential of such a sensor against a simplified mock-up inspired by the sensing requirements of CBEs. The mock-up encapsulates the challenge of CBEs of detecting firm lumps embedded within a softer matrix and as such provides a starting point from which future work can expand on exploring the role of variable compliance in CBE applications. The consistency of breast tissue varies widely (Bickley et al., 2013), so while no single breast model can be perfectly representative of all women, further experiments against a range of realistic stimuli will enable comparisons to be drawn between the sensor’s performance, human sensing ability and the results of previous tactile sensing studies (Egorov and Sarvazyan, 2008; Gwilliam et al., 2010; Mojra et al., 2012). Future work will test performance using medical breast phantoms involving non-flat surfaces, and followed by clinical trials. Testing will also be expanded to investigating stimuli other than lumps as well as exploring its potential in actively actuated palpation.