We utilized a musculoskeletal leg simulation to evaluate the proposed framework because a realistic nonlinear muscle model is available (Hill, 1938). The leg model was constructed on MATLAB/Simscape. The proprioception target state was a joint angle of the leg θ (Figures 2, 3). Soft sensors measured muscle length. Humans perceive joint angles through signals from muscle spindles within their muscles (Erin et al., 2016). Muscle spindles generate sensory signals via primary and secondary afferent fibers. They can be approximated as muscle velocity and length, respectively (Marques et al., 2014). Using the nonlinear musculoskeletal leg and soft sensor model for such proprioception enables simulation based on a reasonable model, and it is similar to soft robot proprioception with embedded soft sensors (Thuruthel et al., 2019). Additionally, the musculoskeletal leg model shares similar characteristics with soft robots, such as static and dynamic nonlinearity (Sugiyama et al., 2024; Polygerinos et al., 2017; Driess et al., 2018; Masuda et al., 2019) and the lack of a unique solution to achieve the desired robot state (i.e., motor equivalence problem) (Carpenter, 1968; Hirashima and Oya, 2016; Almanzor et al., 2023). Besides, prior studies on proprioception with a redundant sensor configuration also utilized musculoskeletal systems (Kawaharazuka et al., 2022; Thuruthel et al., 2020).

We modeled the sensors as soft resistive strain sensors (Souri et al., 2020). The model was built based on literature that characterized a soft resistive sensor made from a carbon nanocomposite elastomer (Muth et al., 2014). Note that it is one of the most commonly used materials for such sensors (Yamada et al., 2011; Thuruthel et al., 2019). Soft sensors typically exhibit nonlinear responses due to elastomeric materials, temporal nonlinearity such as response delay, and individual differences (Hegde et al., 2023; Terryn et al., 2022; Thuruthel et al., 2021; Sugiyama et al., 2021). Hence, we applied a second-order delay to the sensor length changes and modeled the strain-resistance characteristics with a quadratic function. Moreover, individual differences were implemented by randomizing the parameters required for these modelings. Finally, we added Gaussian noise for a more realistic simulation (Lin Y.-H. et al., 2023). Through this process, we simulated the typical response and temporal nonlinearities of soft resistive strain sensors made from carbon elastomer. These nonlinearities are also observed for the other sensor materials, such as liquid metal and hydrogel (Park et al., 2012; Shi et al., 2021; Lu et al., 2020; Guan et al., 2023; Cai et al., 2017; Shen et al., 2022; Xu et al., 2019; Wang et al., 2016).

The musculoskeletal leg was actuated with motor babbling data. This process involves the iteration of ramping up/down and holding of input to allow the leg to explore its entire range of motion (Almanzor et al., 2024). The knee joint angle θ , muscle activation signals a , and muscle length l were recorded at a sampling frequency of 10 Hz. Then, each muscle length data was captured as sensor resistance values R . Consequently, the sensor sampling frequency was 10 Hz. During all the experiments, the resistance values obtained were processed using a low-pass filter with a cutoff frequency of 3 Hz. We generated 400 s of training data for the TDFNN and 1,600 s of training data for the LSTM. The training data were normalized and split into training and validation datasets in a ratio of 3:1.

The TDFNN dataset was augmented by zeroing all raw sensor resistance of randomly selected sensors. For each muscle, ten files containing the same 400 s of sensor resistance data were generated. Then, for each file, 0%, 10%, …, up to 90% of sensors were randomly selected and zeroed (i.e., masked). Since four muscles were involved, there are 2 4 possible combinations whether each muscle contains masked sensors or not. Thus, the ten masked files of each muscle were combined. This process was followed by concatenating them with the corresponding masked or unprocessed sensor data from the other muscles. The concatenation was performed in 2 4 − 1 combinations, excluding the case where no muscle contained masked data. As a result, the TDFNN dataset was composed of 10 × ( 2 4 − 1 ) = 150 files with different combinations of masked sensors.

The stochastic LSTM had one LSTM layer with a size of 1,600 and a fully-connected layer for final output calculation. The mini-batch size was 8. ADAM optimizer with a learning rate of 5.0 × 1 0 − 5 was utilized. During the training process, the LSTM was trained using 150 steps of sequential data extracted from the dataset. The leg (muscle) and sensor responses included delay that can be modeled as first-order and second-order delay systems, respectively. The maximum time constant of muscle activation was 1.2 s, and the settling time of sensors was 2.0 s. Thus, the sequence length was set to 150, which is sufficiently higher than the total response delay (3.2 s). As a result, the LSTM effectively learned time-series characteristics between muscle activations and sensor responses. The loss function was calculated only with data at t ≥ 50 . Typically, an initial sensor value is essential for the calculation of μ ̂ t and σ ̂ t 2 from the control input history. However, elasticity, common among soft actuators, gradually decreases the effect of their initial state. Therefore, by learning from data where the effect of the initial state was assumed to be sufficiently minimized, the LSTM was able to achieve the accurate estimation of μ ̂ t and σ ̂ t 2 without the initial state. The calculation threshold was set to 50, which was higher than the total response delay.

Regarding the TDFNN, the 1-D convolution and max pooling were carried out separately for each muscle and sample time steps. The kernel size was 3 with a stride of 1, and the output channels were 3. The pooling size and stride were 3. The max pooling results were concatenated into a vector and input to the fully-connected layer. The number of layers was 3, each with 500 hidden neurons. The input sequence length was set to 20 based on the sensor settling time of 2.0 s. The batch normalization layer was applied after the 1-D convolution layer to avoid overfitting. The mini-batch size was 512. ADAM optimizer with a learning rate of 1.0 × 1 0 − 4 was employed.

First, we evaluated the fault detection and graceful degradation capabilities of the framework under diverse sensor degradation. We conducted five 100-s simulations using motor babbling inputs. For each trial, we generated five evaluation datasets: Normal (i.e., baseline), Lost, Stretch, Offset, and Deviation. These four degradation scenarios were designed to simulate diverse failure modes of soft sensors based on the literature. They effectively represented the fragility of soft sensors for framework evaluation. In each scenario, 30% of the sensors were randomly selected, and their readings were distorted as follows:

• Lost: Readings from subjected sensors were zeroed to simulate simple sensor loss (e.g., sensor rupture) (Lin Y.-H. et al., 2023; Thuruthel et al., 2019).

In addition to this Separate dataset, we prepared a Consecutive dataset to evaluate the framework under actual deployment conditions, where degradation occurs consecutively during a single leg movement. To create the Consecutive dataset, we processed 100-s sensor readings, randomly selecting 30% of the sensors every 20 s to represent one of five states. The states followed the order: Normal, Lost, Stretch, Offset, and Deviation. We utilized the same five 100-s simulation data used for the Separate dataset. During the evaluation with the Consecutive dataset, the LSTM hidden states were not reset.

The stochastic LSTM and TDFNN were trained for 75 epochs and 30 epochs, respectively. Overfitting did not occur for both networks. Once trained, the networks did not experience any further retraining or intervention. Figure 5 shows the average root mean squared error (RMSE) of proprioception for five trials. The blue bar describes the result when all sensor data were directly input to the TDFNN without being zeroed. Table 2 lists the corresponding RMSEs. Note that for the calculation of the RMSE, the first 50 steps of the results were ignored as the LSTM training process did not use data at 0 ≤ t ≤ 50 . The results labeled as w/o Fault Detection represent the framework’s performance without fault detection. These results effectively simulate existing methods for soft sensor proprioception that do not include fault detection (see Section 1 for details). These results serve as a meaningful proxy for the evaluation with existing methods.

As shown in Figure 5, the proposed framework demonstrated graceful degradation and maintained accurate proprioception across all degradation scenarios and dataset types. For the Separate dataset, the average RMSEs remained nearly constant at 6.44 ° , 7.50 ° , 7.03 ° , and 6.71 ° for Lost, Stretch, Offset, and Deviation scenarios, respectively. Similarly, the Consecutive dataset resulted in a consistent trend, with RMSEs of 6.50 ° , 6.77 ° , 6.04 ° , and 6.78 ° for the corresponding scenarios. The differences in sensor degradation resulted in RMSE variations of only 1.06 ° and 0.74 ° , validating the effectiveness of the framework. In addition, Table 2 shows that the RMSE increase reached a maximum of 18.0 ° without fault detection, highlighting its importance. Figure 6 shows an example of proprioception conducted with the Consecutive dataset. Even if the degradation type was switched every 20 s, the proprioception RMSE was 5.92 ° , almost identical to the baseline RMSE of 5.24 ° . Figure 6B confirms that the LSTM performed precise fault detection with accurate sensor signal estimation. False positives at the beginning of the estimation followed by large σ ̂ t 2 are due to the LSTM training process. The LSTM could not perform estimation when the effect of the initial state did not decrease sufficiently. Subsequent small false positives of the RF sensor are due to noise.

Next, we evaluated the accuracy retention capability of the proposed framework, increasing the number of degraded sensors. Using the 100-s simulation data described in Section 3.1, we generated ten datasets for each trial by applying Offset degradation. In each of the ten datasets, 0%–90% of randomly selected sensors were degraded.

Figure 7 and Table 3 presents the average proprioception RMSEs with different percentages of degraded sensors. The proposed framework tolerated the degradation in 50% of all sensors, with an average RMSE increase of only 3.30 ° . In contrast, without the fault detection component, even the 20% degradation led to the average RMSE increase of 21.3 ° , demonstrating the effectiveness of the proposed framework. When the percentage of degraded sensors exceeded 60%, the proprioception RMSE began to rise gradually. This increase is due to the lack of features for proprioception, as an input vector to the fully-connected layer contained more zeros. However, it is noteworthy that the RMSE increase was only 6.97 ° with the 80% loss of the sensors.

Finally, the scalability of the proposed framework was evaluated. We prepared two additional models for the investigation, the RFLB and 6-muscles model (Figure 3). The RFLB model was developed as a proprioception target, incorporating both different actuation and sensor morphology. The 6-muscle model was designed as a proprioceptive target with significantly higher nonlinearity. As described in Figure 3, the RFLB model only included the RF and LB muscles to actuate the knee angle, while the number of sensors per muscle was doubled. On the other hand, the 6-muscles was a 2-DoF system with two states to perform proprioception (the hip and knee angle). Six muscles interacted with each other for the actuation. Moreover, the RF and LB muscles acted as bi-articular muscles. Consequently, the 6-muscles model had much higher system nonlinearity than the original model.

The same evaluations as in Section 3.1 were conducted using these two models. New networks were prepared and trained for each model. The stochastic LSTM/TDFNN were trained for 75/25 epochs for the RFLB model and 10/10 epochs for the 6-muscles model to avoid overfitting. The training data collection process, the other network’s parameter settings, and the evaluation procedure were the same as the original model, except for the TDFNN dataset for the RFLB model. We prepared three additional datasets to train the TDFNN effectively. Each dataset was created following the same procedure as the original model. Since there are 2 2 combinations whether each muscle contains the degraded sensors or not, the TDFNN dataset consisted of ( 1 + 3 ) × ( 10 × ( 2 2 − 1 ) ) = 120 files with different healthy/zeroed sensor combinations. Note that the TDFNN dataset for the 6-muscles model contained 10 × ( 2 6 − 1 ) = 620 files because the use of six muscles resulted in 2 6 combinations of whether each muscle contains zeroed sensors or not.

Figure 8 shows the evaluation results with the RFLB and 6-muscles model. Table 4 presents the corresponding proprioception RMSEs. For the RFLB model, while the baseline RMSE showed a slight increase, the proposed framework demonstrated graceful degradation comparable to that of the original model. In each scenario, the average RMSEs were 8.30 ° , 9.74 ° , 9.13 ° , and 9.08 ° for the Separate dataset and 10.0 ° , 9.00 ° , 10.3 ° , and 8.83 ° for the Consecutive dataset. RMSE variations remained within 1.44 ° or 1.47 ° . The maximum RMSE increase was only 2.22 ° in the Stretch scenario with the Separate dataset. With the 6-muscles model, reasonable proprioception RMSEs were maintained for all scenarios, with only a minimal increase in baseline RMSE. Across the four degradations, the average RMSE only changed by no more than 2.29 ° (hip joint) and 2.79 ° (knee joint) for the Separate dataset and 3.04 ° and 1.78 ° for the Consecutive dataset. Notably, the maximum RMSE increase was just 2.99 ° for the knee joint angle (Offset, the Separate dataset). While the Consecutive dataset resulted in prominent RMSE increases compared to the other models, the maximum increases were still limited to 4.49 ° and 3.20 ° (Deviation) despite the significant nonlinearity of the 6-muscles model.

In this paper, we propose a novel learning-based graceful degradation framework for redundant soft sensor systems. For the first time, the proposed framework realizes graceful degradation for soft sensor proprioception against diverse sensor degradation scenarios.

We evaluated the proposed framework using a simulated musculoskeletal leg with soft sensors based on sufficiently nonlinear models. The soft sensor model was sufficiently reliable as it approximated the behavior of an experimentally verified soft sensor in the literature. This simulation allowed precise control over actual sensor degradation to evaluate the framework effectively. As a result, the experimental results demonstrated the framework’s excellent capability for graceful degradation, providing a solid understanding of its general behavior and performance. As shown in Figures 5, 6, 8, the framework adapted to four different sensor degradation scenarios and adequately retained the original proprioception accuracy. The fault detection with the stochastic LSTM was precise and essential for the framework’s graceful degradation. Under the Lost scenario, RMSEs were similar to those without fault detection as the scenario zeroed the values of affected sensors, resulting in identical original and processed sensor readings. In contrast, turning off the fault detection led to a significant RMSE increase in the Stretch, Offset, and Deviation scenarios. Particularly in the Offset scenario, sensor reading amplification due to degradation reduced proprioception accuracy. This is attributed to the max-pooling layer in the TDFNN, which caused amplified resistance values to affect the proprioception process directly. In the stretch scenario, the RMSE increase occurred for the same reason, while the minor amplification of sensor responses had less impact on the proprioception. In the Deviation scenario, RMSE increases were smaller than in Offset because the degradation both amplified and reduced sensor responses. When sensor responses were reduced, the max-pooling layer could exclude the affected readings from being input to the TDFNN, depending on the state of the proprioception target. Owing to the framework’s architecture, the proposed method successfully addressed both amplification and attenuation of sensor response. As a result, versatile graceful degradation was achieved across all the sensor failure modes. Additionally, Figure 7 shows noteworthy results that the framework could tolerate the deactivation of up to 80% of the constituent sensors. Furthermore, the experiments also revealed the scalability of the framework. Even when the proprioception target’s morphology and sensor configuration changed (RFLB model) or the system’s nonlinearity significantly increased (6-muscles model), the framework achieved graceful degradation across all degradation scenarios. Although small RMSE increases were observed for the evaluation with the 6-muscles model (the Consecutive dataset), this was due to the LSTM underfitting. The additional hip joint and bi-articular muscles of the 6-muscles model significantly complicated the forward healthy sensor model learned by the LSTM. Nevertheless, we did not change the hyperparameters and the amount of the LSTM dataset for comparison. As a result, the range of ± 3 σ (Equation 1) expanded, and false negatives impaired the accurate proprioception of the TDFNN (Supplementary Figure S2 in the supplemental material). Thus, fine-tuning the hyperparameters, such as Bayesian optimization (Akiba et al., 2019), will enhance the framework’s scalability.

Our work is distinguished from other existing research by its capability to tolerate diverse soft sensor degradation and availability for proprioception. Soft sensors have been widely modeled using learning-based approaches, and researchers have realized graceful degradation for soft sensor proprioception (Thuruthel et al., 2019; Wang et al., 2023). Researchers have also performed graceful degradation for soft sensor exteroception (Shih et al., 2020; Lo Preti et al., 2022; Dingley et al., 2023) and multimodal sensory systems (Zambelli et al., 2020; Chen et al., 2021; Lee et al., 2021; Liu et al., 2017; Zhi-Xuan et al., 2020; Wu and Goodman, 2018). However, these methods were not equipped with a fault detection component, and the evaluation scenarios were limited to complete sensor loss. On the other hand, soft sensors undergo various degradation due to their softness and nonlinearity (Terryn et al., 2021; Khatib et al., 2021; Porte et al., 2024; Terryn et al., 2022; Shen et al., 2016). Thus, direct input of distorted sensor readings will impair proprioception accuracy. Our framework employed fault detection based on the stochastic LSTM and tolerated diverse sensor degradation. The comparison with w/o Fault Detection results highlights significant improvements in our framework over existing approaches, particularly in Stretch, Offset, and Deviation scenarios. In addition, our complete framework also improved RMSE performance in the Lost scenario compared to existing methods, even though the evaluation setups were not identical. Specifically, our evaluation showed an RMSE increase of only 0.85 ° (0.61% of the leg joint’s range of motion), whereas existing literature (Thuruthel et al., 2019; Wang et al., 2023) reported RMSE increases exceeding 2.1% with 5% less sensor loss.

To our knowledge, no study has realized this capability for soft sensor proprioception. Fault detection has been applied to soft tactile sensing (Lo Preti et al., 2022) and multimodal sensing (Lee et al., 2021); however, signal-based or reconstruction-based fault detection utilized in these works is not suitable for proprioception. Due to the non-unique mapping illustrated in Figure 1, these approaches cannot determine whether the variation in sensor readings is caused by the deformation of the proprioception target or by sensor degradation, especially when the distortion in sensor readings is indistinct. In contrast, our LSTM-based fault detection avoided such a situation due to non-unique mapping by comparing sensor readings with the estimated responses of healthy sensors. Consequently, our framework achieved fault detection for proprioception against diverse sensor degradation. Furthermore, this fault detection and the subsequent zeroing process eliminated the need for seminal characterization and inclusion of all possible degradation to prepare the training dataset, typically required by the reconstruction approach (Lee et al., 2021; Zambelli et al., 2020). Despite the various types of degradation that soft sensors exhibit, our framework only requires its users to augment the obtained data by randomly zeroing sensor readings.

In summary, soft sensors are subjected to various types of degradation during operation. To address the degradation, the proposed graceful degradation framework for redundant soft sensors achieved consistently reliable proprioception. Unlike self-healing soft sensors, this framework realizes instantaneous adaptation to degradation without downtime or reduced sampling frequency. This capability allows soft robots to maintain proprioception under real-world deployments, where they experience repeated large deformations and various sources of damage. Due to their softness and nonlinearity, consistently reliable proprioception feedback is crucial for soft robots to retain performance. Therefore, the proposed framework will contribute significantly to enhancing the robustness of soft robots, maintaining their intelligent autonomy in real-world applications (e.g., soft grippers and rescue robots).

Our framework has three main directions for improvement: (1) adapting to damages and disturbances affecting the proprioception target itself, (2) evaluating the framework with an actual soft robot, and (3) further simplifying the training process.

With regard to the first point, the framework assumes that the proprioception target remains unaffected by damage to its body or external loads. The damages and disturbances can lead to sensor readings deviating from the estimated healthy values. As a result, all sensors are incorrectly identified as degraded, and proprioception accuracy will decrease. We will address this limitation by implementing a meta-voting algorithm that cancels sensor deactivation when the number of simultaneously degraded sensors exceeds a threshold. We conducted a preliminary experiment to investigate the effectiveness of this approach. While motor babbling input randomly actuated the Original model, we fixed the foot position (i.e., knee joint angle). We modified the proposed framework to cancel zeroing sensor readings if more than 50% of all sensors were simultaneously detected as degraded for five consecutive steps. As shown in Figure 9B, the LSTM output incorrect healthy sensor estimates after fixing the foot. Nevertheless, the FNN maintained accurate proprioception because the meta-voting canceled the incorrect sensor zeroing (Figure 9). In contrast, all sensors were zeroed without the meta-voting, and proprioception accuracy fell. We will further modify this algorithm so that the proposed framework can address the damages and external loads to a proprioception target. Furthermore, we will explore the use of tactile sensors in this modification (Lo Preti et al., 2022). By incorporating exteroceptive sensor data, the LSTM can account for the impact of damage and external loads on healthy sensor readings. As a result, the LSTM may directly compensate for these effects.

For future work (2), while we used a sufficiently nonlinear model in the simulation, we excluded hysteresis and drifts to simplify the modeling. Thus, evaluating the framework with an actual soft robot will highlight the framework’s practical effectiveness. Note that the simulation already incorporated temporal nonlinearity as a second-order delay. Additionally, the proposed framework consists of the LSTM and TDFNN. Hence, the framework can handle these nonlinearities through hyperparameter tuning.

Finally, regarding the third point, we augmented the training dataset by randomly zeroing sensor readings. However, this step can potentially be omitted by applying dropout before the 1-D convolution layer during training (see Figure 2). This minor adjustment will further simplify the training process.