CGM data were collected at the Complejo Hospitalario Insular-Materno Infantil de Las Palmas de Gran Canaria from 41 people with T1D that use a sensor from the FreeStyle Libre (Abbott) family. Additional data, such as physical activity, insulin administration, or carbohydrates, were not available for any of the subjects involved in this study. The raw data were extracted from the LibreView^TM (Abbott) website by an endocrinologist in raw Comma-Separated Values (CSV) format. Participants were invited to participate and were given oral and written information. All participants signed a written informed consent form. Individual's data were stored on a secure server only accessible through a virtual private network by people working on the WARIFA project (31) after being given individual access credentials.

Supplementary Table S2 summarizes each subject's more relevant information regarding AI processing associated with an anonymous ID. To train DL models, a sufficient number of samples for each subject is needed. Among the 41 subjects there were cases in which this requirement was not fulfilled, due to sensor malfunction or due to multiple sensor replacements in a short period of time. Moreover, different sampling periods in the sensors could imply variations in the considered architectures. Thus, only the subjects that met the following inclusion criteria were considered in this study: 1.

To have at least 1 year of CGM readings with the same sensor. Sensor malfunction (e.g., reading interruptions) was considered and further treated to conform the instances to train and validate the DL models.

The data curation phase was performed by filtering out the subjects that did not meet the inclusion criteria previously described. The framework was executed one time per included subject (i.e., 29 times) using the first recorded year of each subject. At this point, there was a unique sequence of CGM readings for each subject with their associated timestamps to which a pre-processing step was applied. At that point, data were still not suitable for the training of the DL models.

Then, an automatic and parametric subject-per-subject sequence-to-sequence (from now, seq-to-seq) (i.e., an input sequence and an output sequence) dataset generation was developed (Figure 1B). The considered parameters were the window length, denoted as N (96 in this work), the step from one sequence to the next one (set to one in this case), and prediction steps, which is the result of the division between the Prediction Horizon (PH), i.e., how far ahead the model predicts, and the sensor sampling period. For this case, this parameter was set to four and two (60- and 30-min PH/15 min sampling period, respectively). Notice that, with this approach, there is a trade-off between the window input length and number of training instances; a larger input length will likely provide more information to the DL model, but fewer training instances will be generated, potentially limiting the prediction performance.

Firstly, the time intervals between two consecutive readings were computed considering the sensor timestamps associated with all readings. Although the nominal sampling period of all the sensors included in this work is 15 min, the actual interval between consecutive samples may vary from this reference value (Supplementary Figure S10). Therefore, to be compliant with the assumption of data provided in periodic time slots, two samples are actually consecutive when the time between them is lower than twice the sampling period (i.e., 30 min). As depicted in Supplementary Figure S6, it is assumed that, if the sensor reading is delayed or advanced in 2 × sampling period minus 1 s (i.e., 29:59 min), that reading corresponds to the next timeslot.

Every time that two consecutive samples in the CGM sequence surpassed 29:59 min, a new data block was created. Therefore, the number of blocks depends on the number and location of interruptions in the data (Figure 1B). Furthermore, the minimum number of consecutive samples to form a usable block (i.e., to form an instance suitable to train the DL models) is equal to the length of the input sequence plus the length of the output sequence. Otherwise, those samples are discarded and will not be used to generate and train the AI models. Notice that the ideal case would be to have only one data block without interruptions. After the data block generation, each data block is swept into steps of one to generate the model inputs (X) and the associated outputs (Y). Thus, block length determines the number of instances a particular block provides. After the sequences were generated from all data blocks, they are concatenated as instances to perform the min-max normalization as described in Equation 1.(1)inorm=i−SminSmax−Smin,where i represents a given sample inorm is the normalized sample, and Smax and Smin the global maximum and minimum values of a subject's data.

Additionally, there were a few cases where the CSV file did not follow LibreView's standard way of presenting the data. In such cases, glucose concentration values were considered corrupted, and thus treated as interruptions.

ISO 15197:2015 (32) establishes two specific criteria for glucose concentration monitoring systems, both based on the discrepancy between measured values and their reference counterparts. Although this standard was developed for glucose sensor measurement errors, its criteria can be extrapolated to evaluate errors between the actual glucose concentration and a value predicted by a model, tailoring the prediction problem assessment to the glucose monitoring context. AI-based models, such as those proposed in this work, are often trained with the aim of minimizing standard loss functions based on squared or absolute errors. Although theoretically both approaches lead to the same optimal solution, in practice a better result is expected if the loss functions are adapted to the required validation performance. A customized loss function has therefore been designed to enforce compliance with the criteria specified in the ISO standard.

Firstly, the two validation criteria, i.e., the one referring to the acceptable error zone in terms of accuracy and the one attending to regions A and B in the Consensus Error Grid [CEG—also called Parkes Error Grid (PEG)] (24, 26), have been analyzed by comparing the maximum admissible error values for each case. As shown in Supplementary Figure S7, the former criterion is the most restrictive except in the narrow range of reference values from 164 to 174 mg/dl, where the maximum admissible error is established by the boundary of region A in the CEG. Hence, it could be assumed that error values of 15 mg/dl or 15% of the reference value, depending on whether the actual concentration is less than 100 mg/dl or not, respectively, define the upper bound of the admissible error.

The strategy for designing an error function explicitly considering this upper bound is to force a barrier in the form of a steep slope of the function in the neighborhood of the threshold value. This can be achieved by introducing an additive term to the quadratic error that can be considered negligible (ideally zero) in the admissible region but large (ideally infinite) elsewhere. Nevertheless, the desirable properties of the error function must be preserved to facilitate the optimization process of AI model training, specifically its increasing monotonicity with the magnitude of the error and its smoothness. It is inspired by the design of low-pass filters, where the constraints are similar, with a low attenuation in the admissible region and a cut-off value beyond which the attenuation is high. Therefore, the additive term γ shown in the following Equation 2 is proposed:(2)γ(ϵ)=Kϵ2nwhere the variable ϵ refers to the normalised error in the prediction of a sample, i.e., the error itself if less than 100 mg/dl or the error multiplied by 100r, being r the reference value associated with that sample, and K is the proper constant to conform the transition in a way that the term remains below a prescribed tolerance in the admissible region and surpasses an established threshold for large error values.

The overall expression for the error function (LISO), including the quadratic error and the additive term γ is detailed in Equation 3.(3)LISO(ϵ)=ϵ2+γ(ϵ)=ϵ2+Kϵ2nFigure 2 highlights the different behavior of LISO with respect to the squared error around the 15 mg/dl limit prescribed in ISO 15197:2015. Its steeper slope enhances the reduction of the error for non-compliant samples compared to those already within the acceptable range.

This function should be considered as a modified version of the quadratic error of a sample and therefore allows composing loss functions for collections of samples by addition or averaging as well as the use of its square root to work in the same units in which the error is defined (mg/dl in the case of interstitial glucose concentration prediction). In this work, n has been set to 40, and K set to 0.1/14⁸⁰.

Figure 1 shows the proposed framework to develop a personalized AI-based DIY module for interstitial glucose forecasting for people with T1D. It has been developed in Python 3.10.13 (33), using TensorFlow 2.10.0 (34) for the development of the DL models. This work was developed using an AMD Ryzen 5 3600 6-core processor and an NVIDIA GeForce RTX 4070 Ti GPU to speed up the training process of the large number of models that were generated. The choice of this approach was based on the following ideas: a.

the robustness of the DL models in the training and validation phase, considering different groups of three months within a year to train and validate the models.

For the interstitial glucose forecasting, a seq-to-seq approach (35) was proposed due to its proven effectiveness in time series forecasting (36). In the seq-to-seq approach, the input of the model is one sequence (or more) of a certain length, and the model outputs a sequence of a certain length (usually shorter than the input for time series forecasting) that represents the subsequent time instants in the nearest future.

As a first approximation, the input sequence length was set as to 96 samples [i.e., 24 h at a sampling period of 15 min, which corresponds to an average period that includes basal glucose regulation mechanisms (37)]. This was established considering that all CGM sensors included in this study had a sampling period of 15 min, based on clinical knowledge, subject's autocorrelation studies, and considering the trade-off between the input length and the number of generated training instances. As a second input, the first difference of the CGM input window was computed to feed the DL models with additional information about CGM variations. Besides, two different PHs, have been assessed: 30- and 60-min PHs. Having a sampling period of 15 min implies an output sequence of two and four samples, respectively.

The proposed DL models were based on CNNs (38), and LSTMs (39), due to their promising results reported in time series forecasting, and specifically, in glucose levels forecasting (24–26, 40). Three different DL models, together with a baseline model, have been evaluated in this work: 1.

Naïve: For the sake of comparison, a naïve forecasting (24) approach was included in the study. It consists of making the prediction by outputting the last values of the input sequence. This elementary prediction is considered the baseline to compare the performance of the proposed DL-based models.

Adam optimizer was used to train the models, using the standard Mean Squared Error (MSE) as the loss function, as well as using the novel LISO loss previously described. Due to the extensive computational time associated with the large number of generated models in this experiment, early stopping was implemented after two epochs without reducing the loss function. The delta, (i.e., the minimum change required in the loss function value to consider a model improvement) was set to 0.0001 for both loss functions aiming at a fair comparative. This delta was considered the optimum after heuristic experimentation. In the same way, it was noticed that the models performed better when not only the CGM was considered as input, but also the CGM's first difference. Experiments also showed that introducing the second difference worsened the prediction performance. Hence, the DL models had two input features: the CGM and its first difference.

Finally, considering that hypoglycemic events are a severe health threat, the output sequences that contained samples in the hypoglycemic range were weighted the most to train the DL models, followed by the output sequences containing samples in the hyperglycemic range. Sequences that were within the target CGM range were not weighted beyond considering its occurrence including its probability in the equation. Besides, the occurrence of hypoglycemic samples is lower than the hyper- and normal CGM ranges. This together with the fact that such occurrence varies among subjects, the abovementioned weighted factor was multiplied by the probability of a given sample to be in range, or in the hyper- and hypo- ranges. This introduces the models' personalization from the training process. Depending on the number of samples in the different ranges, the weighting varies. Generally, the samples to train the models were weighted as described in Equations 4–6 for output sequences that contained hypo-, hyperglycemic samples, and samples that were in the target range, respectively. The weights multiplying the inverse of the probability of occurrence were obtained heuristically.(4)Whypo=2x1/p(hypo)(5)Whyper=1.1x1/p(hyper)(6)Winrange=1/p(inrange)Where W represents the weights for a given subject's hypoglycemic (Whypo), hyperglycaemic (Whyper) and in-range (Winrange) sample, and p refers to the probabilities of a given sample to belong to the hypoglycemic range p(hypo), hyperglycaemic range p(hyper), and an in-range sample p(inrange). Notice that, for each subject, the probability is computed as the number of samples in each range divided by the total number of samples.

From the set of sequences collected, the data partition was conducted as depicted in Figure 1C. To analyze if the performance of the different AI models depends on the period of the year used to train them, which can be related to sensor misfunction, holidays period, etc., the models were cross-validated using a 4-fold approach, where each fold contained data from a consecutive 3-month period. Hence, a 9-month period and a 3-month period were used for training and validation, respectively, for each fold. For every subject, the oldest timestamp marked the beginning of the first fold. The second, third, and fourth folds were generated adding 3, 6, and 9 months to the first timestamp, respectively. Notice that there were some instances that contained data from different days, and subsequently, from different months. For the sake of consistency, borderline instances that contained data from different folds were discarded for cross-validation. Furthermore, due to the heterogeneity in the sensor reading interruptions, there were folds with more instances than others. This could have an impact on the final regression performance of the models between folds from the same subject. Finally, every fold was shuffled before feeding the DL models.

Notice that every architecture was trained and validated with the standard MSE and the LISO loss functions, and this was performed for the 4 folds. Moreover, the same methodology was applied for two PHs: 30 and 60 min. Hence, and taking into account that the naïve approach does not need to be trained, for each subject, 48 AI models (3 architectures × 2 loss functions × 4 folds × 2 PHs) were generated. Considering the 29 subjects, 1,392 personalized AI-based interstitial glucose forecasting models were generated and evaluated in this work. This large number of models strongly limited the number of experiments that could be performed following this approach.

Commonly, prediction models for interstitial glucose level forecasting are evaluated using classical regression evaluation metrics, such as Root Mean Squared Error (RMSE), Mean Absolute Error (MAE) or Mean Absolute Percentage Error (MAPE), described in Equations 7–9, respectively, where y^i is the forecasted value, yi the corresponding ground truth, and N the number of considered values. In this work, they have been also used. However, only a few studies in the literature report the results achieved using the CEG and, to the best of our knowledge, none of them uses the complete ISO 15197:2015 (32) criteria for such evaluation since it is a highly restricted evaluation metric.

ISO 15197 is an international standard on in vitro diagnostic test that addresses self-testing blood glucose monitoring systems for managing diabetes mellitus within the realm of in vitro diagnostic test systems. These systems are designed for self-measurement by individuals without specialized medical training for the purpose of managing their condition. This standard, which has been adapted from the sensor accuracy measurement to the error between the prediction and the actual value in this work, as mentioned before in the text, establishes that the blood glucose monitoring systems (in this work, extrapolated to prediction error), must meet these two criteria: •

95% of the measured (predicted) glucose values must be within ±15 mg/dl for blood glucose concentrations below 100 mg/dl. For values equal or greater than 100 mg/dl, the margin of error is percentage and is set at ±15%. This is illustrated in Supplementary Figure S8a.

After performing the personalized AI-models development and validation, a Python application was developed to encapsulate the DIY framework in a usable module. Additionally, the Docker tool (46, 47) was used to containerize the app, enabling its use in different Operating Systems (OSs), including those used in smartphones. In this work, this module was validated in Windows 10 and Ubuntu LTS 24.04, to prove that this module was OS-agnostic. Only the LSTM for 30-min PH was implemented in this stage, since it was the model with the shortest training times, and this was sufficient to test the DIY module functionality.

The design of the application aimed to increase the users' empowerment (30) regarding their data and the AI models generation using them. Considering the inclusion criteria, the preprocessing and the obtained results regarding prediction performance of this work, there are cases where the implementation of this AI-based framework is not feasible, e.g., to provide less than a year of data to generate the model, the fact that such year of data does not provide enough samples to properly train the DL architectures, etc. All this information will be prompted to the users when needed. The users will also have the option to skip this information in case they are not interested. Four main scenarios were considered for the design of this tool. A key point is that all these scenarios are triggered by the user by calling the application. In the current stage, this means uploading their CGM sensor data to the DIY module. The possible scenarios that the tool contemplate are: 1.

User's first use providing enough data. The DL models are generated and training following a 4-fold cross-validation approach. The best model is selected and locally stored to be the one that will provide the personalized CGM predictions.

Figure 3 shows the comparison of the prediction performance when training the LSTM, Stacked-LSTM and Dil-UNet models with LISO (green) and the standard MSE (grey). Notice that the naïve model does not require any training and hence any loss function. The upper part of Figure 3 illustrates the metrics for 30-min PH, and the lower part for 60-min PH. This evaluation was performed in terms of RMSE, (Figures 3a,f), MAE, (Figures 3b,g), MAPE, (Figures 3c,h), ParkesAB, (Figures 3d,i), and ISOZone, (Figures 3e,j). They comprise the results obtained for the included 29 subjects. Notice that for RMSE, MAE, and MAPE, the lower their value, the better the prediction performance, zero being the ideal value. Conversely, ParkesAB and ISOZone are percentage metrics, where the ideal case would be to have 100% of the points within the acceptable zones. Nonetheless, a ParkesAB value of 99% and an ISOZone value of 95% are compliant with the ISO 15197:2015 standard (32).

These results show that the proposed LISO training provided a similar prediction performance than the standard MSE for all models and for both PHs. Both LISO and MSE showed higher variability for the 60-min PH (Figures 3f–j), than for the 30-min PH (Figures 3a–e). The 30-min prediction implies less uncertainty than the 60-min prediction and, therefore, both error magnitude and variability increase in the latter case. MSE and LISO training presented equivalent performance, although the median metrics in MSE metrics showed slightly better performance than LISO. In the case of the 60-min prediction, LISO training improves the Dil-UNet performance in terms of all metrics (Figures 3f–j). Besides, this improvement is the largest among all the models comparing both loss functions. Considering that the 60-min PH is a more complex task than the 30-min analogue, and that the Dil-UNet has around 15 and 8 times more parameters than LSTM and Stacked-LSTM respectively (see Supplementary Table S1), this suggests that the larger the PH and the more complex the model, the more evident is the LISO influence in glucose forecasting. Besides, it can also be observed that the performance limitations inherent to the proposed DL architectures are not overcome by the use of any loss function. As an example, the use of LISO to train the Dil-UNet, improves the prediction performance compared to the MSE training, but it does not outperform the Stacked-LSTM or the LSTM.

Tables 1, 2 show the mean and Standard Deviation (SD) of the obtained results for 30-min and 60-min PHs, respectively, considering the four validation folds in all subjects (n = 29) for the four evaluated models. Results are grouped by the loss function used to train the DL models.

The three proposed DL models outperformed the naïve approach (set as baseline for the minimum requirements regarding prediction performance) for all metrics and both PHs. These results show that, among the proposed DL models, there is no architecture that works optimally for both PHs. Moreover, LSTM-based models outperform the Dil-UNet for both loss functions and PHs. This might be related to the fact that a recurrent neural network as the LSTM and Stacked-LSTM are based on an architecture that implements memory cells that preserve their state over time, theoretically more suitable for time series prediction purposes than other architectures that are in principle more complex but are not specifically designed to catch the temporal dependencies (48). Besides, regardless of the employed loss function to train the models, LSTM showed slightly better prediction metrics for 30-min PH, whereas Stacked-LSTM, which has more parameters (see Supplementary Table S1), did it for 60-min PH.

LSTM and Stacked-LSTM presented slightly better results when trained with the standard MSE than when trained with LISO. Focusing on the difference of all metrics, it is noticeable that LISO influences more the Stacked-LSTM than the LSTM. This could be related to the fact that the Stacked-LSTM possesses twice the number of parameters than LSTM (see Supplementary Table S1) and hence is more “flexible” than the LSTM.

Finally, focusing on the diabetes-specific metrics, it is worth noticing that the ParkesAB criterion (i.e., to be greater than 99%) was fulfilled by all models regardless of the loss function used for the 30-min PH. However, this requirement was not fulfilled for the 60-min PH. All models left from 2.3% to 3% below the minimum threshold. The ISOZone metric, which is the most restrictive requirement based on the ISO 15197:2015 standard (32), was not fulfilled in any case (i.e., to be greater than 95%), for any of the PHs, being around 16% and 39% below the threshold for 30-min and 60-min PHs, respectively. This might be related to the architectural limitations of the proposed DL models.

The results presented in the previous section provided insights about the overall differences between the DL models evaluated for interstitial glucose forecasting but did not consider the inter- and intra-subject variability (49, 50). Due to the fact that this work presents a DL-based DIY module that follows a subject-oriented approach, an individual evaluation of the DL models that would derive in personalized AI feedback is essential. Figures 4, 5 shows the subject-wise diabetes-specific prediction metrics, namely ParkesAB and ISOZone metrics after training the models only with LISO loss functions for 30-min and 60-min PHs, respectively. Supplementary Figure S2, Supplementary Figure S3 show the ParkesAB and ISOZone metrics after training with the MSE for 30- and 60-min PH, respectively, which are equivalent to those obtained after the LISO training. Subjects' IDs were sorted by the number of available CGM samples to train and validate the DL models following the proposed 4-fold trimester-wise cross-validation approach. These two metrics have been analysed in detail because they contemplate the prediction errors that are clinically relevant and are specific for the interstitial glucose prediction task. Additionally, RMSE was also analysed for 30- and 60-min PH after training with both loss functions (Supplementary Figure S4, Supplementary Figure S5, respectively), since it is the metric that is usually taken as the reference to compare between different models in the literature.

From Tables 1, 2, no significant differences are observed between the means in any of the evaluated metrics. In all cases, such differences are clearly lower than the average SD. This implies that there are not statistically significant differences in favor of any of the metrics.

It is worth noting that, although a sufficient number of samples is required to properly train a DL model, it can be observed that more CGM samples do not imply more accurate predictions in the proposed DL models. This effect is related to: (1) the glucose patterns heterogeneity among subjects (i.e., inter-subject variability) (49), specially in people with T1D (50); (2) the day-to-day CGM variability (i.e., intra-subject variability) (51); and (3) the various factors that influence the subject's glucose dynamics (such as insulin administration, subject metabolism, carbohydrate intake, physical activity) that are not contemplated in the proposed DL models training because only CGM data were employed in this study.

In concordance with the results exposed in the global validation (Table 1), in the vast majority of the cases the LSTM-based models presented better prediction performance in terms of RMSE (Supplementary Figure S4, Supplementary Figure S5) than the naïve model for both PHs, even though for some subjects the naïve model showed lower variability (e.g., subjects #045, #048, and #068, Supplementary Figure S4a). In the case of Dil-UNet, most of the cases presented lower median than the naïve model but its strong dependency on the fold (i.e., high intra-subject variability) makes it a less robust model than the naïve approach for a few subjects (e.g., subject #48, Supplementary Figure S4b). Dil-UNet showed the largest variability within the four validation folds for both PHs (Supplementary Figure S4, Supplementary Figure S5) in almost all subjects trained with both loss functions, with a few exceptions (e.g., subject #004 and #014, Supplementary Figure S4a). This means that, in general and for both PHs, the Dil-UNet prediction performance is more dependent on the folds (i.e., more sensible to the intra-subject variability) than the LSTM-based models. Thus, the interstitial glucose prediction reliability depends more on the time of the year used for the model training, making it a less robust model in terms of RMSE than the LSTM-based models. This could be related to the fact that Dil-UNet has a significantly larger number of parameters than the other two models (see Supplementary Table S1), so more samples might be needed to achieve better prediction performance.

The improvement presented by the proposed models compared to the naïve model becomes more evident studying the ParkesAB metric for both PHs. The training of the DL models using both loss functions outperformed the naïve approach, although in a few cases some models presented slightly higher variability (e.g., subject #007, Figure 4a). In the case of 30-min PH (Figure 4a; Supplementary Figure S2a), the only exceptions were subjects #068 and #058, which are the ones that had the fewest available samples for training. In the case of 60-min PH, the three DL models of the seven subjects with fewer samples (#063, #055, #061, #057 #003, #068, and #058) presented equivalent ParkesAB (Figure 5a; Supplementary Figure S3a). This happened with the training of both functions, but it was more evident with LISO training. Apart from these exceptions related to the lack of enough samples, LSTM-based models always showed higher ParkesAB, and also Dil-UNet (except for subjects #065 and #046).

Prediction performance in terms of ParkesAB and ISOZone evidences the RMSE limitations to properly characterize the interstitial glucose forecasting model performance. Firstly, model variability among folds related to the lack of training samples showed by the diabetes-specific metrics was not reflected by RMSE. Besides, ParkesAB and ISOZone give more importance to samples that are clinically relevant (i.e., prediction errors of hypoglycemic and hyperglycemic events), it therefore provides more information on whether the prediction model is suitable for this specific task. As an example, comparing RMSE and ParkesAB metrics for the 30-min PH training with MSE loss function (Supplementary Figure S4a, Supplementary Figure S2a, respectively) of subjects #067 and #025, it is observed that #067 presented lower RMSE (i.e., better prediction) in all models than subject #025. Nonetheless, subject #025 presented a higher ParkesAB (i.e., better prediction in the diabetes context). Hence, even though subject #067 predictions showed a lower RMSE, the predictions of subject #025 will be more clinically reliable and relevant. Thus, lower RMSE does not always imply better interstitial glucose predictions for people with T1D. This was consistent when comparing also the RMSE with ISOZone (Figures 4b,5b; Supplementary Figure S2b, Supplementary Figure S3b).

Besides, even though more training samples do not ensure better prediction performance, obtained results have set a threshold regarding available training CGM samples to achieve a fairly reliable glucose prediction. Such threshold is different for each PH. Setting the limit where the naïve approach (i.e., the predicted sequence consists of the last samples of the input) achieves equivalent ParkesAB and ISOZone in terms of median and SD than all three proposed DL models, the threshold for 30-min PH is ∼1,500 CGM training samples, whereas for 60-min PH it is ∼5,000. Such a difference might be related to the complexity of the task. A more complex prediction requires, in principle, more training samples to properly learn the phenomenon. This phenomenon occurred for both loss functions and both PHs (Figures 4c,d, 5c,d). Notice that both diabetes-specific metrics show consistency in setting these thresholds. Hence, the proposed DIY module will discard potential users that provide CGM data that would end in a training and validation dataset with less training samples than the correspondent threshold, since it has been demonstrated that this number is not enough to generate a reliable interstitial prediction model using the proposed DL models.

ParkesAB metric is based on the least restrictive requirement of ISO 15197:2015 (32), that establishes that at least 99% of the prediction errors must fall within certain limits considering the clinically critical glucose concentrations. A DL model whose predictions meet this requirement can be considered clinically safe under this criterion. Studying this metric for the 30-min PH (Figures 4c,d training with LISO, and Supplementary Figure S2a, Supplementary Figure S3a training with MSE) the three proposed DL models met this requirement within the four validation folds of the 29 subjects. Also, the naïve approach, except for subjects #065 and #001, met this ISO requirement, which explains why the most restrictive criteria included in the ISO standard has been also evaluated in this work (i.e., ISOZone). Conversely, for the 60-min PH, only 7 out of the 29 subjects met this criterion training with MSE (Supplementary Figure S3a), and 9 out of 29 subjects training with LISO (Figure 5A). Hence, even though Table 1 shows unfavorable results of the proposed novel loss function, more subjects are included in the ParkesAB acceptable zone for 60-min PH. This evidences the need for performing a subject-oriented evaluation of the DL models when proposing approaches such as a DIY using diabetes-specific metrics (52). However, the results obtained reveal the need for further improvement in the proposed architectures.

ISOZone metric, is the most restrictive condition of ISO 15197:2015 (32), setting the minimum of the prediction errors in 95% within the acceptable zone. Thus, fulfilling this requirement would lead to more clinically safe actuations than only fulfilling ParkesAB requirement. Unfortunately, the ISOZone metric did not surpass 95% in any personalized DL model, confirming the fact further improvements in the architectures should be assessed (Figures 4b,5b; Supplementary Figure S2b, Supplementary Figure S3b).

Finally, assessing the inter-subject variability, it is observed that none of the three proposed DL models work optimally for all subjects, although LSTM-based models presented the best average prediction performance (Table 1). For example, for the 60-min PH, Dil-UNet prediction in subject #055 shows the best ParkesAB trained with LISO, slightly better than LSTM-based models. However, the same model presented by far the worst ParkesAB in subject #65 compared to LSTM-based models, and slightly worse than the naïve approach (Figure 5a). The same occurs with the ISOZone (Figure 5b). Regarding RMSE, the Stacked-LSTM model trained with MSE presented relatively low variability within folds for both PHs and loss functions, although there are some exceptions (Supplementary Figure S4, Supplementary Figure S5). However, mean and SD variations in all metrics are comparable within all models. Considering that besides the inter-subject variability, the CGM samples of the subjects can present different day-to-day and intra-subject variability (51), this suggests that the variability and prediction capabilities are strongly dependent on the subject. There is not a DL model that clearly outperforms the rest, but all of them present better prediction performance in the same subjects, although there are some variations related to how a certain architecture has learnt about a given trend of the subject's CGM data.

There are many studies in the scientific literature that have assessed the glucose forecasting problem. However, for a fair comparison with this work, the following criteria were used for the selection of other studies: a.

Explicit indications that the glucose prediction was performed in people with T1D.

Focusing on De Paoli's work, it is also the only one reporting the development of a personalized model for each subject. Hence, a fair comparison can be assessed. However, they did not report prediction performance at 60-min PH, and neither did diabetes-specific metrics. Our proposed framework outperformed the results obtained by De Paoli's work, which is based on Jump Neural Networks (40), in terms of RMSE at 30-min PH. Thus, among the only two studies that reported personalized models using only CGM data, our work provided the best prediction performance.

Although the prediction performance of the DIY-based models is mainly limited by the use of only CGM data, our models achieve comparable performance in terms of RMSE to the rest of considered works for 30-min PH. For 60-min PH, these differences are more significant. The model that presents the best RMSE for both PHs is the N-BEATS (44). For 30-min PH, our LSTM showed similar performance (21.86 ± 2.86 vs. 18.22). This difference is more noticeable for 60-min PHs, where N-BEATS outperforms the Stacked-LSTM (38.44 ± 4.55 against 31.66). Nonetheless, this work only evaluated this model with 6 subjects, whereas we used 29. Moreover, they used several variables as input, including CGM, finger stick glucose, insulin, and carbohydrates intake, among others. It is also worth noticing that our work is the only one reporting ISOZone metric, which is the most restrictive criteria of the ISO 15197:2015 standard. Hence, this can serve as a benchmark for comparison in future developments of processing algorithms.

Finally, the ParkesAB metric, which is a metric that indicates if the clinical decision based on the predicted CGM labels is clinically safe for the subject, was reported only in two of the studies found in the literature. For the 60-min PH, only one work reported this result without reaching the minimum requirement (≥99%), but our work was closer to the minimum than the LSTM-based model proposed by Mosquera-Lopez et al. (26) (97.77% and 97.60%, respectively). For the 30-min PH, the two studies that reported this metric fulfilled the ParkesAB requirement. It is worth noticing that the personalized DIY DL models developed in this work presented the best ParkesAB metric for both evaluated PHs, evidencing again the limitations of the RMSE as the standard for glucose prediction models comparison. Thus, this approach can be considered the clinically safest among the ones included in this comparative, being the only one that followed a subject-oriented approach for the DL training and validation using only 1-year CGM data from one subject to train the model. Unfortunately, the evaluation of the proposed framework using the OhioT1DM dataset (the most common dataset for glucose forecasting benchmarking) (53) was not feasible. Our work uses trimester-wise partitions based on 1 year of CGM data, and the OhioT1DM dataset provides only 2 weeks of diabetes-related readings, making impossible to do a fair comparison.

As a final validation of the DIY DL-based framework for personalized interstitial glucose prediction, additional data from the 29 subjects included in this work were collected to further evaluate the consistency of the above presented results. For every subject, a DL-model was generated using the whole year that was previously partitioned following a 4-folds using LISO. Then, the new collected test set was used to evaluate the models. Sensor replacement or data unavailability due to reading interruptions produced a decrease in the number of subjects available for this evaluation. The number of days included in the test set was varied, including 30, 90, 180, and 365 days. This was assessed to analyze if the prediction performance differed depending on the length of the test set. As shown in Figure 6, from the initial 29 subjects, three test sets of subjects were obtained: (1) 20 subjects for testing 30 and 90 days; (2) 19 subjects for testing 180 days; and (3) 7 subjects for testing 365 days. RMSE, ParkesAB, and ISOZone metrics were analyzed. The boxplots comprise data from n subjects, being n dependent on the number of days of the test set. Moreover, Supplementary Tables S3, Supplementary Table S4 show the mean and SD corresponding to these results for 30- and 60-min PH, respectively.

Figure 6; Supplementary Tables S3, Supplementary Table S4 show that, for both PHs, the prediction performances barely change when increasing the number of days included in the test set. The greatest change in terms of the median and variability is observed when the test set includes 365 days. This is more evident for 60-min PH (Figure 6B). However, this is likely related to the significant decrease in the number of subjects included. It can be concluded that, following the DIY-approach after training with 1 year of CGM data, testing with one month of data is representative of the overall personalized model performance.

Comparing results of Table 1, that shows the prediction metrics after training the models with 9 months, to Supplementary Tables S3, Supplementary Table S4, that shows RMSE, ParkesAB and ISOZone metrics after training with the whole year, it is observed a general improvement in RMSE and diabetes-specific metrics. However, the 4-folds cross-validation included 29 subjects, and in the test only 20 subjects were included (for 30 and 90 testing days) due to data unavailability. Thus, this performance improvement might be related both due to training with more data (∼25% more, assuming equal distribution of the CGM readings within the year), and due to include less subjects in the evaluation. Furthermore, it is worth noting that for a 60-min PH, the difference between the Dil-U-Net and the LSTM-based models are less, suggesting that the former benefits more from being trained with more data than the LSTM-based. Concluding, the use of an additional test set validates the prediction performance previously shown and, hence, the competitive performance of the DIY model proposed in this work respect the state-of-the-art.

The design of this open-source module has considered its portability for its standalone use, its easy integration in broader applications regardless of the OS running the software, and its modular design so that researchers and developers can test novel architectures following the proposed DIY approach. To prove that this module is OS-agnostic (i.e., it can run over different OSs), the Python application has been deployed to run in a Docker (46) container, proving its functionality in two different OSs: Windows 10 and Ubuntu LTS 24.04. Although the current version of this module relies on Docker and terminal prompting from a PC and this is not ideal for the use of most of the people with T1D, the dockerization of the app facilitates its fast and straightforward integration in a smartphone application, enhancing its potential daily use. The DIY module usage documentation is available in a GitHub public repository (see “Data availability” section).

For simplicity, only the LSTM architecture was implemented, since it presented the shortest training times with competitive performance. Figure 7 shows the workflow of the DIY module from the user's perspective, in a smartphone, even this module is ready to run on a PC, including the first and subsequent uses, considering the cases when the user provides enough data or not, illustrating the information that is prompted to the user. This design was developed through a co-creation process after carrying out several focus groups that included potential users of this module, and it pursuits user engagement, useful interaction, and information supply regarding their personalized AI models.