Torsades de Pointes (TdP) is a fatal arrhythmia induced by drugs. The major challenge in developing new drugs is to filter life-threatening drugs, which can lead to TdP, and prevent them from entering the market (Sager et al., 2014; Vicente et al., 2018). From the 1990s to the early 2000s, many drugs were withdrawn from the market due to concerns about inducing TdP (Roden, 2004; Onakpoya et al., 2016; Vicente et al., 2018). To detect drugs that have the risk of TdP, the International Council on Harmonization (ICH) has established a guideline based on the mechanism of pharmaceutically occurring QT interval prolongation, which is blockage of the I_Kr channel, a human ether-à-go-go-related gene (hERG) channel current (International Council on Harmonisation, 2005a; Harmonisation, 2005b). Although this guideline can accurately detect proarrhythmic drugs in new drug development, the strict regulation due to the low specificity of this guideline may disrupt the development of a potential therapeutic drug that prolongs the QT interval but never leads to TdP (Llopis-Lorente et al., 2020).

To address this problem, a comprehensive proarrhythmia assay (CiPA) was proposed at a Think Tank meeting at the US Food and Drug Administration (FDA) headquarters in 2013 (Sager et al., 2014; Strauss et al., 2019). CiPA, as a new paradigm, assesses the proarrhythmogenic risk of a drug through in silico simulation using a human cardiomyocyte model integrating multi-ion channels of pharmacological in-vitro data (Li et al., 2019). As part of this CiPA project, many researchers have suggested evaluation methods and TdP metrics to assess the proarrhythmic risk of drugs based on the in silico simulation. Lancaster and Sobie (2016) suggested several metrics derived from action potential (AP) and intracellular calcium (Ca²⁺) concentration from in silico simulations observing drug response in human ventricular cell models, such as AP duration at 50% repolarization (APD₅₀), AP duration at 90% repolarization (APD₉₀), Ca duration at getting to 50% baseline (CaD₅₀), and Ca duration at getting to 90% baseline (CaD₉₀). The FDA proposed qInward and qNet as promising TdP risk metrics computed from in silico simulations for categorizing the proarrhythmic risk of drugs as high, intermediate, and low (Chang et al., 2017a; Li et al., 2017, 2019). Through several studies, some of these in silico features were reported to be highly accurate in classifying the torsadogenic and non-torsadogenic drugs.

In our previous study, we validated the robustness of 12 in silico features using an ordinal logistic regression (OLR) model by comparing the classification performances of metrics according to the used in-vitro experimental datasets (Jeong et al., 2022a). However, as the results of the OLR model using 12 in silico features were desirable, the single value of the in silico feature was perceived as limiting to classifying the three TdP risks. Therefore, this study proposes a convolutional neural network (CNN) model using the variability of promising in silico TdP metrics hypothesizing that the variability of in silico features based on beats contains more information than the single value of in silico features. First, we determined a CNN classifier by comparing the classification performance of the proposed model according to the variabilities of seven in silico biomarkers obtained from in silico drug simulation using specific in-vitro data. We then evaluated the robustness of the proposed CNN classifier using the variability of in silico biomarkers obtained from different in-vitro data.

We demonstrated the performance of the proposed CNN classifier for 16 test drugs based on seven in silico feature variabilities calculated from drug simulation using in-vitro datasets of Chantest et al. (Table 2; Figure 3). The model performance was excellent for classifying high-risk and low-risk drugs with an AUC of 0.94 and 0.93, respectively, using qInward variability. Especially for the low-risk drugs, qInward variability demonstrated the best performance compared to other in silico feature variabilities, with less than or approximately 0.70 AUC. The classification accuracy of the intermediate-risk drugs using qInward variability with the CNN classifier was moderate at 0.75 AUC, but it was the highest among the in silico feature variabilities. In the CNN classifiers using qInward variability, the LR + and LR-for three TdP-risk were satisfied to over minimal limitation level the excellent level of LR + > 10 and LR− < 0.1, a suitable level for LR + > 5 and LR − < 0.2, and the minimally acceptable level for LR + > 2 and LR − < 0.5.

The proposed CNN classifiers using the in silico feature variabilities mostly classified the high-risk drugs with a good performance of over 0.80 AUCs, apart from CaD₉₀ variabilities, which demonstrated very poor performance of 0.46 AUC. CaD₉₀ variability showed only a medium level of accuracy for the other two groups of TdP-risk drugs (both 0.71 AUCs). The intermediate-risk drugs were classified at a moderate accuracy of approximately 0.70 AUCs through the proposed CNN classifier with six in silico feature variabilities, except for qNet variability. qNet variability classified only high-risk drugs effectively, showing good performance of 0.83 AUC. APD₅₀ variability showed moderate outcomes for classifying intermediate-risk and low-risk drugs as both 0.70 AUCs. The variabilities of qNet, CaD₅₀, APD₉₀, and dVm/dt_{Max_repol} were low for classifying low-risk drugs; in particular, CaD₅₀ variability was abysmal at 0.25 AUC.

To validate the proposed CNN classifier, we compared the classification performance using the in silico feature variabilities computed from the three in-vitro datasets of Li dataset, Chantest dataset, and Nanion dataset (Tables 3, 4; Figure 4). Table 3 summarizes the proposed CNN classifiers for classifying 16 test drugs. When categorizing the high-risk and low-risk drugs in the combined datasets, the performances of the CNN classifier using qInward variability, which was the best model with an excellent level of AUC in Chantest datasets, decreased to moderate levels of 0.75 and 0.78 AUCs, respectively. Conversely, the classification performance of the intermediate-risk drugs increased a bit to 0.79 AUC when using qInward variability. The CNN classifier using dVm/dt_{max_repol} was the best when classifying the TdP-risk for 16 test drugs of combined datasets, but the performances were just medium levels as 0.77: high-risk, 0.79 for intermediate-risk, and 0.80 for low-risk (Table 3; Supplementary Figure S2). Two CNN classifiers using qInward variability and dVm/dt_{max_repol} variability satisfied the minimally acceptable LR+ and LR-levels in the three TdP-risks.

Table 4 summarizes the model performances in classifying TdP-risk for all 28 drugs using in silico feature variabilities of merged datasets. The proposed CNN classifier showed poor accuracy in classifying the high-risk drugs with AUC ranging from 0.62 to 0.68 in all the in silico feature variabilities. The CNN classifier using qInward variability categorized low-risk drugs to acceptable levels of 0.82 AUC and intermediate-risk drugs to moderate levels of 0.75 AUC. In the combined dataset, the classification performances for intermediate-risk drugs among all 28 drugs and 16 test drugs were modest in most of the in silico feature variabilities. However, when using APD₉₀ variability and dVm/dt_{Max_repol} variability, the intermediate-risk drugs among all 28 drugs did not demonstrate high performance but were classified reasonably with an AUC of 0.81. For detecting the high-risk drugs, qNet variability showed good performance in 16 test drugs of Chantest dataset but poor performance in both 16 test drugs and all 28 drugs of merged datasets, which were 0.54 and 0.68 AUCs, respectively. Conversely, the classification performances for intermediate-risk and low-risk drugs using the CNN classifier using qNet variability increased to medium levels (approximately 0.75 for 16-test drugs and all 28 drugs in the combined datasets.

We compared the classification performance of the proposed CNN classifier using the variability of in silico biomarkers with that of the OLR classifier using a single value for in silico biomarkers (Supplementary Table S2). The single values of qNet, APD50, APD90, and dVm/dt_{max_repol} classified high-risk drugs with excellent accuracy and low-risk drugs with moderate accuracy for the 16 test drugs in the Chantest dataset. In the merged dataset, the high-risk drugs were moderately distinguished by qNet, dVm/dt_{Max_repol}, APD50, and APD90, and the low-risk drugs could be roughly classified by only qNet and dVm/dt_{Max_repol}, but not very accurately; their AUCs ranged from 0.7 to 0.79. However, the classification performance for intermediate-risk drugs was poor in all OLR classifiers using single values, less than 0.6 AUC. However, the CNN classifiers using the variabilities of qInward and dVm/dt_{Max_repol} could roughly classify the 16-test drugs into three levels of TdP risk, despite not being perfect.

This study suggested a CNN classifier using the variability of in silico biomarkers to assess the TdP-risk of drugs under the hypothesis that the variability of in silico biomarkers is more likely to have more features based on the TdP-risk than every single in silico biomarker. Hinterseer et al. suggested that the variability of QT intervals based on beats affects drug-induced long-QT syndrome (Hinterseer et al., 2008). The mechanism of pharmaceutically occurring QT interval prolongation involves the blockage of ionic channels. Because in silico TdP metrics are obtained from AP and ionic currents computed from simulations with blockages of ionic channels as drug responses, the variability of the in silico TdP metric according to the beats also reflects the characteristics of torsadogenic drugs. Indeed, among the in silico features used in this study, the variabilities of qInward, CaD₅₀, and CaD₉₀ were better for classifying the TdP-risk of drugs, especially for intermediate-risk and low-risk drugs, than their single values. Notably, the CNN classifier using qInward variability was the best model to classify the high- and low-risk in the 16-test drugs of the Chantest datasets.

The single values of the in silico features were used to classify proarrhythmic risk using the threshold value derived from the OLR model (Li et al., 2019). The OLR model determined each threshold based on its distribution according to the main classification targets, such as high-risk, intermediate/low-risk, low-risk, and high/intermediate-risk. Therefore, the OLR model using the single values of the in silico features showed good performance for high-risk and low-risk drugs, but low performance for intermediate-risk drugs (Supplementary Table S2). Among the single in silico features, qNet, APD₅₀, APD₉₀, and dVm/dt_{Max_repol} were able to classify high-risk drugs well for 16-drugs in the Chantest datasets. The accuracies of APD₅₀ and APD₉₀ were higher than those of the CNN classifier using the qInward variability. However, the overall classification performances for the three proarrhythmic risks were better in the CNN classifier using qInward variability than in the OLR models. We believe this is because of the loss function used when training the CNN classifier, categorical cross-entropy, which updated the trainable parameters such as weights and bias to improve the classification accuracies of the three risk levels.

In this study, the CNN classifier using qInward variability exhibited the best performance in categorizing the proarrhythmic risk of drugs. In the 16 test drugs of Chantest data set, no other risk drugs were included among the drugs classified as high-risk through CNN classifiers using qInward variability (LR + > 10 for high-risk drugs). However, high-risk drugs can likely be classified into other risk groups (LR−= 0.27, high-risk drugs). On the other hand, for the 16 test drugs of the combined datasets, the possibility for high-risk drugs to be classified as other-risk was almost zero in the CNN classifier using qInward variability (LR-for high-risk drugs; a median of 0.00, and range of 0.00–1.45). The proposed model has little possibility of classifying low-risk drugs in the Chantest datasets as other-risk drugs, but the classified drugs as low-risk drugs may be other proarrhythmic risks (LR+ = 4.67 and LR- = 0.00 and low-risk drugs) (Roman et al., 2002).

We validated the classification performance of the proposed CNN model using in silico feature variabilities computed from three in-vitro datasets. Based on the results for the merged dataset, the proposed CNN classifier appeared to be specialized in classifying intermediate- and low-risk drugs. We believe that the machine learning features extracted from qInward variability through the proposed CNN classifier had the advantageous characteristics of determining intermediate- and low-risk drugs. However, the classification performance showed significant differences according to the used in-vitro datasets used. In addition, the proposed CNN classifier using qInward variability was under fitted to the 12-training datasets; thus, the classification performance for all 28-drugs was less than that for 16 test drugs, even in the Chantest dataset (Supplementary Table S3; Supplementary Figure S2). These results suggest the need for the proposed CNN model to be trained and tested using various types of drugs.

Because the in silico simulation results depend on in-vitro experimental data, many researchers have tried to validate the robustness of in silico features in classifying the proarrhythmic risk of drugs. Some researchers have validated the in silico features using uncertainty quantification algorithms and population modeling. Llopis-Lorente et al. improved the accuracy of the drug-induced TdP risk assessment by considering population variability (Llopis-Lorente et al., 2022). Chang et al. validated the robustness of TdP risk separation using qNet value and uncertainty quantification algorithms (Chang et al., 2017a). Han et al. suggested the selection method of calibration drugs and calibrated the previous OLR model by validating it using two lab-specific datasets (Han et al., 2020). In a previous study, we validated 12 promising in silico features using OLR, based on in-vitro datasets used in the AP simulation (Jeong et al., 2022a). In this study, we determined the CNN classifier using the variability of in silico features calculated using the Chantest dataset and validated the corresponding robustness using three merged in-vitro datasets.

This study has several limitations. First, we did not consider the mechano-electric feedback, which indicates that the mechanical contraction of the cell affects electrophysiological activity. Second, in an actual body environment, heart rate varies with time. Drugs can individually affect QT interval and heart rate, and the complexity of the QT interval assessment increases owing to heart rate variations (Roden, 2004). This study used the in silico feature variability under the static heart rate condition, which was 30 bpm, to mimic the bradycardia condition, without varying the heart rate in real time. We did not consider heart rate variability when performing in silico simulations. Therefore, the in silico simulation results could be slightly different from the clinical outcomes. However, as the tendency of the drug to affect the ionic channels is the same, we speculate that these limitations do not critically affect the results and do not change the entire conclusion of the study.