The Naive Bayes classifier (NBC) is a probabilistic ML algorithm widely used for both binary and multiclass classification tasks (Rish, 2001; Hastie et al., 2009). It is rooted in Bayes’ theorem, which quantifies the probability of an event based on prior knowledge of related events. The “Naive” component of its name comes from the assumption that input features are conditionally independent, simplifying calculations and enhancing computational efficiency. Thus, the NBC computes the conditional probability of a given instance belonging to a specific class by making the “naive” assumption of feature independence. Mathematically, it leverages Bayes’ theorem: P C k | x = P C k | x ∏ i = 1 n P x i | C k P x , where C k is the class, x is the feature vectors.

The NBC makes the “naive” assumption that features are independent given the class. This strong assumption might not hold in all real-world scenarios. However, despite this simplification, it often performs surprisingly well and is computationally efficient. Critical steps to train and use NBC are listed below:

1. Calculate Class Priors: Estimate the prior probabilities P C k | x for each class based on the training data.

Variations of NBCs are based on types of data and assumptions. Some common variations are:

• Gaussian Naive Bayes: Assumes features follow a Gaussian (normal) distribution.

NBC is a straightforward yet effective classifier, particularly suitable for binary classification tasks, provided that the assumption of feature independence is reasonably met. While it may not be the optimal choice for all data types, it serves as a standardized baseline classifier and is extensively employed in the prediction DILI through QSAR modeling (Ai et al., 2019; Williams et al., 2019; Wu Y. et al., 2021). For instance, Zhang et al. (2016) employed NBC to construct a computational model for assessing DILI risk. Their model exhibited a 94.0% accuracy in 5-fold cross-validation during the training phase, with a concordance rate of 72.6% on an external test set. They identified key molecular characteristics associated with DILI risk.

Tang et al. (2020) developed QSAR models for mitochondrial toxicity using five machine learning methods, including NBC along with various chemical signatures. They adopted a threshold moving strategy to rectify data imbalance and implemented consensus models to enhance prediction performance, achieving up to 88.3% accuracy in external validation. Notably, the study highlighted the significance of substructures such as phenol, carboxylic acid, nitro compounds, and aryl chloride in classification. In another work, Rao et al. (2023), proposed an integrated artificial intelligence (AI)/ML model that employed physicochemical properties and in silico off-target interactions to predict the severity of DILI for small molecules. They utilized data from 603 compounds categorized by the U.S. Food and Drug Administration (FDA) as Most DILI, Less DILI, and No DILI, and combined the NBC with other ML approaches to enhance DILI prediction, surpassing the performance of QSAR models based solely on chemical properties.

The Support Vector Machine (SVM) is a potent and versatile machine learning algorithm used for a range of tasks, including regression, binary, and multiclass classification, as is the case in predicting DILI through QSAR modeling with chemical structures (Li et al., 2020a; Tang et al., 2020; Wu Z et al., 2021; Rao et al., 2023). Its primary goal is to identify a hyperplane that maximizes the margin between the nearest data points from the two classes. The fundamental concept is to optimize this margin between classes, resulting in improved generalization to new, unseen data. These closest data points are referred to as “support vectors.” The margin is defined as the distance between the hyperplane and these support vectors. Mathematically, the SVM tries to solve the following optimization problem: 1 2 min w , b w 2

Subject to y i w ∙ x i + b ≥ 1 f o r i = 1 : n , where:

• w is the weight vector perpendicular to the hyperplane.

The above optimization problem ensures that data points are correctly classified with a margin. Support vectors are the data points that lie on the margins or violate the margin constraint. In many cases, the data may not be linearly separable in the original feature space. To handle these cases, SVMs often use a kernel trick. A kernel function transforms the original feature space into a higher-dimensional space, where the data might become separable. Common kernel functions include:

• Linear Kernel: K x , x ′ = x ∙ x ′ .

In some cases, the data might not be perfectly separable, or there could be outliers. In such situations, SVM employed a soft margin to allow for some misclassification by introducing a slack variable. The optimization problem becomes: 1 2 min w , b w 2 + C ∑ j = 1 n τ j

Subject to: y i w ∙ x i + b ≥ 1 − τ j f o r i , j = 1 : n , τ j ≥ 0 , where C is a hyperparameter that controls the trade-off between maximizing the margin and minimizing misclassification. Major steps to train and apply the SVM Classifier are listed below.

1. Data Preparation: Gather and preprocess the data, ensuring it is properly labeled, and features are appropriately represented.

SVM has been applied extensively in predicting DILI, particularly in scenarios with limited data, owing to its robust prediction accuracy and computational efficiency. Notable studies (Wang et al., 2019; Li et al., 2020b; Hemmerich et al., 2020; Mora et al., 2020; Wu Z. et al., 2021) have employed SVM for DILI prediction. In a comprehensive analysis conducted by Wu Y. et al. (2021), involving 14 sets of QSAR data and 16 ML algorithms, the radial basis function SVM (rbf-SVM) emerged as the top-performing method among all ML techniques, underscoring its efficacy in this domain.

However, certain limitations were associated with the SVM algorithm. SVM tends to be computationally expensive and may not be well-suited for very large datasets. When the dataset exhibits extra noise, such as overlapping target classes, SVM’s performance can be compromised. Furthermore, SVM may perform suboptimally when the number of features for each data point exceeds the number of training data samples. These considerations are critical when deciding on the suitability of SVM for specific DILI prediction tasks.

The Random Forest classifier is a robust ensemble ML algorithm frequently employed for both binary and multiclass classification tasks. It is an extension of the decision tree algorithm, having the primary objective of enhancing generalization and mitigating overfitting by forming an ensemble of multiple decision trees. In binary classification, the Random Forest classifier is designed to classify new instances into one of two classes based on their features. The process involves the construction of multiple decision trees during the training phase, and their collective predictions are amalgamated to reach the final classification decision. Key steps for training and applying the Random Forest classifier are outlined below.

1. Bootstrapped Sampling: For each tree in the forest, a random subset of the training data is selected with replacement. This process is known as bootstrapped sampling. It creates diversity among the trees, as each tree is trained on a slightly different data subset.

Compared with other machine learning methods, random forest has several unique characteristics:

• Reduced Overfitting: The ensemble of trees helps to mitigate overfitting by averaging out the noise and biases present in individual trees.

Random Forest emerges as an invaluable machine learning technique for the classification of liver toxicity. Gadaleta et al. (2018) employed Random Forest classifiers and DRAGON molecular descriptors to create QSAR models designed to predict molecular initiating events leading to hepatic steatosis. They effectively used a Balanced Random Forest classifier, alongside the strategy of under-sampling, to construct robust QSAR models from unbalanced DILI datasets. Both techniques yielded comparable predictive results, achieving approximately 75% accuracy in toxicity prediction.

The Multilayer Perceptron (MLP), also known as an Artificial Neural Network (ANN), is a fundamental neural network architecture used for a wide range of machine learning applications, including classification and regression. An MLP consists of multiple layers of artificial neurons, typically structured into an input layer, one or more hidden layers, and an output layer. Each neuron within a layer is connected to each neuron in the layers above and below it, creating a densely interconnected network. Connections between neurons, represented as weights (often denoted as W), are learned during the training process. The output of each neuron is determined by applying an activation function, such as the sigmoid, ReLU, or tanh function, to a weighted sum of its inputs. Mathematically, the output (O) of a neuron in a hidden or output layer is computed as follows: y = f ∑ i = 1 n w i . x i + b , where y is the output of the neuron. f is the activation function. w i represents the weight associated with the i-th input connection. x i is the i-th input to the neuron, and b is the bias term. The training process involves adjusting these weights and biases using techniques like backpropagation and gradient descent to minimize a loss function, allowing the MLP to learn complex relationships within the data.

MLPs are versatile and can approximate a wide range of functions, making them a popular choice for various ML applications. Cruz-Monteagudo et al. (2008) investigated computational approaches for predicting idiosyncratic hepatotoxicity using 3D chemical structures such as linear discriminant analysis (LDA) and ANNs. The RBF architecture was used in a neural network classification method that used the same descriptors as those in the LDA model. In the training series, this RBF neural network outperforms the LDA model, achieving an accuracy of 91.07%, sensitivity of 92.00%, and specificity of 90.32%. Examination of the Receiver Operating Characteristic (ROC) curve proved its continuously superior performance.

A Deep Neural Network (DNN) can be mathematically represented as a composition of functions (Schmidhuber, 2015). Given an input vector x , the output y of a DNN with L layers can be expressed as: y = f L ∘ f L − 1 ∘ … ∘ f 2 ∘ f 1 x

Each layer l applies a linear transformation z l = W l a l − 1 + b l followed by an activation function a l = σ z l , where W l is the weight matrix and b l is the bias vector for that layer. The final output is obtained by applying an appropriate activation function at the last layer. Like MLP, DNN can be trained by adjusting the weights and biases to minimize a chosen loss function through techniques like backpropagation and optimization algorithms. DNNs are excellent for automatically extracting key features from large inputs, making them the perfect choice for transcriptomic data containing a wide variety of features.

DNNs (Hinton et al., 2006) have been effectively used to address the challenge of predicting various types of chemically induced liver injuries, including biliary hyperplasia, fibrosis, and necrosis, using DNA microarray data (Feng et al., 2019). Wang et al. (2019) used multi-task DNNs to evaluate gene and pathway-level feature selection strategies for these liver injuries. The DNN models exhibited high predictive accuracy and endpoint specificity, surpassing the performance of Random Forest and SVM models. In another study, Li et al. (2020b), developed a DNN model with eight layers using transcriptome profiles of human cell lines to predict DILI. The model leveraged a substantial binary DILI annotation dataset, achieving AUCs of 0.802 and 0.798 for the training and independent validation sets, respectively. These results outperformed traditional machine learning algorithms, including K-nearest neighbors, SVM, and Random Forest.

In a study conducted by Kang and Kang (2021), a DNN-based model was designed to predict DILI risk. This model used extended connectivity fingerprinting of diameter 4 (ECFP4) to represent molecular substructures. The data for this predictive model was meticulously collected from various sources, including publications like DILIrank and LiverTox. A model was developed through stratified 10-fold cross-validation, and the best DNN model showed an accuracy of 0.731, a sensitivity of 0.714, and a specificity of 0.750 when validated in the complete applicability domain. Jain et al. (2021) used a large-scale acute toxicity dataset encompassing over 80,000 compounds measured against 59 toxicity endpoints. They compared multiple single and multitask models using RF, DNN, CNN, and GNN approaches and found that multitask DL methods performed best.

Convolutional Neural Networks (CNNs) are mainly applied in image and speech recognition. These networks are well-suited for capturing spatial hierarchies and local patterns within images. CNNs typically incorporate convolutional layers for feature extraction, pooling layers for dimensionality reduction, and fully connected layers for classification. Architectures like AlexNet, VGG, ResNet, and InceptionNet have consistently demonstrated exceptional performance on various image classification tasks (Krizhevsky et al., 2012; Szegedy et al., 2015; Yamashita et al., 2018; Jain et al., 2021). Mathematically, CNN can be written as follows: Let X be the dataset having m number of images. The input feature size is denoted as n × n . Then the convolution layer is written as: A i j l = σ ∑ p = 1 f ∑ q = 1 f W p , q l A i + p − 1 , j + q − 1 l − 1 + b l A l = c o n v A l − 1 , F l

Here, l = 1 , … , L are the number of convolution layers. Next, the CNN has the pooling layer: P i j l = ⁡ max   A p i , p j , l A p i , p j + 1 l , … , A p i + p − 1 , p j + p − 1 l P l = m a x p o o l A l , P

Flatten the pooled feature maps to obtain a vector of size F . Now the fully connected layer is defined as: Z l = W l ∙ A l − 1 + b l Z l = F C A l − 1 , H l

The output layer has a single neuron for binary classification or multiple neurons for multi-class classification: Y = σ Z L

A suitable loss function, such as binary cross-entropy, is used for classification. The network is trained using gradient descent-based optimization to minimize the chosen loss function.

CNN was also used for DILI prediction (Nguyen-Vo et al., 2020; Chen X. et al., 2022). Nguyen-Vo et al. (2020) introduced a novel computational model for the prediction of DILI utilizing CNNs and molecular fingerprints based on 1,597 compounds. The model came up with an average accuracy of 0.89, a Matthews correlation coefficient of 0.80, and an AUC of 0.96.

Graph Neural Networks (GNNs) are a class of neural networks explicitly tailored for operating on graph data structures. They are particularly well-suited for tasks involving graphs, such as social network analysis, chemical structure analysis, and computational vision (Zhou et al., 2020). Node-level tasks are used in DILI prediction and chemical structure analysis, and involve predicting the properties or characteristics of individual chemical components, such as molecules, within a graph or network structure.

To learn node representations, GNNs combine information from nearby nodes, effectively capturing intricate relationships in graphs. For example, let G = V , E be the molecular graph, where V is the set of nodes (atoms) and E is the set of edges (bonds). The graph convolutional layer updates node representations based on their neighbors’ features. Let X be the initial node features (molecular fingerprint-embedded features) for all nodes in the graph. The output of the l − th graph convolutional layer can be represented as X l using the following equation: X l = σ D ^ − 1 2 A ^ D ^ − 1 2 X l W l ,

Here, A ^ = A + I is the adjacency matrix of the graph with added self-loops, D ^ is the diagonal of matrix A ^ , W l is the learnable weights for l − th layers, and σ is the activation function. Pooling or aggregation layers were incorporated to combine node features across different neighborhoods. Similar with the CNN architecture, one or more fully-connected layers were used to learn higher-level representations from the aggregated features.

The final layer produces the network’s output. Depending on the task (e.g., regression, classification), the number of neurons and the activation function in the output layer can be adjusted. The forward pass through the GNN can be represented mathematically as written below. Let X 0 be the molecular fingerprint-embedded features. The graph convolution is: X l = σ D ^ − 1 2 A ^ D ^ − 1 2 X l W l Aggregated Features = Pooling / Aggregation  X l D 1 = Dense  Aggregated Features , neurons , activation , … , D k = Dense  D k − 1 , neurons , activation Output = Dense  D k , output neurons , output activation Here, k represents the number of dense layers in the network.

GNNs have demonstrated their efficacy in addressing node-level tasks related to DILI predictions (Hwang et al., 2020). Ma et al. (2020) used a MV-GNN based model as a backbone to propose a property augmentation approach to involving more data for four datasets with liver toxicity-relevant properties. The GNN-based approach significantly outperformed existing baselines on DILI datasets, achieving an impressive 81.4% accuracy using cross-validation with random splitting. Lim et al. (2023) introduced a novel technique known as supervised subgraph mining (SSM). SSM effectively identifies explicit subgraph features through iterative optimization of graph transitions. This approach surpasses conventional machine learning methods such as SVM, Random Forest, k-Nearest Neighbors, and deep learning neural networks in DILI classification using two datasets, DILIst and TDC-benchmark. By employing structure-based pattern matching, the proposed approach can also identify subgraph characteristics associated with specific medication groups.

Mathematically, a recurrent neural network (RNN) can be represented as follows: at each time step t , the RNN takes an input vector x t , and computes the hidden state h t and the output y t using the following equations: h t = f W h h * h t − 1 + W h x * x t + b h y t = g W y h * h t + b y Here, h t represents the hidden state at time t , x t is the input at time t , y t is the output at time t , W h h , W h x , W y h are weight matrices, b h and b y are bias vectors, f and g are activation functions (typically sigmoid or hyperbolic tangent for f and softmax for g ). The hidden state h t captures information from previous time steps, allowing RNNs to model temporal dependencies in sequential data.

Xu et al. (2015) employed undirected graph recursive neural networks (UGRNN) to develop DL models for predicting DILI for drugs and small moleculars. Their DL-combined model outperformed ANN and DNN models, achieving an accuracy of 86.9% and an AUC of 0.955 when predicting the DILI of 198 drugs in the external validation set. The model also successfully identified important molecular substructures relevant to DILI, demonstrating the power of DL in this context. In another study Ruiz Puentes et al. (2021), investigators proposed using PharmaNet, a machine learning method that employs RNNs, to search for novel pharmaceutical candidates. PharmaNet was applied to discover ligands for 102 cell receptors and achieved impressive performance with a 97.7% Receiver Operating Characteristic curve-Area Under the Curve (ROC-AUC).

Generative Adversarial Network (GAN) is an advanced generative model composed of two neural networks: a generator and a discriminator. These networks are trained in opposition to each other. The generator’s objective is to create synthetic data that is virtually indistinguishable from genuine data, while the discriminator’s role is to differentiate between real and generated data. GANs operate through a minimax game where the generator and discriminator compete. As training progresses, the generator becomes increasingly skilled at generating realistic data, while the discriminator becomes better at distinguishing between real and fake data. This dynamic process drives the generator to produce high-quality synthetic data, establishing GANs as a foundational technology in a wide range of applications, such as picture production, style transfer, and data augmentation.

Chen Z. et al. (2022) developed Tox-GAN, which employed deep GANs to generate fresh animal study results without the need for extra tests. They demonstrated its effectiveness by creating transcriptome profiles with remarkable similarity (0.997 ± 0.002 in intensity and 0.740 ± 0.082 in fold change) to real-world data obtained from rat liver toxicogenomic studies. In a related study, Li et al. (2023) introduced the TransOrGAN framework, which aims to map gene expression patterns across multiple rodent organs, sexes, and ages. TransOrGAN generated synthetic transcriptomic profiles with an average cosine similarity of 0.984 compared to their corresponding real profiles. This proof-of-concept study involved 288 samples from nine different organs, showcasing the potential of TransOrGAN to generate realistic transcriptomic data for various research applications.

The field of Natural Language Processing (NLP) has undergone a transformative shift with the introduction of transformer-based models (Kang et al., 2020). These models have enabled the automatic analysis and comprehension of text data in scientific literature. In the domain of DILI studies, NLP models have proven to be valuable tools for extracting insights from textual sources.

Zhan et al. (2022) developed NLP techniques specifically for biomedical texts, allowing the automated processing of 28,000 titles and abstracts retrieved from the PubMed database. By comparing five different text embedding techniques, they found that the model using term frequency-inverse document frequency and logistic regression performed best, with an accuracy of 0.957 on the validation set. Wu Z. et al. (2021) employed a NLP approach based on Bidirectional Encoder Representations from Transformers (BERT) to classify DILI and decipher the meanings of complex text in drug labeling documents. This AI-based model utilized BERT’s power to enhance understanding of text data, particularly in the context of drug safety assessments.