Within the realm of molecular design, virtual screening (VS) has conventionally been employed to pinpoint molecules potentially yielding optimal experimental outcomes (Shoichet, 2004). Contrasting de novo molecular design, the source of molecules is distinctive: in virtual screening, the structure is known in advance, while in molecular de novo design, it is an attempt to generate the structure to be evaluated. Although virtual screening libraries have become very large according to the standards of drug discovery, the chemical space corresponding to these libraries only occupies a small part. When considering such a compound library, the evaluation method may inevitably sacrifice the accuracy of prediction. By using de novo molecular design to generate molecules in a directional way, computational workers hope to cross the chemical space more effectively and obtain the best chemical solution while analyzing fewer molecules than large chemical libraries. In addition, for a given target, there may be many acceptable regions in chemical space. Hence, the objective of the molecular design approach is to strike a balance between exploring global solutions and harnessing local minima (Schneider, 2010; Müller et al., 2022).

Recently, with the advancement of artificial intelligence (AI), new practical experience has been gained in the field of drug discovery (Ding et al., 2017, 2022; Chu et al., 2022). In typical data domains like computer vision (Voulodimos et al., 2018; Borhani et al., 2022) and natural language processing (NLP) (Chowdhary, 2020; Ferruz et al., 2022), deep generative models have significantly advanced in representing data distributions (Meyers et al., 2021). Such techniques are also employed to mimic molecular distributions, understand the probabilistic distributions of vast molecule sets, and produce novel molecules by drawing samples from these distributions (Dauparas et al., 2022). In the realm of molecular structure generation, a variety of deep learning models have been suggested by scholars. These encompass techniques such as generative adversarial networks (GANs), variational autoencoders (VAEs), and recurrent neural systems (RNNs) (Creswell et al., 2018; Korshunova et al., 2022). In these methods, a molecule is represented as a simplified molecular-input line-entry system (SMILES) (Weininger, 1988). Most of the current molecular generation models are based on conditional molecular design and finally generate new molecules with properties close to the predetermined target conditions (Xu et al., 2019; Walters and Barzilay, 2020).

The descriptors of SMILES are generally implemented by using long-term and short-term memory networks (LSTM). Serving as a unique temporal cycle neural network, LSTM was crafted explicitly to tackle the pervasive issue of long-term dependencies inherent in traditional RNNs. Due to the characteristics of this cyclic algorithm, the cyclic structure and chiral center of molecules expressed by SMILES are presented more perfectly. LSTM can be used to generate molecular sets with or without filters. Grisoni et al. suggested the use of bidirectional generative RNNs for designing molecules based on SMILES. In pursuit of this, they employed two proven bidirectional approaches and pioneered a novel technique for augmenting data and generating SMILES strings, termed as bidirectional molecule design by alternate learning (BIMODAL) (Grisoni et al., 2020). In addition, Li et al. studied the ability of RNN-based de novo molecular design method to produce new molecular inhibitors in the research field of chemical space (Li et al., 2020). In their quest to formulate novel inhibitors for proto-oncogene serine/threonine protein kinase 1 (PIM1) and CDK4 kinase, they evaluated four compounds. Their efforts culminated in the identification of a potent PIM1 inhibitor and two primary compounds that hinder CDK4 activity.

There is also a class of deep learning algorithms for automatic encoders, such as VAE and adversarial auto-encoder (AAE), which use the description method of molecules in latent space to generate molecules. On the one hand, the molecular features of the training set are stored in latent space by encoder, and on the other hand, these molecular features are reconstituted into new molecules by decoder. Owing to this approach's utilization of continuous latent space accumulation, the newly generated molecular set retains the physicochemical property distribution inherent in the training set. Many models have been proposed that employ reasonable substructures as building blocks for generating high-quality molecules. Previous studies introduced a model termed chemical-vae (Gómez-Bombarelli et al., 2018), designed to produce novel molecules, enabling effective exploration and refinement within expansive chemical compound spaces. In order to generate effective molecular graphs, MHG-VAE proposed molecular hypergraph grammar (MHG) to encode chemical constraints (Kajino, 2019). The authors have proposed a reaction model to forecast the interaction among reactants, resulting in the creation of novel molecules. In lieu of VAE, the objective function incorporates minimization to acquire model parameters.

Another popular deep learning algorithm is GAN. The algorithm uses two functions, the generator and the discriminator, against each other to generate the desired molecules. Because of the discontinuity of the atoms that make up the molecule, the discriminator can't directly feedback the information to the generator. Referring to the method adopted in NLP, the information feedback is realized by a reward function or policy gradient. The reward equation serves as a filtering criterion. It not only preserves the property distribution of the generated set akin to the training set but also nudges the property distribution of the created set to shift toward a different direction. This algorithm can use various molecular description methods, such as SMILES, Latent space, or graph, and can meet various requirements by combining various screening conditions, so it is a potential algorithm. Prykhodko et al. introduced LatentGAN, a novel deep learning framework that integrates an autoencoder with a generative adversarial neural network, tailored for de novo molecular design (Prykhodko et al., 2019).

To refine a sequence-based generative model specifically for molecular de novo design, Marcus and team formulated a technique capable of learning to construct structures with predetermined desirable characteristics, employing enhanced episodic likelihood (Olivecrona et al., 2017). M Popova et al. proposed a unique computational approach for the de novo design of molecules with targeted characteristics, named ReLeaSE, which utilizes deep learning and reinforcement learning methodologies (Popova et al., 2018). These methods used additional optimization processes instead of directly generating molecules of intended properties, which becomes inefficient.

The model consists of three 250-dimensional gated recurrent unit (GRU) networks: the predictor network, the encoder network and the decoder network. The predictor and encoder are made up of bidirectional GRUs network, while the decoder is a unidirectional GRU.

In order to forecast the subsequent character within the SMILES strings that depict molecular structures, the final layer incorporates a dense output layer coupled with a neuron unit that utilizes a softmax activation function. In this context, the synthesis of donepezil is showcased as a case study. Principally prescribed for Alzheimer's management, donepezil can be represented by the SMILES notation: “O=C(C(C=C(OC)C(OC)=C1)=C1C2)C2CC(CC3)CCN3CC4=CC=CC=C4”. The initial data for the system comprises a “one-hot” delineation of a SMILES string, whereby each string undergoes segmentation into various tokens. Here, the inaugural token is “O”, transformed into a “one-hot” vector and fed into the linguistic model. Subsequently, the model revises its concealed state and forecasts the probability spread over forthcoming viable tokens, decoded as “=” in this instance. Supplying the one-hot representation of “=” prompts the model to modify its concealed state during the forthcoming cycle, leading to the revelation of the succeeding token. This recurrent process, tackling one token at a time, persists until the “\n” character surfaces, signifying the culmination of the SMILES sequence, thus generating the final SMILES notation for donepezil (Figure 1).

In the case of the previously delineated predictor and encoder networks, we introduce distinct fixed-form distributions. Specifically, we define q_ϕ(y ∣ x) and q_ϕ(z ∣ y, x), each parametrized by ϕ. These distributions aim to approximate the true posterior distribution, employing a widely employed method in efficient variational inference, as described in Equations (1, 2):

Where x represents a molecule and y represents its continuous valued properties. Given a variable x, the properties y are predicted as Equation (3):

In the molecule generation process, we use the decoder network p_θ(x ∣ y, z) to generate molecules by the following equation:

The definition of loss function refers to previous research (Kingma et al., 2014). In this study, the variational lower bound −L(x, y) of the log-probability of a labeled instance (x, y) is showed in Equation (5):

Given the data distributions of labeled p~l(x,y), the loss function is defined as Equation (6):

where the last term is mean squared error for generative learning.

We use the decoder network p_θ(x ∣ y, z) to generate a molecule. A molecule representation x^ is obtained from y and z by Equation (4). At each time step j of the decoder, the output x^(j) is predicted by conditioning on all the previous outputs (x⁽¹⁾, ..., x^(j−1)), y, and z, because we decompose p_θ(x ∣ y, z) as Equation (7)

The model undergoes training for 300 cycles utilizing the Adam optimizer. To mitigate the risk of overfitting, we employ early stopping during the training process. This means that if the model's performance on the validation set deteriorates compared to the previous cycle, we halt the training and adopt the parameters from the prior iteration as the final outcome. The loss stabilizes after the 25th cycle, as depicted in Figure 2.

The model was implemented using TensorFlow (v2.4.0) in Python (v3.8). We trained it on an NVIDIA 3090 GPU with the learning rate set to 0.001. And the metrics we used are as follows:

• Validity: The model's learning capability is evidenced by the rating of authentic molecules among the synthesized compounds as follows (Equation 8).

• Uniqueness: the percentage of molecules that were really unique when they were generated. Low uniqueness points to recurrent molecule production and a model with little distribution learning as follows (Equation 9).

• Novelty: the percentage of authentically unique molecules that were generated but were not in the training set as follows (Equation 10).

Where X is the list of molecules from the provided training set, n is the number of generated samples, and V is the list of created chemically valid molecules.

For the pretraining set, we collected 310,000 SMILES strings of drug-like molecules from the ZINC database (Irwin and Shoichet, 2005) with molecular weight (MolWt) ranging from 200 to 500 and logP ranging from 0 to 5. Figures 3, 4 show the larger property distribution of the ZINC database, the larger the property distribution, the stronger the fitting ability of our model. Furthermore, in order to maintain the standardization and unity of data, we used RDkit toolkit (Landrum et al., 2013) to canonicalize the SMILES strings.

We collected molecules with pIC50 or pEC50 greater than 6 as the active molecules of AChE, delete duplicate molecules, and got the fine-tuning sets from Binding DB (Liu et al., 2007). Molecules from SciFinder (Gabrielson, 2018) that fit the aforementioned requirements were added to the fine-tuning set to enlarge it. Finally, 4,996 molecules were obtained and canonicalized using the RDKit toolkit.

In order to show the diversity of molecules in the data set, we screened molecules with QED (quantitative estimate of drug-likeness) values greater than 8 and logP values between 0 and 3, then randomly selected 10 molecules to calculate molecular similarity. The results are shown in Figure 5.

Some properties of the molecules used in the training process of the model are the following:

Acetylcholine (ACh) is the first neurotransmitter discovered by human beings, and its mediated neurotransmission is the basis of nervous system function. Nevertheless, AChE is an efficient hydrolase that catalyzes the hydrolysis of ACh, which regulates the concentration of ACh at synapses and then terminates ACh-mediated neurotransmission. Thus, in this segment, our goal was to create a compound library targeting the AChE receptor. This would aid in the synthesis of newer inhibitors that are not only more potent but also exhibit reduced side effects.

We gathered molecules from Binding DB that displayed activity toward AChE receptors to formulate the training dataset. In the end, 4,996 molecules were utilized to constitute the fine-tuning dataset. As depicted in Figure 8, we chose 500 molecules synthesized by the fine-tuned model, which exhibit physicochemical attributes and occupy the chemical space analogous to the fine-tuning set. Additionally, the distributions of QED in the generated molecules are similar to those in the fine-tuning set of compounds. Furthermore, the distribution of SAScore is concentrated between 1 and 5, as shown in Figure 9, indicating that most of the molecules generated are easy to synthesize.

Figure 10 shows examples of the generated molecules. Simultaneously, we compared the molecules with donepezil (Birks and Harvey, 2018) (ID: E20) by ECFP4 (Rogers and Hahn, 2010) similarity method (Figure 11). As shown in the Figure 11, the model can effectively generate molecules similar to the training set. This indicates that our model can generate molecules that are effective for ACHE to a great extent. In order to further verify this conclusion, we conducted molecular docking experiments with Autodock Vina (Trott and Olson, 2010). The ligand in the crystal structure of human acetylcholinesterase in complex with donepezil (Dileep et al., 2022) (PDB ID:7E3H) was removed, then recorded the pocket position simultaneously. Molecules in the molecular library are docked with receptors at the pocket position using Autodock Vina. The docking scores are shown in Table 4, and the results showed that the molecules in the molecular library had high affinity with the target receptor (Figure 12). Based on the molecular docking results, as depicted in Figure 12A, the interaction between the small molecule and the protein primarily involves hydrogen bonding and hydrophobic interactions. Specifically, the N atom of the small molecule forms hydrogen bonds with the hydroxyl O atom of the Tyr124 amino acid residue (Tyr124=O... H- N, 2.3Å), as well as with the hydroxyl O atom of the Tyr337 residue (Tyr337=O... H- N, 2.5Å). Additionally, Figure 12B illustrates that the heteroatoms in the small molecule can engage in hydrogen bonding interactions with the active pocket of the protein, with the distribution of hydrogen bond donors and acceptors shown in Figure 12B. Furthermore, the 2D interaction analysis (Figure 12C) revealed that the hydrophobic carbon chain of the small molecule interacts with the hydrophobic amino acids Thr83, sn87, Trp286, Phe338, and Ile451 of the protein. Moreover, the small molecule forms π−π stacking interactions with the amino acid residues Trp86, Trp286, and Trp341, enhancing its binding affinity to the protein. Given the close proximity between the small molecule and other amino acid residues, it is hypothesized that van der Waals interactions may occur between them.