As mentioned above, artificial intelligence techniques generate learning machines using data sets produced during the analysis of a topic of interest. As a result, these machines predict events that could occur in this topic.

Of the learning techniques, two stand out: supervised learning and unsupervised learning. Since everything presented in the paper is related to supervised learning, we describe the essential elements of this learning method for the non-specialist in artificial intelligence. Our presentation of this learning method does not follow any traditional way of presenting it. Instead, we explain the method using an example of heterogeneous catalysis, where the reader is a specialist.

When we develop a new catalyst for hydrodesulfurization, the goal is to bring out a product with low sulfur content. This content (which we describe as the variable Y) is the variable that determines the formulation of the catalyst, as well as the temperature, pressure, amount of hydrogen, and other variables related to the process that will use the catalyst. We mark out these last quantities with the vector X = (X1, X2, X3,…, Xn).

The use of artificial intelligence techniques to develop a new hydrodesulfurization catalyst requires a set of events, called samples, with values (X, Y). They can be obtained by modeling the hydrodesulfurization process or performing experiments in a laboratory or from the variables associated with the process in an industrial plant. Typically, 80% of the samples are used for training the machine, and the rest for testing the machine’s effectiveness to predict new catalysts.

The basic principle of learning is the following. First, we construct the function F(W, X), which estimates the sulfur content and is defined by the parameters W = (W1, W2, W3…). We create a metric to estimate the difference between the prediction of the sulfur content, F(W, X), and their values, Y, in the samples. With the average of these differences, we construct the loss function, J (W, X, Y), whose gradients with respect to each parameter permit updating the parameters. This updating continues until the loss function reaches its minimal value. Figure 1 depicts a general sketch of such a learning machine.

There exist various algorithms to construct a learning machine (Breiman, 2001; Dhillon and Verma, 2020; Scherer et al., 2020; Schütt et al., 2018; Shazeer et al., 2018; Shawe-Taylor and Sun, 2011), from the simplest ones like the Random Forest (Breiman, 2001) which requires only a few samples, to algorithms based on artificial neural networks (Shazeer et al., 2018), which could require a large number samples for training. These last algorithms could contain millions of parameters (Krizhevsky et al., 2017). Going into detail on these algorithms is beyond the purpose of the present manuscript. However, the references for these algorithms are a good start for readers who want to dabble in them.

When a molecule nears a solid, its interaction with the atoms on the solid’s surface determines the adsorption energy. If the solid is a catalyst, this energy determines many catalytic properties. Consequently, the analysis of this energy could help to develop new catalysts.

Although quantum mechanical calculations can provide this energy, they require long calculation times and much computing power. Therefore, the use of only this methodology to search for new catalysts generates a bottleneck.

An alternative to avoid this bottleneck is to generate these energies with the techniques of artificial intelligence. For example, through supervised learning using the Deep Learning technique. In this case, the constructed learning machine correlates the local atomic distribution and the properties of the atoms with the adsorption energy through the function F(W, X).

In this type of learning, it is necessary to define the target variable (the variable Y), which, in the present case, is the adsorption energy. It is also important to propose the variables from the system that determine this energy. For example, these variables can be the atomic properties of the atoms and their local atomic order (Behler, 2011; Back et al., 2019; Gómez-Peralta and Bokhimi, 2021). These variables can also correspond to relationships between the atomic radii (Gómez – Peralta and Bokhimi, 2020).

The function, F(W, X), generated with the Deep Learning technique, contains many parameters with values optimized during the training. Therefore, this learning procedure requires a considerable number of samples obtained from experiments or models. In the present case, each sample (X, Y) contains information related to the local distribution of the atoms and their chemical properties (variable X). In addition, it also includes the adsorption energy generated by quantum mechanical calculations (variable Y).

Thousands of samples (X, Y) are created by building atomic distributions used to perform quantum mechanical computations. The function F(W, X) maps X in Y through the relationship that lies between them without explicit modeling; for one sample, this function outputs a value that compares with the adsorption energy Y generating a difference value. For all samples, the average difference value gives the loss function J (W, X, Y), used to update the parameters W of the function F(W, X). This process repeats until the loss function reaches its minimal value to complete the learning process. It is important to note that the computations include the interaction between the molecule with catalytic and non-catalytic sites.

Of the samples, a percentage, about 80%, is used for training, and the rest to check the ability of the function F(W, X) to predict adsorption energies. This prediction allows finding new catalysts for specific catalytic processes related to the molecule adsorbed on the solid.

As an example of this methodology, (Back et al., 2019) used the Deep Learning technique to predict the adsorption energy (variable Y) of CO or H when they interact, each separately, with various surfaces of pure metals, metal alloys, or intermetallic compounds. Their model is a modification of the previously reported model for the prediction of the physicochemical properties of crystalline compounds (Xie and Grossman, 2018; Chen et al., 2019).

Back et al. identified (variable X) each atom with the group and period to which it belongs, its electronegativity, its covalent radius, its number of valence electrons, its first ionization energy, its electron affinity, the block to which they belong, and its atomic volume. They model the interaction between an atom with its local environment with its associated Voronoi polygon (Blatov, 2004).

The number of samples used by Back et al. for training was 43,237 for each adsorbate on the solid. Each sample had 37 different elements and 96 stoichiometries. In these samples, the number of different spatial groups was 110, while the number of crystal facets was 41.

The learning machine for CO adsorption had 4,938 parameters W, while the machine used for H adsorption utilized 6,738 parameters W.

For 12,000 samples excluded from the training set, the learning machine predicts adsorption energies for CO and H with a Mean Absolute Error (MAE) of only 0.15 eV. This value is lower than that obtained in previous studies by the authors using many alloys as the solid (Tran and Ulissi, 2018).

In heterogeneous catalysis, nanoporous materials are of particular interest because they have a large surface area. Hence, to help to understand the catalytic properties, this area should be determined accurately. The traditional method to get this area is the Brunauer-Emmett-Teller (BET) method, where for measuring this area, the material adsorbs inert molecules at low temperatures. But, the surface area estimation using this method is not always precise, especially for materials with large surface areas, where some of the adsorbed molecules are no more on a monolayer. Thus, for example, the reported surface areas of metal-organic frameworks (MOFs) are exaggerated (Gómez-Gualdrón et al., 2016; Sinha et al., 2019).

Datar et al. (Datar et al., 2020) developed a machine learning approach for estimating surface area to correct this problem. They combine machine learning with molecular modeling to derive argon isotherms at 87 K for over 300 metal-organic frameworks from the CoRE-MOF database (Chung et al., 2014; Chung et al., 2019). They use a Lennard-Jones potential with a cut radius of 12 Å (Rappe et al., 1992). Finally, they applied the BET theory to these isotherms to determine the surface area of these MOFs by using the SESAMI algorithm (Sinha et al., 2019), focusing primarily on the true monolayer area (Gómez-Gualdrón et al., 2016).

Their machine learning model relates the features that describe the adsorption isotherms and the surface area (variable Y). The isotherm features (variable X) were constructed by dividing the pressure into seven regions on a logarithmic scale. These regions are related linearly to the true monolayer areas. The model parameters were optimized using the Least Absolute Shrinkage and Selection Operator (Tibshirani, 1996). It includes regularization to reduce overfitting. For the training, Datar et al. used 40% for the samples for the training and 60% for the testing. The training evolution was analyzed using the cross-validation method (Zhang, 1993).

The learning machine predicts more precise surface areas than the BET method. For example, for samples with surface areas greater than 3,500 m²/ g, with the learning machine, only 2% of the structures were predicted with surface area errors more significant than 20%, while for the BET method, the latter figure was 54%.

With the trained learning machine, the surface area of 68 MOF structures of the CoRE-MOF database was obtained, modeling the isotherms with the adsorption of N₂ (77 K). The predictions of the surface area using these isotherms with the learning machine trained with the isotherms generated with the adsorption of Ar also gave better results than those obtained by the BET approach. The estimated areas with the learning machine were scaled to a ratio of 1.148 since the area associated with the N₂ molecule is 0.163 nm², while the one associated with the Ar atom is 0.142 nm² (Mikhail and Brunauer, 1975).

Discovering new materials is of the utmost importance from both technological and scientific points of view. That is why, in recent years, the use of artificial intelligence techniques to speed up their search has proliferated (Goldsmith et al., 2018; Gupta et al., 2018; Gómez-Peralta and Bokhimi 2020; Chen et al., 2021; Cho and Lin, 2021; Konno et al., 2021). Of particular interest is the finding of nanoporous materials because they have a large surface area that is attractive in heterogeneous catalysis (Cho and Lin, 2021).

These materials can exist in hundreds of thousands. However, nowadays, selecting those with an attractive surface area for heterogeneous catalysis occurs through the slow procedure of trial and error. As an alternative to this discovery procedure, Cho et al. (Cho and Lin, 2021) developed a learning machine based on convolution neural networks (CNN) that predicts the surface area of such materials.

Their methodology analyzes methane adsorption on zeolites, but it applies to any molecule adsorption in any nanoporous material. In their research, Cho et al. used 6,500 zeolite structures with lattice parameters less than 24 Å, selected from the Predicted Crystallography Database (Pophale et al., 2011).

They calculated the methane adsorption isotherm (variable Y) at 300 K on every zeolite structure for 14 different pressure values between 0.0005 and 200 bar. For that, they employed the Monte Carlo method approach, using a Lennard-Jones type interaction potential (García-Pérez et al., 2007) with a cut-off radius of 12 Å. Their learning machine had the LeNet-5 architecture (LeCun et al., 1998), commonly used to detect objects in two-dimensional images (Qin et al., 2020; Jain et al., 2021).

Each zeolite structure was modeled with a three-dimensional image equivalent to a cubic cell with 24 Å per side. Each of the 24 × 24 × 24 image voxel values (variable X) corresponded to its potential as a methane adsorption active site. Filters of 2 × 2 × 2, 3 × 3 × 3, and 5 × 5 × 5 voxels built the different layers of the convolutional neural network. Of the total samples, 90% were for training and 10% for testing the model. The use of data augmentation techniques (Shorten and Khoshgoftaar, 2019) decreased the loss values, achieving squared means errors of up to 0.015 mol/ kg.

Their learning machine predicts adsorption isotherms in fractions of seconds, which contrasts with the times required by using the Monte Carlo method, where to calculate the adsorption isotherm in a zeolite requires dozens of CPU hours. This difference in time is significant since there are hundreds of thousands of such zeolite structures from which it is interesting to obtain their adsorption isotherms.

These studies show the potential of convolutional neural networks to develop other applications in heterogeneous catalysis. For example, for the capture of carbon dioxide or removing H₂S or SO₂ during sour gas sweetening.

The learning machines described above, for their learning, were fed with features defined by the properties of the elements and features obtained with calculations based on the atomic distribution. This methodology allowed generating a considerable number of samples for the training.

In contrast, Williams et al. (Williams et al., 2020) describe a learning machine in which part of the content of the samples comes from experiments, which limited the number of samples to less than 500.

In Machine Learning, when the number of samples is tiny, and the goal is to have a minor error in the predictions, it is convenient to choose an algorithm that learns correctly with this number of samples. One of these algorithms is the Random Forest (Breiman, 2001), which Williams et al. used to create their learning machine based on 100 tree predictors.

Their learning machine looked for new catalysts based on ruthenium for ammonia decomposition. They started with the ruthenium catalyst Ru₄K₁₂, promoted with potassium, and supported on gamma-alumina. Their research sought to predict new catalysts by substituting Ru with 33 different elements (Mg, Ca, Sc, Ti, V, Cr, Mn, Fe, Co, Ni, Cu, Zn, Sr, Y, Zr, Nb, Mo, Rh, Pd, Ag, Cd, In, Sn, Hf, Ta, W, Re, Os, Ir, Pt, Au, Pb, Bi).

To develop their first learning machine, they prepared catalysts by replacing one Ru atom with an M atom (M = Ca, Mn, In). The composition of the catalysts was 3 wt% Ru, 1 wt% M and 12 wt% K. For each catalyst, they measured the ammonia conversion at 250, 300, and 350°C; this conversion is the target variable (variable Y) that supervises the learning. It is to notice that selecting these elements to replace Ru maximizes the difference between the features used in the learning.

The features (variable X) used to characterize the ammonia decomposition process were the catalyst composition, the operating conditions, the synthesis variables, and the atomic properties. Only five of these features were necessary to get accurate predictions, being the reactor temperature the most important. The other four features were related to the electronic structure of the atoms: the number of d-shell valence electrons, the electronegativity, the covalent radius, and the adjusted work function. It is important to remember that in 1985 Falicov et al. proposed that the number of d-shell valence electrons was essential to describe the catalytic activity (Falicov and Somorjai, 1985).

When evaluating the learning machine by replacing Ru with an atom M of the remaining 30 elements, the best ammonia conversions occurred with M = Sr, Mg, Sc, or Y.

These predictions motivated the authors to synthesize different catalysts by substituting Ru with the following 19 elements: Cu, Ni, Cr, W, Hf, Zn, Bi, Pd, Mo, Y, Sc, Sr, Mg, Os, Pt, Au, Nb, Fe, and Rh. When they catalyzed ammonia conversion, the highest conversions corresponded to the substitution of Ru with Sr, Mg, Sc, or Y, which matches the learning machine’s predictions, and demonstrates the usefulness of this machine to predict novel catalysts.

Williams et al. extended their model by generating new sets of catalysts using, for each set, three different atoms to replace Ru, selecting the atoms from the 19 elements mentioned in the previous paragraph, plus Ca, Mg, and In.

With these 22 elements replacing Ru, 60 different sets of three catalysts were built, generating 60 different learning machines. These machines predominantly predict that the catalysts that produce a high ammonia conversion are those in which Ru is replaced by Ca, Sr, Sc, Y, and Mg. These predicted conversions have a 10% error compared with those measured in experiments.

The results show that artificial intelligence techniques and relatively few experiments are sufficient to save both cost and time to discover new catalysts.

Since the inception of machine learning techniques, they have been used to predict sulfur content after hydrodesulfurization (HDS). In 1996, Berger et al. (1996) constructed a learning machine based on a Feed-Forward Neural Network with only one hidden layer containing three nodes. Their experiments provided the 25 samples utilized to train and test this learning machine. Eighteen of them were for its training and seven to test it. For all the hydrodesulfurization experiments, they employed a catalyst with 4.2 wt% Co, 16.7 wt% Mo and 0.4 wt% S, with a hydrogen to oil volume ratio of 500.

The target variable (variable Y) was the sulfur content after the hydrodesulfurization process, with values between 10 and 1870 ppm. The features (variable X) that characterized the process were: temperature, with values between 348 and 360°C, pressure, with values between 600 and 1,200 psi, liquid hourly space velocity with values between 0.4 and 6.0 h⁻¹, and the sulfur content of the heavy atmospheric gas oil, with values between 7,500 and 12,000 ppm. Since, in this study, the experimental sulfur output values have little change in the pressure range used, the study shows that, in this case, the pressure does not have a significant effect on the sulfur output predictions.

Despite the tiny number of samples used in training, the sulfur content predictions with the test samples were within 10% of the experimental values.

Another learning machine, developed by Al-Jamimi et al. (Al-Jamimi et al., 2019), predicts the sulfur content after an HDS process. They used a Support Vector Machine (Shawe-Taylor and Sun, 2011) combined with a genetic algorithm (Katoch et al., 2021) as the learning technique.

The ultimate sulfur content in the product (variable Y), with values between 0.0 and 1,258 ppm, is the variable that supervises the learning. The features (variable X) that characterized the HDS process were temperature, with values of 200 300 and 400°C, pressure, with values between 25 and 75 bar; hydrogen dosage, with values between 0.4 and 0.8 g, the initial sulfur concentration, with values between 439 and 1,500 ppm and the type of fuel. The authors generated 34 samples performing HDS experiments, 24 to train the learning machine, and 10 to test its prediction capability. After its training, the machine delivered predictions regarding the product’s sulfur content within 5.5 percent of the experimental results.

In the literature, some publications also report the use of artificial intelligence techniques to examine the evolution of the HDS process in industrial plants over time (Ma et al., 2020) or the evolution of this process in a refinery, with temperature, pressure, and hydrogen dosage (Arce-Medina and Paz-Paredes, 2009; Al-Jamimi et al., 2018). The literature also covers the use of these techniques for predictions in other oil industry processes (Sagheer and Kotb, 2019; Liu et al., 2020).