Over the years there have been many techniques trumpeted as having great disruptive potential, which were eventually found to have muted applicability. It is rare that a technology comes along that is truly disruptive and is adopted across wide areas of science and engineering. Modern artificial intelligence (AI) is a rare technology where those claims are not overblown. In what follows, we will use the terms “machine learning” and “artificial intelligence” interchangeably.

One of the authors (HK) was an early advocate of the use of AI and computer vision in space sciences with applications in event detection/classification (e.g., Karimabadi et al., 2009), knowledge discovery in simulations and in-situ-visualization (e.g., Karimabadi et al., 2011a; Karimabadi et al., 2011c; 2012; 2013a), and derivation of equations from data (Karimabadi et al., 2007). The impetus for this effort was based on the vision that as our ability to generate data continues to grow exponentially, data driven science would become an indispensable field of scientific knowledge discovery. This vision has since come to pass, but the rate and scale with which this has happened has exceeded all expectations.

Despite the promising results and utility of those early works, including applications of simple neural nets to spacecraft data (e.g., Newell et al., 1991; Boberg et al., 2000), the techniques were not widely adopted. At the time, the field of AI was in a nascent stage in which artificial neural networks (ANNs) had been largely abandoned in favor of “lighter weight” techniques such as support vector machines. These algorithms had limited learning capacity, and relied heavily on hand engineered features, requiring a top-down agent to act as a “God outside the machine” to tell the models which attributes of the world to focus on, rather than allowing the algorithms to learn what is and is not relevant bottom-up, from the data and the model’s objective function. Another factor that limited their utility was their lack of universality. One had to devise special algorithms for problems in computer vision, speech, and audio, among others.

Everything changed in 2012 when AlexNet, a GPU implemented convolutional network (CNN), won ImageNet’s image classification competition by a wide margin. This seemingly overnight success was built upon 7 decades of slowly evolving research in deep learning (see the Supplementary Material for definition of deep learning). The field had to wait for the accessibility of large data sets and the development of GPUs, a widely available relatively inexpensive device with a special kind of massively parallel computational power, before its potential could be realized.

Since AlexNet, advances in AI have fueled adoption of neural algorithms across a myriad of industries and sciences. The first applications of AI in space sciences have been in analysis of spacecraft data (e.g., Camporeale, 2019; Breuillard et al., 2020; Li et al., 2020; Hu et al., 2022) where off-the-shelf AI techniques can be readily applied. However, application of AI in NeuHPC offers a greater opportunity with the potential to qualitatively change the field. The remainder of this article focusses on NeuHPC.

Partial differential equations (PDEs) often lead to extreme multi-scale behavior which makes the resolution of all scales in one simulation impossible. While exascale computing will enable simulations of larger systems (e.g., Xiao et al., 2021; Ji et al., 2022), the extreme multiscale nature of many problems in space sciences requires new techniques. In the global magnetosphere, there are 10⁷ degrees of separation in spatial and temporal scales, putting it beyond the conventional techniques even at exascale. Also, round-off error in time-stepped solvers is severely limiting. Further, exascale simulations present other challenges, from knowledge discovery to the massive datasets, to efficient checkpointing and data management. We consider AI as a core technology and its adoption as critical for meaningful advancement in scientific computing. This belief is based on unique features of neural nets and the rapid and promising advancements of their use in scientific computation.

Table 1 summarizes key features of ANNs that make them especially suitable for overcoming the current HPC challenges by enabling new capabilities not possible with conventional approaches. While in-depth discussion of each topic is beyond the scope of this paper, relevant references are provided for interested reader to learn more. First, automated differentiation (see the Supplementary Material for more details) enables accurate computation of derivatives of arbitrary order (spatial and temporal) to working precision. This mesh-free operation, resulting in mesh invariant solutions, is advantageous over numerical differentiation methods (e.g., finite differencing) which suffer from discretization error with increasing cost and error in higher derivatives. As an example, one can solve the heat equation ∂u/∂t = Δ(u) where the function u is represented as a neural net and the spatial and temporal derivatives are calculated using the chain rule.

Second, the universal approximation theorem (Hornik et al., 1989) implies that ANNs can accurately approximate any function. In contrast to fixed-shaped approximators that have no internal parameters (e.g., polynomials), neural networks consist of parameterized functions, allowing them to take on a variety of different shapes.

Less known but as important is the universal approximation theorem for operators (Chen and Chen, 1995) which states that a neural net with a single hidden layer can accurately approximate any non-linear continuous operator (Lu et al., 2021). The operator can be explicit such as derivatives (e.g., Laplacian), integrals (e.g., Laplace transform) or implicit such as solution operators of a PDE. This offers a unique capability where the network can learn the solution to an entire family of PDEs rather than an instance of a PDE, as in the conventional approaches. Once the model is trained, inference to obtain solutions for different parameters of the PDE is very fast. This can lead to orders of magnitude speedup and enables efficient exploration of the solution space and ensemble modeling which may be prohibitively expensive otherwise.

These capabilities open the door to zero-shot learning, i.e., the operator can be trained on a lower resolution and evaluated at a higher resolution, without seeing any higher resolution data. To this end, Li et al. (2021a) developed the first network (FNO) with zero-shot learning that successfully learns the resolution-invariant solution operator for the family of Navier-Stokes equations in the turbulent regime. This feature of transferring the solution between the meshes works well on both the spatial and temporal domain (Kovachki et al., 2021). We refer the reader to Kim et al. (2021) for discussion and differences of super-resolution reconstruction for paired versus unpaired data. Another useful feature of AI-based solvers is transfer learning. For example, Li et al. (2021b) used a pre-trained model on the Kolmogorov flow to transfer it to different Reynolds numbers.

A wide variety of solutions have been proposed to leverage ANNs in computations across domains such as CFD, material science, and weather forecasting. A detailed review is beyond the scope of the present work. Our goal is simply to bring awareness to promising advances in NeuHPC and provide a starting point for further exploration. Although our focus is NeuHPC, techniques such as system identification can also be applied to spacecraft data either in isolation or in combination with simulation data.

We advocate for the following changes: a) make funding NeuHPC a priority, b) adapt the funding to the pace of AI developments. This means a short leash on grants and strong focus on results measured by well-established benchmarks (see examples of benchmarks below). c) Promotion of interdisciplinary collaboration with AI experts in industry and academia to overcome the fact that most plasma physicists do not have deep expertise in AI.

Given AI’s prowess in predictions, we suggest as starting point proof-of-concept (POC) studies focused on video prediction and error correction:

• Video prediction: Apply off-the-shelf spatio-temporal deep learning models for video prediction (e.g., U-net, ResNet) to simulations. This would create a benchmark (Wang et al., 2020) for comparison with follow up studies using PDE centric AI approaches like PINO or FNO. We suggest 2D hybrid simulations (e.g., KHI) where many training cases can be generated for videos of current density, mixing (see Supplementary Material), among others.

Other POCs of interest that target multi-scale problems include:

• Closure models: Explore derivation of closure models for the island coalescence problem (Karimabadi et al., 2011b).