Generative AI for Computational Science: A Viable Path

With a Worked Out Case

I’ve been thinking long and deep about what generative AI means for computational science. We all know what GenAI is capable of in writing, speech, audio and image generation – I have used many of these tools and they have been an incredible productivity gain! But being a computational scientist and machine learning researcher, I’ve always wondered what GenAI can do in my field.

In a previous post, I painted a gloomy picture about the potential of Generative AI in computational science. Here is the short version of my arguments. Most tasks in computational science are essentially deterministic: solving (non)linear systems, optimization, and simulating differential equations.

Generative AI, on the other hand, is inherently stochastic. There is no unique next token to an input text, no unique next note in a piece of music, and there are an infinite number of stories one can tell given the events that precede it. GenAI samples a probability distribution – this is also why an LLM will never generate the same output for a given input. Log story short, deterministic computational science and probabilistic GenAI can never match on this level.

This analysis is not entirely fair. There is one branch of computational science that does deal with probabilistic methods, namely Uncertainty Quantification (UQ). The main objective of UQ is to characterize probability distributions coming from experiments and simulations. Its main tools are sampling and statistics. As I mentioned in my last post, there is much untapped potential in combining UQ and GenAI, but it must be done the right way.

Let’s explore what a first-principles application of Generative AI for computational science could look like.

On Batteries, Oil Fields and Sponges

Over the holidays, I’ve been playing around with generative AI tools for solving the porous media equation. `Porous Media’ is a general term for any material containing pores. Foam is the classical example of a porous medium: a solid material matrix containing many voids of different sizes. Oil fields are also porous media. An oil well is not just some cavity in the earth’s crust but filled with solid rock and many small pores that contain a mix of oil, water and gas.

The example I want to talk about is ion transport inside batteries. In classical lithium-ion batteries, energy storage is governed by the transport of ions through porous electrodes. Electrodes are complex porous media, combining diffusion, transport and electrochemical reactions. All these phenomena are highly coupled to the exact porous microstructure. Therefore, the exact porosity determines the macroscopic operating conditions such as the current and voltage difference over the electrode.

Let’s get more specific. The microscopic porosity is given by a function ε(x) which is large in regions of high solid density and low in regions with many cavities. Figure 1 below shows that the concentration of solid bubbles is higher for larger . In our electrode example, two physical fields are tied to the porosity: the ion concentration c(x) and the microscopic potential ϕ(x).

Figure 1: Porous medium with corresponding porosity function ε(x). The electrode operating parameters, U0, F_right and l (the typical length-scale of ε(x)) are also shown.

These two fields are tightly coupled through PDEs. The potential does not vary much over time but influences the electrolyte concentration via

\(-\frac{d}{dx}\!\left(k_{\rm eff}(x)\,\phi_x\right) = a_s(x)\,k_{rn}\big(\phi_s(x)-\phi(x)-U_0\big),\)

\(\varepsilon(x)\,\frac{\partial c}{\partial t} \;=\; \frac{\partial}{\partial x}\!\left(D_{\mathrm{eff}}(x)\,\frac{\partial c}{\partial x}\right) \;+\;\frac{1-t_+}{F}\,a_s(x)\,j(x)\)

These equations is the basis for the Newman model of electrolysis. I will not go deeper into this model right now – it gets very complex – the point is we can solve these equations (fast) for c(x) and ϕ(x) given ε(x) and two macroscopic operating conditions:

• The potential difference U_0 over the electrode (typically small or 0)

• The current (or ion flux) F_right on the right boundary.

Enter Uncertainty

So how does the uncertainty enter this picture? Well, the porosity field is unknown. It depends on manufacturing, material quality and precise atomic configuration of the material. For all intents and purposes, is a random function. Therefore, so are c(x) and ϕ(x).

In fact, the electrolyte concentration and potential are not unique to given macroscopic parameters U0 and F_right. They are samples from a conditional probability distribution

\(\mu\left(c(x), \phi(x) \mid U_0, F_{\text{right}}, l\right)\)

The extra parameter l represents the typical spatial scales of ε(x). I will touch on this parameter more in Part 2.

So why is this probability distribution important? The goal is to design electrodes so that the battery can hold and discharge as much energy as possible, but safely and robustly. Because the electrolyte concentration and electric potential determine local reaction rates, overpotentials, and degradation hotspots, their joint distribution across the electrode encodes exactly which microstructures and operating conditions lead to high performance versus which lead to failure. If say, 10% of realizations lead to damage in the battery, we do not want to apply those operating conditions. This, and much more is encoded in this conditional probability distribution.

Sampling as a Fundamental Tool

The conditional distribution μ(c(x), ϕ(x)) can theoretically be derived from first-principles physics, but it is very complex. The distribution is not some simple Gaussian. In fact, we can never write this distribution down explicitly – we just know it exists. The only way to characterize it is by drawing samples. Samples are physically viable realizations of c(x) and ϕ(x). ‘Viable’ in this context means there is some plausible porosity function ε(x), so that solving the porous medium PDE would result in those profiles for the electrolyte concentration and potential.

Sampling is the fundamental tool in the field of uncertainty quantification – and many sampling algorithms have been designed over the years to tackle more and more complex problems. I happen to believe sampling is one of the biggest breakthroughs from the 20^th century – specifically the Metropolis-Hastings algorithms. Of course, I may be biased - don’t take it from me, I wrote an entire PhD about getting Metropolis–Hastings to behave in molecular dynamics.

Generally, sampling algorithms assume that the distribution μ(z) is known in some way. A typical assumption is that it is given by the negative exponential of some energy function U(z)

\(\mu(z) \propto \exp\left(-U(z)\right)\)

Only the normalization constant is not known, but that is fine. An energy function arises naturally in most physical systems - especially those coming from thermodynamics or statistical mechanics - but no such scalar energy exists for the conditional distribution of fields like z = (c(x), ϕ(x)).

So how can we sample when the descriptive energy function is unknown? We learn it. We learn the distribution from forward simulation data that maps known ε(x) to its corresponding solutions (c(x), ϕ(x)). We can even learn it for different operating parameters U0, l, and F_right. Once an approximate energy function U(x) is known, we can proceed to sampling and uncertainty quantification.

A Viable Path for Generative AI

I feel this is the strongest application of generative AI in computational science. Its potential is not limited to uncertainty quantification; in fact, it is likely its weakest application. Topology and design optimization are much higher-impact applications. Indeed, even these engineering tasks can be cast in the language of distributions and sampling. Just to take one example, there is not a unique design for windmill blades that has certain properties, there are infinitely many. (Conditional) distributions enter here naturally too.

In part 2 of this post, I will focus on one GenAI method that is extremely well-suited for computational science: the score-based generative model (SGM). As it turns out, SGMs are also the fundamental technology behind StableDiffusion for image generation from text. The idea behind SGMs is essentially learning a good energy function U(x) for the conditional distribution. It does so not by building U(x) directly but rather its gradient ∇U(x). The gradient is all you need for sampling, and SGMs have a clever way of doing that too!