What Generative AI can(not) do for Computational Science
Generative Artificial Intelligence (GenAI) is omnipresent these days. It is the umbrella term that encompasses Large Language Models (LLMs) such as ChatGPT, Gemini and Claud, image generation tools such as DALL-E and Sora, and also video generation – something that Google’s Nano Banana has gotten eerily good at!
Since I’m a computational scientist at heart, I have been pondering the use of GenAI in my field for a while. A natural question that came to mind is whether there are any (sub-)fields of scientific computing that GenAI could revolutionize?
It boils down to the exact definition of GenAI, of which there are many. The greatest common denominator between the definitions from Wikipedia, generativeai.net, and Google goes something like this:
Generative AI creates new content (text, images, music, code) by learning patterns from massive datasets, acting as a creative tool that generates original outputs.
This is a good definition because it captures what we empirically know about GenAI; they are trained on large datasets, learn patterns, but the ultimate goal is to create something new. Creativity is central to generative AI. That is not to say that large language models or image generation tools are truly creative in the human sense – I don’t believe they can ever be creative – but they do mimic human creativity, which is no small feat!
Taking the Probabilistic View
On a proper abstract level, GenAI models draw samples from a conditional probability distribution
that I call the creativity distribution. The input is most often (tokenized) text and W is the collection of all neural network weights. I am deliberately ignoring all complexity that goes into curating the input/output data, designing the network architecture (transforms and related models) and the increasingly involved (pre- and post-) training algorithms.
Large language models and image/video generation tools are inherently probabilistic; they induce controlled randomness into the output. In that way, they are generating output samples from this creativity distribution. We will never get exactly the same output for identical inputs. Probability distributions are therefore the proper high-level mathematical description of this input-output relationship.
So what about computational science? Let’s focus on hardcore computational science: solving systems of nonlinear equations, calculating eigenvalues, computing minima of functions, higher-order timestepping schemes for stiff dynamical systems, high-fidelity solvers for (partial) differential equations – these tasks are all essentially deterministic. Indeed, a given set of nonlinear equations has a finite number of solutions; Most functions have a finite number of optima; and differential equations typically only have one steady-state solution (setting aside bifurcations, which introduce multiple but still well-defined solutions) and a well-defined flowmap. At this level, there is no output distribution at play; the object we are interested in is fully deterministic and mathematically well defined. As such, I believe GenAI has no direct applications in these fields.
What is Scientific ML About?
The question, then, is not whether AI can be used in computational science, but whether generative models - designed to sample from the creativity distribution - are the right abstraction for problems whose solutions are mathematically well-defined objects. I believe not. That is not to say machine learning and AI have no place in modern computational science, they absolutely do, just not GenAI.
The use-case for scientific machine learning (SciML) is precisely in the complexity of the deterministic input-to-output mapping. Even the highly optimized numerical algorithms in existence are typically too slow for large-scale problems. An AI model that provides a fast approximation of the root of a function, eigenvalues of a matrix, steady-state solution of a PDE, …, could significantly speed up design cycles.
I am very bullish on these foundation models for physics and hardcore science: models that learn physics from data and generates fast and accurate predictions in new experiments. Computations and predictions that used to take days to weeks could potentially be reduced to ‘just’ forward inference of a neural network – on the order of minutes at the most. The prime example of such a foundation model is, of course, AlphaFold3 – an end-to-end inference pipeline for protein structure prediction. This is a task that even the most advanced and purely computational algorithms could not solve.
For fundamental physics, foundation models are proving harder to train. I haven’t done the math, but the effective dimensionality of all of physics does not seem much bigger than that of protein folding, so why haven’t we solved this problem yet? I speculate we need a different kind of neural network architecture that is tailored for fundamental science. Transformers have been instrumental in advancing GenAI but they are not the right tool for computational science. The reason is precisely because GenAI is not directly useful for deterministic systems. We have come full circle!
Uncertainty, not Creativity
There is one area of computational science that I have completely left out of my analysis: uncertainty quantification (UQ). UQ is a very broad field, but its central objective is to study how uncertainty propagates through physical models and numerical algorithms. Randomness arises because inputs, parameters, or models are uncertain. The governing equations remain deterministic. Probability enters the problem through epistemic uncertainty (limited knowledge of parameters or models) or aleatoric uncertainty (inherent variability in measurements or forcing). That is to say, given a probability distribution
describing the uncertainty in the respective inputs and model parameters, we are interested in the conditional output distribution
Which captures aleatoric uncertainty; and the marginal output distribution
which also includes epistemic uncertainty. Sampling algorithms like (Markov chain-) Monte Carlo are central to studying these last two distributions.
The careful reader will notice the similarity between the UQ output distribution and the GenAI input-output distribution. This is where I believe the most profound application of generative AI in computational science lies: learning the probability P(output) induced by physcis; and afterwards sampling it.
Score-based Generative Models (SGMs) are the strongest bridge linking UQ and GenAI. SGMs can learn the probability distribution of a few datapoints and generate new samples from said distribution. They are the underlying technology to Stable Diffusion – a powerful image and video generation model. Even more, SGMs are rooted in probability theory and stochastic calculus – both very deep branches of mathematics. Indeed, the main workhorse of score-based generative models is a stochastic differential equation whose dynamics are explicitly designed to transform probability measures. I won’t go deeper into SGMs here, but the blog on Jakiw’s website does an excellent job explaining them on a fundamental level without getting lost in the math.
In a follow-up post, I will show how GenAI – and SGMs in particular – can be used for uncertainty quantification in a concrete, real-life example.
My Final Thoughts
The central message of my post is not that generative AI is irrelevant to computational science, but that its role is often unclearly defined. At its core, computational science is concerned with computing well-defined objects governed by physical laws – however complex they may be. For such problems, determinism is not a weakness to be overcome, but a structure to be exploited. Generative models, whose defining feature is sampling from learned output distributions, are therefore not the right tool. Deterministic foundation models that learn the input-output map pave the way forward, especially when those tasks are complex or high-dimensional.
Probability enters computational science for a different reason: uncertainty. In uncertainty quantification, distributions are not learned for creativity, but arise from model uncertainty, input uncertainty and measurement noise. It is at this interface that a meaningful connection to generative modeling emerges. The future impact of generative AI in computational science will come from making uncertainty-aware computations faster, more scalable, and more faithful to the underlying physics.