Skip to main content
Back to timeline
arXivSource publication:

Learn a function representation once, then freeze it: Daedalus beats GAOT on 23 of 28 PDE tasks and trains 2.1 times faster on average

Synopsis

Daedalus separates function-representation learning from operator learning: a functional autoencoder is pre-trained only on analytic functions synthesized on the fly, with no solver and no labels, and the latent space it induces is frozen and reused across tasks. On 28 benchmark problems spanning fluid dynamics, solid mechanics, acoustics, and the Helmholtz and Poisson equations, operators on the frozen representation have lower relative error than the end-to-end framework GAOT on 23 of them and train 2.1 times faster per problem on average; that comparison excludes the one-time backbone pre-training.

Source-provided article image: Self-Supervised Learning of Universal Latent Spaces for Operator Learning
arXiv

Interpretation

A function representation can be learned once on its own, as pre-trained representations were in natural language processing and computer vision, and then reused unchanged across partial-differential-equation tasks instead of being rebuilt for each one. Earlier neural operators fitted their representation maps together with the operator against labeled data for a single task, so the reduced space was a by-product of one supervised problem and was rebuilt whenever the equation, geometry, discretization, or modality changed. One frozen backbone reconstructs PDE fields of several kinds and natural photographs never seen in training, including compressible flow with shocks, hyperelastic stress fields, and acoustic waves in a layered medium, all differing from the training distribution in physics and geometry.

Freezing the representation maps lets a dataset be encoded once and the operator trained entirely on cached embeddings, so the cost of a step no longer grows with the resolution of the data. End-to-end frameworks encode and decode the full point cloud at every step, so step cost rises with the number of points and dense data must be downsampled to fit in memory. From 4 thousand to 67 million points per function, latent training runs at the same rate and within the same memory; on a dense problem of 262 thousand points in 4096 partitions, a single 24 GB consumer GPU trains a 101-million-parameter processor, where the largest GAOT that fits has about one million parameters.

Each partition is carried into a common local frame before encoding, so position, size, amplitude, and optionally orientation are removed from the learned embedding and only the normalized morphology remains. Earlier nonlinear representation maps had no explicit basis, and the interpretability of a fixed basis was lost along with it, while transformations such as translation and scaling could only be learned approximately. On an unstructured point cloud the encode-decode map is exactly equivariant to translation, dilation, and amplitude, with an equivariance error well below the reconstruction error; for rotation the reconstruction error stays constant over a full turn and the equivariance error stays below it.

The latent space can be studied in its own right: the whole hypersphere decodes to valid fields, and distances on the sphere track distances between the decoded fields. The latent spaces of existing neural operators are shaped by a single operator and entangled with it, so they cannot be examined apart from that task. Across embedding pairs built at controlled cosine similarity, the median of the two distances stays within about 5% of the line of slope one over the whole range, so an objective defined on embeddings measures what the corresponding objective on fields would.

Perspective

The demonstrated setting is two-dimensional: a high-dimensional backbone for analysis and a low-dimensional one for operators, both pre-trained on multi-mode anisotropic sinusoidal functions on the unit disk. The operator benchmark covers 28 problems, 17 time dependent and 11 steady, on fixed grids and on point clouds whose geometry varies between samples. It suits researchers and engineers concerned with the training cost and memory footprint of PDE surrogates.

The representation is bounded by a frequency band fixed by the training distribution: a partition whose effective spatial frequency lies outside that band cannot be represented accurately, and how finely a field can be partitioned is limited by its sampling, with overlapping partitions relaxing that bound but raising each partition's effective frequency. Rotation equivariance is available but not enforced in the trained backbones, and on four problems of the accuracy benchmark GAOT is more than 5% more accurate than Daedalus.