How do machine learning models ensure they preserve physics constraints like energy conservation during fast approximations?

# How Machine Learning Models Preserve Physics Constraints

This is a genuinely hard problem, and the archive shows several complementary strategies emerging.

The Core Tension

Machine learning models are function approximators — they find patterns in data. Physics constraints like energy conservation are *symmetries*, not patterns. A neural network trained on noisy experimental data has no intrinsic reason to conserve energy if the data itself is slightly noisy or incomplete. Yet when you're using ML to speed up expensive simulations, violating these constraints can cascade into nonsense predictions.

Three Main Strategies

1. Encode symmetries into the architecture itself

Rather than hoping the network learns conservation laws, you build them in from the start. In particle physics, this means using equivariant architectures — neural networks whose structure respects the mathematical symmetries of the problem (like rotations or Lorentz transformations) [AG-2026.03-1363]. The network is then *geometrically constrained* to respect these laws.

A gentler version uses "soft constraints": instead of rigidly enforcing symmetries, you add a penalty term to the loss function that encourages (but doesn't require) the network to respect them [AG-2025.11-1116]. This lets the model decide how strictly to follow each symmetry, which is useful when real experiments slightly violate idealized physics.

2. Use compressed, physically regular representations

When solving density functional theory — a quantum mechanics problem where energy minimization finds ground states — small numerical noise in the neural network's output can violate the *variational principle* (energy should always decrease as you optimize). The solution: train a variational autoencoder to learn a compressed, smooth representation of physically valid density profiles, then do energy minimization in that compressed space [AG-2024.03-1817]. By restricting the network's output to realistic profiles, you automatically preserve physical consistency.

3. Stabilize training with physics-inspired optimization

Even if you've designed the network carefully, training it can be unstable. The Energy Conserving Descent optimizer uses classical Hamiltonian dynamics (the same mathematics that governs energy conservation in mechanics) to guide the neural network's learning process, reducing fluctuations and keeping the solution near the true minimum [AG-2025.01-1027]. This is particularly useful for likelihood estimation in particle physics, where you need precision over many training runs.

A Real Example: Relativistic Hydrodynamics

In relativistic fluid simulations, you compute "conserved variables" (energy, momentum density) from "primitive variables" (density, pressure, velocity). This conversion must respect thermodynamic constraints. A recent method trains neural networks to invert this conversion ~400× faster than traditional root-finding, but only by keeping the network's error below chemical accuracy (10⁻⁶) — tight enough that the hydrodynamics code downstream stays stable [AG-2024.12-226].

The Deeper Lesson

The most robust approaches don't ask the network to *learn* physics from scratch. Instead, they ask it to learn the *residuals* — the deviations from known physics — or to learn only within a space that's pre-constrained to be physical. This is closer to how physicists actually reason: you don't rederive Newton's laws from data; you assume them and fit the details.

Notably, one system (AI-Newton) can actually *rediscover* fundamental conservation laws like energy conservation purely from multi-experiment data, though this is discovery *ex post facto*, not fast approximation [AG-2025.04-1324].