How do neural surrogate models trained on simulations handle parameter regimes far outside their training data?

This is a great question about a practical problem in ML-assisted physics. The preprints you've provided address parts of this puzzle, though they approach it from different angles than direct extrapolation.

What the archive shows about out-of-distribution robustness

The core tension you're asking about—neural surrogates trained on a bounded region of parameter space suddenly asked to predict far outside it—surfaces most directly in work on uncertainty quantification.

When surrogates must operate beyond their training regime, the key insight is that *calibrated uncertainty estimates become essential* [AG-2024.12-1502]. That paper trains neural surrogates to simultaneously predict both the target (e.g., loop amplitudes) and an uncertainty, disentangling systematic from statistical error. The authors validate this via "pull distributions," asking: when the model says it's uncertain, is it *actually* uncertain at the right level? This matters precisely because you can't assume the model knows what it doesn't know [AG-2024.12-1502].

A complementary warning comes from work on physics-informed neural networks (PINNs) with limited data: standard confidence metrics can fail when models are constrained by physical priors. Overconfidence can masquerade as physics-constraint fidelity, which is dangerous if you're extrapolating [AG-2025.09-1089]. The authors propose an "information density quantity" to diagnose when overconfidence is real versus artifact—a diagnostic you'd want before trusting far-extrapolated predictions.

Active learning as a hedge

Rather than relying on extrapolation, the most successful approach in the archive is iterative retraining. The gravitational wave detector design work demonstrates a loop: train the surrogate, use it to propose new designs via inverse optimization, *verify those designs with the slow simulator*, then retrain [AG-2025.11-530]. This sidesteps the extrapolation problem by bringing risky regions back into the training set. It's slower than pure extrapolation but catches failure modes early.

Similarly, for high-dimensional parameter estimation in BSM physics, amortized methods (Neural Posterior Estimation, Neural Ratio Estimation) are tested via the "Test of Accuracy with Random Points" [AG-2025.02-1282]—essentially asking: does the surrogate remain valid when sampled at random, unseen points? The fact that this test is named and standardized suggests the community takes out-of-distribution failure seriously.

Scaling as a clue

One indirect signal: scaling laws for amplitude surrogates show that prediction error decreases predictably with training data size and network capacity [AG-2026.01-1235]. This means if you want to extend your safe parameter region, you have a quantified recipe—though extrapolation itself isn't addressed. The implication is that more training data buys you a larger safe zone, but the boundary is still real.

What's missing

The archive doesn't directly tackle the question: *how far can a surrogate extrapolate?* None of these papers systematically measure extrapolation range as a function of dimensionality, nonlinearity, or training set size. They instead focus on detecting when you're in danger (uncertainty calibration) and avoiding the need to extrapolate (active learning loops).