What if you retrained the surrogate on mixed geometry configurations—would it generalize better than single-parameter optimization?

Your question is asking whether training a surrogate model on *diverse* geometry configurations would help it generalize better than tuning a *single* parameter at a time. This is a smart question about the design of machine-learning-assisted optimization pipelines.

The preprints here address surrogate-guided optimization in quantum systems, but they don't directly compare single-parameter versus mixed-geometry training strategies. However, I can extract some relevant principles:

What the archive shows about surrogate training:

The core idea—using machine-learning surrogates to replace expensive simulations—is well-established [AG-2024.07-2351]. Surrogates work by learning the shape of the objective landscape from a limited number of function evaluations, then guiding the optimizer toward promising regions without running the full simulation each time.

Generalization across parameter space:

One key result is that learned optimizers trained on *generic* problems do generalize out-of-distribution better when they incorporate problem structure. In the quantum setting, using the Fubini-Study metric tensor (which encodes quantum geometry) during surrogate training improved generalization compared to naive machine-learning approaches [AG-2024.07-2231]. The intuition: a surrogate that "knows" the geometry of your optimization landscape learns more robust decision rules.

By extension, training on mixed-geometry configurations would likely give your surrogate exposure to the *manifold structure* of your problem space rather than a thin slice of it. This should help it learn which features (parameter combinations, geometry modes) actually drive the objective, rather than overfitting to correlations in a single-parameter regime.

The caveat:

The archive doesn't directly address whether mixed-geometry training beats single-parameter optimization in your specific setting—you'd need to benchmark that empirically on your problem.