If noise is inferred flexibly during analysis, how do you avoid over-fitting the model to random fluctuations in that specific dataset?

Your question targets a genuine risk in modern data analysis: when you give a model flexibility to learn noise characteristics from the data itself, how do you prevent it from fitting spurious patterns rather than real signal?

The archive addresses this through regularization and validation across independent data.

[AG-2025.01-376] tackles this head-on in gravitational-wave noise modelling. Their transdimensional Bayesian approach uses a principled prior structure (power laws + Lorentzians + shapelets) that constrains complexity *before* looking at data, rather than letting the model freely absorb noise. Crucially, they validate on "well-known events in the third Gravitational-Wave Transient Catalog"—independent astrophysical sources used to check whether the inferred noise model produces physically sensible parameter estimates (shifts up to 7% in credible intervals), not just a good fit to one dataset.

[AG-2026.04-372] uses a complementary strategy for glitch-robust inference: they train on *synthetic* contaminated data generated by a neural glitch simulator, then test on held-out real data. This separation prevents overfitting to idiosyncrasies of one observing run. They also employ multiple validation metrics—not just standard coverage diagnostics, but the continuous ranked probability score—to catch cases where a model fits the training data well but produces poorly calibrated predictions.

[AG-2024.08-021] avoids the trap differently: rather than flexibly inferring noise from a single long stretch, they break the mission into shorter segments with different noise characteristics, then combine them. Each segment's noise is constrained by physical expectations (satellite operations, known glitches), and the model is validated by comparing to Fisher Information Matrix estimates of what's theoretically optimal.

The common thread: don't infer noise parameters from the signal you're trying to measure. Use independent validation data, impose prior structure grounded in physics, or split the analysis so noise and signal constraints come from different parts of the dataset.