How does simulation-based inference avoid combinatorial explosion compared to traditional bump-hunt methods?

# How Simulation-Based Inference Avoids Combinatorial Explosion

The key difference is that traditional bump-hunt methods work *backwards* from data to theory, while simulation-based inference (SBI) works *forwards* from theory to data—and this reversal lets neural networks compress a massive search space without exhaustively sampling it.

The Bump-Hunt Problem

In a traditional bump hunt, you're looking for a signal hiding in high-dimensional data (say, 100+ features per event). You partition the feature space into bins, look for statistically unlikely excesses, and then try to map those excesses back to the underlying physics parameters you care about. The combinatorial explosion comes from two places: the number of possible bin configurations explodes with dimensionality, and you have to repeat the whole search for every hypothesis about what new physics might be present.

How SBI Sidesteps This

SBI inverts the problem [AG-2024.07-1503]. Instead of asking "given these data features, what physics could produce them?", you ask "given a physics hypothesis, what data would it produce?" You run your simulator (which encodes all your known physics) on millions of parameter combinations, train a neural network to learn the inverse map from data back to parameters, and then apply that network to real data once.

The neural network learns a *compressed representation* of how parameter space maps to observable space. You don't need to enumerate all possible excesses—the network finds the low-dimensional manifold where data and parameters agree. This is why SBI can handle poor detector resolution where traditional methods fail: it works directly from low-level event features without requiring you to construct explicit high-level observables like invariant mass [AG-2024.07-1503].

Scaling to Complex Problems

The efficiency gains compound when you combine multiple processes or high-dimensional parameter spaces. On global LHC analyses with four di-boson processes, SBI outperforms traditional histogram-based methods [AG-2025.09-1245]. For extreme mass-ratio black hole mergers, sequential SBI compressed an 11-dimensional search space by a factor of 10⁶–10⁷ [AG-2025.05-261]—something that would be completely infeasible with a grid search or bump-hunt binning scheme.

When simulators are expensive, multilevel variants [AG-2025.06-087] further optimize by using multiple simulators of different computational cost and accuracy, much like multilevel Monte Carlo in uncertainty quantification.

Why This Matters

SBI trades the combinatorial cost of exhaustive search for the statistical cost of learning a neural network—a much better bargain in high dimensions. It also naturally incorporates known physics structure (through the simulator itself), whereas traditional methods must explicitly construct observables that capture this structure.