Antigravity Q&A
Why does Bayesian model comparison avoid the look-elsewhere effect when scanning doesn't?
Answered by claude-haiku-4-5 · 10 May 2026
Answer
This is a great question about statistical methodology, though the preprints provided don't directly address the look-elsewhere effect or why Bayesian model comparison avoids it. Let me explain the core idea, then note what's missing.
The Core Insight
The "look-elsewhere effect" is a multiple-testing problem: if you scan many hypotheses (or parameter space regions) without accounting for how many you've tested, you'll inevitably find fluky matches to noise. Traditional scanning — like plotting significance across a parameter grid — treats each point as an independent test, so your false-positive rate compounds.
Bayesian model comparison sidesteps this through a fundamentally different accounting mechanism. When you compare models using the Bayesian evidence (the probability of the data under each model, marginalized over all parameters), you're not cherry-picking the best fit in some region — you're weighing the entire predictive power of each model. Models that fit a narrow slice of parameter space well but waste probability mass on implausible regions get penalized. This built-in "Occam's razor" [AG-2025.02-241] suppresses spurious detections.
More concretely: a scanning approach finds "the best fit anywhere." A Bayesian comparison asks, "which model predicted this data?" A model that predicts the data anywhere does worse than one that predicted it *here specifically*.
What the Archive Offers
The preprints discuss Bayesian model comparison in detail — how to compute Bayes factors robustly using methods like the Savage-Dickey density ratio [AG-2025.06-064] or evidence ratios [AG-2025.02-241] — and warn against common pitfalls like circular analysis [AG-2024.09-163]. They also show that presenting results as "Bayes factor surfaces" [AG-2024.01-1092] avoids prior-dependent biases.
However, none directly compare scanning vs. Bayesian methods in the context of the look-elsewhere effect, so I cannot cite a specific preprint that explains *why* one avoids it and the other doesn't.
Sources · 8
- 54%astro-ph.COA general Bayesian model-validation framework based on null-test evidence ratios, with an example application to global 21-cm cosmologyAG-2025.02-241
- 53%hep-phA comparison of Bayesian sampling algorithms for high-dimensional particle physics and cosmology applicationsAG-2024.09-1351
- 53%astro-ph.IMLikelihood-free Model Selection in Cosmic Reionization with Three-dimensional Tomographic 21 cm Lightcone ImagesAG-2025.02-137
- 53%astro-ph.COSavage-Dickey density ratio estimation with normalizing flows for Bayesian model comparisonAG-2025.06-064
- 51%hep-phThe Bayes factor surface for searches for new physicsAG-2024.01-1092
- 51%astro-ph.IMUse Model Averaging instead of Model Selection in Pulsar TimingAG-2024.09-163
- 50%astro-ph.IMWhat You Don't Know Won't Hurt You: Self-Consistent Hierarchical Inference with Unknown Follow-up Selection StrategiesAG-2026.05-067
- 50%gr-qcComparing astrophysical models to gravitational-wave data in the observable spaceAG-2025.07-357
Keep exploring
- How does the Occam's razor penalty change if your model has many more free parameters than competitors?
- Does Bayesian model comparison still avoid false positives if two competing models have nearly identical predictive distributions?
- What happens to the Bayes factor when the true effect actually exists in only a tiny parameter subregion?
This is a research aid — not a peer review. Verify sources before citing.