Posterior Predictive Checks for Gravitational-wave Populations: Limitations and Improvements

Abstract

When selecting a model to characterize an astrophysical population, it is crucial to assess whether that model fits the data and, if not, how it can be improved. To this end, posterior predictive checks (PPCs) are a widely-used statistical test of model fit when inferring gravitational-wave source populations. However, PPCs exhibit limitations when assessing single-event parameters with large measurement uncertainty, like the spin tilt angles of the binary black holes (BBHs) observable with the LIGO-Virgo-KAGRA (LVK) detectors. When single-event inference is prior-dominated, traditional PPCs fail to flag even very poor model fits. In this work, we assess the efficacy of various alternative PPCs on poorly-constrained parameters. We compare PPCs conducted on event- vs. data-level parameters (e.g. posterior samples vs. maximum likelihood points), and explore two additional event-level PPCs: partial predictive checks and split predictive checks. Independent of measurement uncertainty, we find that PPCs on maximum likelihood parameters are always more discerning of model misspecification than any event-level PPC. However, when investigating simulated GWTC-3.0-like catalogs, none of the alternative PPCs show significant improvement over those traditionally used, indicating that at that sensitivity, any limited information in the data about spin tilts is insufficient to diagnose model misspecification. Finally, we apply our suite of PPCs to the spin magnitude and tilt distributions inferred in the most recent LVK catalog, GWTC-4.0. We conclude that the Gaussian Component Spins model used therein under-predicts BBHs with large spin magnitudes and over-predicts those with perfectly anti-aligned tilts.