Predictions with limited data: Bayesian (X)PINNs, entanglement surfaces and overconfidence

Abstract

Solving differential equations from limited or noisy data remains a key challenge for physics-informed neural networks (PINNs), which are typically applied to already known and smooth solutions. In this work, we explore Bayesian PINNs and extended PINNs, (B-(X)PINNs), to solve non-linear second order differential equation typical for high energy theory, where data is only available from the boundary domain, to benchmark suitable approaches to PINNs in this category. In particular, we consider an entangling surface; a differential equation typical in holography. We perform asymptotic analysis to generate analytical training data from the boundary domain. We also explore the meaning of overconfidence in models that are constrained by physical priors and argue that standard overconfidence metrics are not suitable to consider when dealing with B-PINNs. Overconfidence can be a natural feature and not a bug in systems with soft or hard constraints on the loss function; one have to look at when the overconfidence is an artifact of the model adhering to the physics constraints. To diagnose this effect, we introduce an information density quantity, and a local physics-constraint coupling (PCC) metric, to capture locally to what extent the enforced physics collapses the posterior distribution. We also consider these quantities for a Liouville-type equation and the Van der Pol equation to probe apparent overconfidence further.