Bayesian forecasting with information theory

Abstract

Forecasting techniques for assessing the power of future experiments to discriminate between theories or discover new laws of nature are of great interest in many areas of science. In this paper, we introduce a Bayesian forecasting method using information theory. We argue that mutual information is a suitable quantity to study in this context. Besides being Bayesian, this proposal has the advantage of not relying on the choice of fiducial parameters, describing the "true" theory (which is a priori unknown), and is applicable to any probability distribution. We demonstrate that the proposed method can be used for parameter estimation and model selection, both of which are of interest concerning future experiments. We argue that mutual information has plausible interpretation in both situations. In addition, we state a number of propositions that offer information-theoretic meaning to some of the Bayesian practices such as performing multiple experiments, combining different datasets, and marginalization.