GR-Athena++: magnetohydrodynamical evolution with dynamical space-time

Abstract

We present a self-contained overview of GR-Athena++, a general-relativistic magnetohydrodynamics (GRMHD) code, that incorporates treatment of dynamical space-time, based on the recent work of (Daszuta+, 2021)[49] and (Cook+, 2023)[45]. General aspects of the Athena++ framework we build upon, such as oct-tree based, adaptive mesh refinement (AMR) and constrained transport, together with our modifications, incorporating the Z4c formulation of numerical relativity, judiciously coupled, enables GRMHD with dynamical space-times. Initial verification testing of GR-Athena++ is performed through benchmark problems that involve isolated and binary neutron star space-times. This leads to stable and convergent results. Gravitational collapse of a rapidly rotating star through black hole formation is shown to be correctly handled. In the case of non-rotating stars, magnetic field instabilities are demonstrated to be correctly captured with total relative violation of the divergence-free constraint remaining near machine precision. The use of AMR is show-cased through investigation of the Kelvin-Helmholtz instability which is resolved at the collisional interface in a merger of magnetised binary neutron stars. The underlying task-based computational model enables GR-Athena++ to achieve strong scaling efficiencies above $80\%$ in excess of $10^5$ CPU cores and excellent weak scaling up to $\sim 5 \times 10^5$ CPU cores in a realistic production setup. GR-Athena++ thus provides a viable path towards robust simulation of GRMHD flows in strong and dynamical gravity with exascale high performance computational infrastructure.