New GPU developments in the Madgraph CUDACPP plugin: kernel splitting, helicity streams, cuBLAS color sums

Abstract

The first production release of the CUDACPP plugin for the Madgraph5_aMC@NLO generator, which speeds up matrix element (ME) calculations for leading-order (LO) processes using a data parallel approach on vector CPUs and GPUs, was delivered in October 2024. This was described in previous publications by the team behind that effort. In this paper, I describe my work on some additional developments and optimizations of CUDACPP, mainly but not exclusively for GPUs. The new approach, which represents a major restructuring of the CUDACPP computational engine, primarily consists in splitting the ME calculation, previously performed using a single large GPU kernel, into many smaller kernels. A first batch of changes, involving the move to separate "helicity streams" and the optional offloading of QCD color sums to BLAS, was recently merged into a new CUDACPP release, in collaboration with my colleagues. Since then, I have completed a second batch of changes, involving the possibility to split the calculation into groups of Feynman diagrams in separate source code files. This new feature makes it possible to compute QCD matrix elements for physics processes with a larger number of final state gluons: in particular, I present the first performance results from CUDACPP for the $2\!\rightarrow\!6$ process $gg\!\rightarrow\!t\bar{t}gggg$ on CPUs and GPUs and the $2\!\rightarrow\!7$ process $gg\!\rightarrow\!t\bar{t}ggggg$ on CPUs, which involve over 15k and 230k Feynman diagrams, respectively. I also take this opportunity to describe in detail some previously undocumented features of the CUDACPP software, both in the GPU and vector CPU implementations.