Investigating 1-Bit Quantization in Transformer-Based Top Tagging

Abstract

The increasing scale of deep learning models in high-energy physics (HEP) has posed challenges to their deployment on low-power, latency-sensitive platforms, such as FPGAs and ASICs used in trigger systems, as well as in offline data reconstruction and processing pipelines. In this work, we introduce BitParT, a 1-bit Transformer-based architecture designed specifically for the top-quark tagging method. Building upon recent advances in ultra-low-bit large language models (LLMs), we extended these ideas to the HEP domain by developing a binary-weight variant (BitParT) of the Particle Transformer (ParT) model. Our findings indicate a potential for substantial reduction in model size and computational complexity, while maintaining high tagging performance. We benchmark BitParT on the public Top Quark Tagging Reference Dataset and show that it achieves competitive performance relative to its full-precision counterpart. This work demonstrates the design of extreme quantized models for physics applications, paving the way for real-time inference in collider experiments with minimal and optimized resource usage.