Discrete Codebook Design for Self-interference Suppression in mmWave ISAC

Abstract

This paper presents discrete codebook synthesis methods for self-interference (SI) suppression in a mmWave device, designed to support full-duplex (FD) integrated sensing and communication (ISAC). We formulate a signal-to-interference-and-noise ratio (SINR) maximization problem that optimizes the receiver (RX) and transmitter (TX) codewords, aimed at suppressing the near-field SI signal while maintaining the beamforming gain in the far-field sensing directions. The formulation considers the practical constraints of discrete RX and TX codebooks with quantized phase settings, as well as a TX beamforming gain requirement in the specified communication direction. Under an alternating optimization framework, the RX and TX codewords are iteratively optimized, with one fixed while the other is optimized. When the TX codeword is fixed, the RX codeword optimization problem is formulated as an integer quadratic fractional programming (IQFP) problem. Using Dinkelbach's algorithm, we transform it into a sequence of subproblems in which the numerator and denominator are decoupled, and solve these subproblems efficiently by the spherical search (SS) method. This approach is referred to as FP-SS. When the RX codeword is fixed, the TX codeword optimization is similarly an IQFP problem, but an additional TX beamforming constraint for communication must be considered; it is solved through Dinkelbach's transformation followed by the constrained spherical search (CSS), which we refer to as FP-CSS. We prove that both methods find the optimal solutions to their respective codebook optimization problems. Simulations show that FP-SS and FP-CSS achieve the same SI suppression performance as their corresponding exhaustive search (ES) methods, confirming their optimality, but at much lower complexity. Integrating them into the alternating optimization framework yields even better SI suppression performance.