Data-Free Knowledge Distillation for LiDAR-Aided Beam Tracking in MmWave Systems

Abstract

We propose a data-free knowledge distillation (DF- KD) framework for LiDAR-aided mmWave beam tracking, where the objective is to predict the optimal current and future beams from a sequence of past LiDAR measurements. Specifically, we propose a knowledge inversion approach where a generator synthesizes LiDAR-like sequences from random noise, using a metadata loss to align the teachers internal feature statistics of synthetic and real data, without access to raw LiDAR samples. The student model is then trained exclusively on the synthetic data using either the Kullback- Leibler (KL) divergence loss or a proposed mean squared error (MSE) loss between the teachers and students raw output logits. Simulation results on the DeepSense dataset demonstrate the effectiveness of the proposed approach. In particular, the proposed convolutional neural network-gated recurrent unit (CNN-GRU) teacher architecture yields superior DF-KD student performance compared to GRU-only alternatives, and the MSE loss achieves performance comparable to the standard KD loss while requiring fewer hyperparamete