ISAC and Vision Fusion for Fine-Grained Low-Altitude Target Recognition

Abstract

In this paper, we propose an integrated sensing and communications (ISAC) and vision fusion framework for fine-grained low-altitude target recognition. Specifically, we first utilize ISAC system to estimate the position of the low-altitude target. Then we adjust the working parameters of the Pan-TiltZoom (PTZ) camera based on the estimated target position, such that the camera can capture the image of tiny low-altitude target from several hundred meters away. After obtaining the wireless echo signal and visual image of low-altitude target, we employ the short-time Fourier transform (STFT) to obtain the micro-Doppler (mD) spectrum of the target from wireless echo signal, and design a conditional generative adversarial network (cGAN)-based denoising network to optimize the quality of the mD spectrum. Meanwhile, we employ YOLOv11 to detect the low-altitude target from visual image, and then crop the smallsized feature image of the target from the original image. Next, we design a fine-grained low-altitude target recognition network with MobileViT, which can fuse the optimized mD spectrum and the cropped feature image to distinguish the subcategory of low-altitude target. Moreover, we generate a joint ISAC and vision dataset (JIVD) for low-altitude target monitoring based on AirSim and Wireless InSite, which includes diverse target subcategories, scenarios, and weather conditions. The effectiveness and superiority of the proposed scheme have been demonstrated by simulation results.