Deep Reinforcement Learning Coordinated Receiver Beamforming for Millimeter-Wave Train-ground Communications

Abstract

As more and more people choose high-speed rail (HSR) as a means of transportation for short trips, there is ever growing demand of high quality of multimedia services. With its rich spectrum resources, millimeter wave (mm-wave) communications can satisfy the high network capacity requirements for HSR. Also, it is possible for receivers (RXs) to be equipped with antenna arrays in mm-wave communication systems due to its short wavelength. However, as HSRs run with high speed, the received signal power (RSP) varies rapidly over a cell and it is the lowest at the edge of the cell compared to other locations. Consequently, it is necessary to conduct research on RX beamforming for HSR in mm-wave band to improve the quality of the received signal. In this paper, we focus on RX beamforming for a mm-wave train-ground communication system. To improve the RSP, we propose an effective RX beamforming scheme based on deep reinforcement learning (DRL), and develop a deep Q-network (DQN) algorithm to train and determine the optimal RX beam direction with the purpose of maximizing average RSP. Through extensive simulations, we demonstrate that the proposed scheme has better performance than the four baseline schemes in terms of average RSP at most positions on the railway.