arXiv subjects

Sheng Chen

Publications and source records attributed to Sheng Chen.

At least 127 records · Page 7Linked to original sources

arXiv2022

Transformer-Empowered 6G Intelligent Networks: From Massive MIMO Processing to Semantic Communication

It is anticipated that 6G wireless networks will accelerate the convergence of the physical and cyber worlds and enable a paradigm-shift in the way we deploy and exploit communication networks. Machine learning, in particular deep learning (DL), is expected to be one of the key technological enablers of 6G by offering a new paradigm for the design and optimization of networks with a high level of intelligence. In this article, we introduce an emerging DL architecture, known as the transformer, and discuss its potential impact on 6G network design. We first discuss the differences between the transformer and classical DL architectures, and emphasize the transformer's self-attention mechanism and strong representation capabilities, which make it particularly appealing for tackling various challenges in wireless network design. Specifically, we propose transformer-based solutions for various massive multiple-input multiple-output (MIMO) and semantic communication problems, and show their superiority compared to other architectures. Finally, we discuss key challenges and open issues in transformer-based solutions, and identify future research directions for their deployment in intelligent 6G networks.

cs.IT↗

arXiv2022

Integrated Sensing and Communication with mmWave Massive MIMO: A Compressed Sampling Perspective

Integrated sensing and communication (ISAC) has opened up numerous game-changing opportunities for realizing future wireless systems. In this paper, we propose an ISAC processing framework relying on millimeter-wave (mmWave) massive multiple-input multiple-output (MIMO) systems. Specifically, we provide a compressed sampling (CS) perspective to facilitate ISAC processing, which can not only recover the high-dimensional channel state information or/and radar imaging information, but also significantly reduce pilot overhead. First, an energy-efficient widely spaced array (WSA) architecture is tailored for the radar receiver, which enhances the angular resolution of radar sensing at the cost of angular ambiguity. Then, we propose an ISAC frame structure for time-varying ISAC systems considering different timescales. The pilot waveforms are judiciously designed by taking into account both CS theories and hardware constraints induced by hybrid beamforming (HBF) architecture. Next, we design the dedicated dictionary for WSA that serves as a building block for formulating the ISAC processing as sparse signal recovery problems. The orthogonal matching pursuit with support refinement (OMP-SR) algorithm is proposed to effectively solve the problems in the existence of the angular ambiguity. We also provide a framework for estimating the Doppler frequencies during payload data transmission to guarantee communication performances. Simulation results demonstrate the good performances of both communications and radar sensing under the proposed ISAC framework.

cs.IT↗

arXiv2022

Multiple-Objective Packet Routing Optimization for Aeronautical ad-hoc Networks

Providing Internet service above the clouds is of ever-increasing interest and in this context aeronautical {\it{ad-hoc}} networking (AANET) constitutes a promising solution. However, the optimization of packet routing in large ad hoc networks is quite challenging. In this paper, we develop a discrete $ε$ multi-objective genetic algorithm ($ε$-DMOGA) for jointly optimizing the end-to-end latency, the end-to-end spectral efficiency (SE), and the path expiration time (PET) that specifies how long the routing path can be relied on without re-optimizing the path. More specifically, a distance-based adaptive coding and modulation (ACM) scheme specifically designed for aeronautical communications is exploited for quantifying each link's achievable SE. Furthermore, the queueing delay at each node is also incorporated into the multiple-objective optimization metric. Our $ε$-DMOGA assisted multiple-objective routing optimization is validated by real historical flight data collected over the Australian airspace on two selected representative dates.

cs.IT↗

arXiv2022

Sampling Is All You Need on Modeling Long-Term User Behaviors for CTR Prediction

Rich user behavior data has been proven to be of great value for Click-Through Rate (CTR) prediction applications, especially in industrial recommender, search, or advertising systems. However, it's non-trivial for real-world systems to make full use of long-term user behaviors due to the strict requirements of online serving time. Most previous works adopt the retrieval-based strategy, where a small number of user behaviors are retrieved first for subsequent attention. However, the retrieval-based methods are sub-optimal and would cause more or less information losses, and it's difficult to balance the effectiveness and efficiency of the retrieval algorithm. In this paper, we propose SDIM (Sampling-based Deep Interest Modeling), a simple yet effective sampling-based end-to-end approach for modeling long-term user behaviors. We sample from multiple hash functions to generate hash signatures of the candidate item and each item in the user behavior sequence, and obtain the user interest by directly gathering behavior items associated with the candidate item with the same hash signature. We show theoretically and experimentally that the proposed method performs on par with standard attention-based models on modeling long-term user behaviors, while being sizable times faster. We also introduce the deployment of SDIM in our system. Specifically, we decouple the behavior sequence hashing, which is the most time-consuming part, from the CTR model by designing a separate module named BSE (behavior Sequence Encoding). BSE is latency-free for the CTR server, enabling us to model extremely long user behaviors. Both offline and online experiments are conducted to demonstrate the effectiveness of SDIM. SDIM now has been deployed online in the search system of Meituan APP.

cs.IR↗

arXiv2022

KSSOLV 2.0: An efficient MATLAB toolbox for solving the Kohn-Sham equations with plane-wave basis set

KSSOLV (Kohn-Sham Solver) is a MATLAB toolbox for performing Kohn-Sham density functional theory (DFT) calculations with a plane-wave basis set. KSSOLV 2.0 preserves the design features of the original KSSOLV software to allow users and developers to easily set up a problem and perform ground-state calculations as well as to prototype and test new algorithms. Furthermore, it includes new functionalities such as new iterative diagonalization algorithms, k-point sampling for electron band structures, geometry optimization and advanced algorithms for performing DFT calculations with local, semi-local, and hybrid exchange-correlation functionals. It can be used to study the electronic structures of both molecules and solids. We describe these new capabilities in this work through a few use cases. We also demonstrate the numerical accuracy and computational efficiency of KSSOLV on a variety of examples.

physics.comp-ph↗

arXiv2022

CLIP2TV: Align, Match and Distill for Video-Text Retrieval

Modern video-text retrieval frameworks basically consist of three parts: video encoder, text encoder and the similarity head. With the success on both visual and textual representation learning, transformer based encoders and fusion methods have also been adopted in the field of video-text retrieval. In this report, we present CLIP2TV, aiming at exploring where the critical elements lie in transformer based methods. To achieve this, We first revisit some recent works on multi-modal learning, then introduce some techniques into video-text retrieval, finally evaluate them through extensive experiments in different configurations. Notably, CLIP2TV achieves 52.9@R1 on MSR-VTT dataset, outperforming the previous SOTA result by 4.1%.

cs.CV↗

arXiv2022

CausalMTA: Eliminating the User Confounding Bias for Causal Multi-touch Attribution

Multi-touch attribution (MTA), aiming to estimate the contribution of each advertisement touchpoint in conversion journeys, is essential for budget allocation and automatically advertising. Existing methods first train a model to predict the conversion probability of the advertisement journeys with historical data and calculate the attribution of each touchpoint using counterfactual predictions. An assumption of these works is the conversion prediction model is unbiased, i.e., it can give accurate predictions on any randomly assigned journey, including both the factual and counterfactual ones. Nevertheless, this assumption does not always hold as the exposed advertisements are recommended according to user preferences. This confounding bias of users would lead to an out-of-distribution (OOD) problem in the counterfactual prediction and cause concept drift in attribution. In this paper, we define the causal MTA task and propose CausalMTA to eliminate the influence of user preferences. It systemically eliminates the confounding bias from both static and dynamic preferences to learn the conversion prediction model using historical data. We also provide a theoretical analysis to prove CausalMTA can learn an unbiased prediction model with sufficient data. Extensive experiments on both public datasets and the impression data in an e-commerce company show that CausalMTA not only achieves better prediction performance than the state-of-the-art method but also generates meaningful attribution credits across different advertising channels.

cs.IR↗

arXiv2022

Arithmetic purity of strong approximation for complete toric varieties

In this article, we establish the arithmetic purity of strong approximation for smooth loci of weighted projective spaces. By using this result and the descent method, we also prove that the arithmetic purity of strong approximation with Brauer-Manin obstruction holds for any smooth and complete toric variety.

math.AG↗

arXiv2022

A Novel DeBERTa-based Model for Financial Question Answering Task

As a rising star in the field of natural language processing, question answering systems (Q&A Systems) are widely used in all walks of life. Compared with other scenarios, the applicationin financial scenario has strong requirements in the traceability and interpretability of the Q&A systems. In addition, since the demand for artificial intelligence technology has gradually shifted from the initial computational intelligence to cognitive intelligence, this research mainly focuses on the financial numerical reasoning dataset - FinQA. In the shared task, the objective is to generate the reasoning program and the final answer according to the given financial report containing text and tables. We use the method based on DeBERTa pre-trained language model, with additional optimization methods including multi-model fusion, training set combination on this basis. We finally obtain an execution accuracy of 68.99 and a program accuracy of 64.53, ranking No. 4 in the 2022 FinQA Challenge.

cs.CL↗

arXiv2022

Contrastive Information Transfer for Pre-Ranking Systems

Real-word search and recommender systems usually adopt a multi-stage ranking architecture, including matching, pre-ranking, ranking, and re-ranking. Previous works mainly focus on the ranking stage while very few focus on the pre-ranking stage. In this paper, we focus on the information transfer from ranking to pre-ranking stage. We propose a new Contrastive Information Transfer (CIT) framework to transfer useful information from ranking model to pre-ranking model. We train the pre-ranking model to distinguish the positive pair of representation from a set of positive and negative pairs with a contrastive objective. As a consequence, the pre-ranking model can make full use of rich information in ranking model's representations. The CIT framework also has the advantage of alleviating selection bias and improving the performance of recall metrics, which is crucial for pre-ranking models. We conduct extensive experiments including offline datasets and online A/B testing. Experimental results show that CIT achieves superior results than competitive models. In addition, a strict online A/B testing at one of the world's largest E-commercial platforms shows that the proposed model achieves 0.63\% improvements on CTR and 1.64\% improvements on VBR. The proposed model now has been deployed online and serves the main traffic of this system, contributing a remarkable business growth.

cs.IR↗

arXiv2022

AutoFAS: Automatic Feature and Architecture Selection for Pre-Ranking System

Industrial search and recommendation systems mostly follow the classic multi-stage information retrieval paradigm: matching, pre-ranking, ranking, and re-ranking stages. To account for system efficiency, simple vector-product based models are commonly deployed in the pre-ranking stage. Recent works consider distilling the high knowledge of large ranking models to small pre-ranking models for better effectiveness. However, two major challenges in pre-ranking system still exist: (i) without explicitly modeling the performance gain versus computation cost, the predefined latency constraint in the pre-ranking stage inevitably leads to suboptimal solutions; (ii) transferring the ranking teacher's knowledge to a pre-ranking student with a predetermined handcrafted architecture still suffers from the loss of model performance. In this work, a novel framework AutoFAS is proposed which jointly optimizes the efficiency and effectiveness of the pre-ranking model: (i) AutoFAS for the first time simultaneously selects the most valuable features and network architectures using Neural Architecture Search (NAS) technique; (ii) equipped with ranking model guided reward during NAS procedure, AutoFAS can select the best pre-ranking architecture for a given ranking teacher without any computation overhead. Experimental results in our real world search system show AutoFAS consistently outperforms the previous state-of-the-art (SOTA) approaches at a lower computing cost. Notably, our model has been adopted in the pre-ranking module in the search system of Meituan, bringing significant improvements.

cs.IR↗

arXiv2022

Mobility Support for Millimeter Wave Communications: Opportunities and Challenges

Millimeter-wave (mmWave) communication technology offers a potential and promising solution to support 5G and B5G wireless networks in dynamic scenarios and applications. However, mobility introduces many challenges as well as opportunities to mmWave applications. To address these problems, we conduct a survey of the opportunities and technologies to support mmWave communications in mobile scenarios. Firstly, we summarize the mobile scenarios where mmWave communications are exploited, including indoor wireless local area network (WLAN) or wireless personal area network (WPAN), cellular access, vehicle-to-everything (V2X), high speed train (HST), unmanned aerial vehicle (UAV), and the new space-air-ground-sea communication scenarios. Then, to address users' mobility impact on the system performance in different application scenarios, we introduce several representative mobility models in mmWave systems, including human mobility, vehicular mobility, high speed train mobility and ship mobility. Next we survey the key challenges and existing solutions to mmWave applications, such as channel modeling, channel estimation, anti-blockage, and capacity improvement. Lastly, we discuss the open issues concerning mobility-aware mmWave communications that deserve further investigation. In particular, we highlight future heterogeneous mobile networks, dynamic resource management, artificial intelligence (AI) for mobility and integration of geographical information, deployment of large intelligent surface and reconfigurable antenna technology, and finally, the evolution to Terahertz (THz) communications.

cs.IT↗

arXiv2021

Deep Learning for Beam-Management: State-of-the-Art, Opportunities and Challenges

Benefiting from huge bandwidth resources, millimeter-wave (mmWave) communications provide one of the most promising technologies for next-generation wireless networks. To compensate for the high pathloss of mmWave signals, large-scale antenna arrays are required both at the base stations and user equipment to establish directional beamforming, where beam-management is adopted to acquire and track the optimal beam pair having the maximum received power. Naturally, narrow beams are required for achieving high beamforming gain, but they impose enormous training overhead and high sensitivity to blockages. As a remedy, deep learning (DL) may be harnessed for beam-management. First, the current state-of-the-art is reviewed, followed by the associated challenges and future research opportunities. We conclude by highlighting the associated DL design insights and novel beam-management mechanisms.

cs.IT↗

arXiv2021

Physics-Guided Generative Adversarial Networks for Sea Subsurface Temperature Prediction

Sea subsurface temperature, an essential component of aquatic wildlife, underwater dynamics and heat transfer with the sea surface, is affected by global warming in climate change. Existing research is commonly based on either physics-based numerical models or data based models. Physical modeling and machine learning are traditionally considered as two unrelated fields for the sea subsurface temperature prediction task, with very different scientific paradigms (physics-driven and data-driven). However, we believe both methods are complementary to each other. Physical modeling methods can offer the potential for extrapolation beyond observational conditions, while data-driven methods are flexible in adapting to data and are capable of detecting unexpected patterns. The combination of both approaches is very attractive and offers potential performance improvement. In this paper, we propose a novel framework based on generative adversarial network (GAN) combined with numerical model to predict sea subsurface temperature. First, a GAN-based model is used to learn the simplified physics between the surface temperature and the target subsurface temperature in numerical model. Then, observation data are used to calibrate the GAN-based model parameters to obtain better prediction. We evaluate the proposed framework by predicting daily sea subsurface temperature in the South China sea. Extensive experiments demonstrate the effectiveness of the proposed framework compared to existing state-of-the-art methods.

cs.LG↗

arXiv2021

Structure Parameter Optimized Kernel Based Online Prediction with a Generalized Optimization Strategy for Nonstationary Time Series

In this paper, sparsification techniques aided online prediction algorithms in a reproducing kernel Hilbert space are studied for nonstationary time series. The online prediction algorithms as usual consist of the selection of kernel structure parameters and the kernel weight vector updating. For structure parameters, the kernel dictionary is selected by some sparsification techniques with online selective modeling criteria, and moreover the kernel covariance matrix is intermittently optimized in the light of the covariance matrix adaptation evolution strategy (CMA-ES). Optimizing the real symmetric covariance matrix can not only improve the kernel structure's flexibility by the cross relatedness of the input variables, but also partly alleviate the prediction uncertainty caused by the kernel dictionary selection for nonstationary time series. In order to sufficiently capture the underlying dynamic characteristics in prediction-error time series, a generalized optimization strategy is designed to construct the kernel dictionary sequentially in multiple kernel connection modes. The generalized optimization strategy provides a more self-contained way to construct the entire kernel connections, which enhances the ability to adaptively track the changing dynamic characteristics. Numerical simulations have demonstrated that the proposed approach has superior prediction performance for nonstationary time series.

eess.SP↗

arXiv2021

Which Heroes to Pick? Learning to Draft in MOBA Games with Neural Networks and Tree Search

Hero drafting is essential in MOBA game playing as it builds the team of each side and directly affects the match outcome. State-of-the-art drafting methods fail to consider: 1) drafting efficiency when the hero pool is expanded; 2) the multi-round nature of a MOBA 5v5 match series, i.e., two teams play best-of-N and the same hero is only allowed to be drafted once throughout the series. In this paper, we formulate the drafting process as a multi-round combinatorial game and propose a novel drafting algorithm based on neural networks and Monte-Carlo tree search, named JueWuDraft. Specifically, we design a long-term value estimation mechanism to handle the best-of-N drafting case. Taking Honor of Kings, one of the most popular MOBA games at present, as a running case, we demonstrate the practicality and effectiveness of JueWuDraft when compared to state-of-the-art drafting methods.

cs.AI↗

arXiv2021

Deep Learning Assisted Calibrated Beam Training for Millimeter-Wave Communication Systems

Huge overhead of beam training imposes a significant challenge in millimeter-wave (mmWave) wireless communications. To address this issue, in this paper, we propose a wide beam based training approach to calibrate the narrow beam direction according to the channel power leakage. To handle the complex nonlinear properties of the channel power leakage, deep learning is utilized to predict the optimal narrow beam directly. Specifically, three deep learning assisted calibrated beam training schemes are proposed. The first scheme adopts convolution neural network to implement the prediction based on the instantaneous received signals of wide beam training. We also perform the additional narrow beam training based on the predicted probabilities for further beam direction calibrations. However, the first scheme only depends on one wide beam training, which lacks the robustness to noise. To tackle this problem, the second scheme adopts long-short term memory (LSTM) network for tracking the movement of users and calibrating the beam direction according to the received signals of prior beam training, in order to enhance the robustness to noise. To further reduce the overhead of wide beam training, our third scheme, an adaptive beam training strategy, selects partial wide beams to be trained based on the prior received signals. Two criteria, namely, optimal neighboring criterion and maximum probability criterion, are designed for the selection. Furthermore, to handle mobile scenarios, auxiliary LSTM is introduced to calibrate the directions of the selected wide beams more precisely. Simulation results demonstrate that our proposed schemes achieve significantly higher beamforming gain with smaller beam training overhead compared with the conventional and existing deep-learning based counterparts.

eess.SP↗

arXiv2021

A note on tight cuts in matching-covered graphs

Edmonds, Lovász, and Pulleyblank showed that if a matching covered graph has a nontrivial tight cut, then it also has a nontrivial ELP-cut. Carvalho et al. gave a stronger conjecture: if a matching covered graph has a nontrivial tight cut $C$, then it also has a nontrivial ELP-cut that does not cross $C$. Chen, et al gave a proof of the conjecture. This note is inspired by the paper of Carvalho et al. We give a simplified proof of the conjecture, and prove the following result which is slightly stronger than the conjecture: if a nontrivial tight cut $C$ of a matching covered graph $G$ is not an ELP-cut, then there is a sequence $G_1=G, G_2,\ldots,G_r, r\geq2$ of matching covered graphs, such that for $i=1, 2,\ldots, r-1$, $G_i$ has an ELP-cut $C_i$, and $G_{i+1}$ is a $C_i$-contraction of $G_i$, and $C$ is a $2$-separation cut of $G_r$.

math.CO↗