arXiv subjects

Jie Li

Publications and source records attributed to Jie Li.

At least 613 records · Page 34Linked to original sources

arXiv2020

Video Face Super-Resolution with Motion-Adaptive Feedback Cell

Video super-resolution (VSR) methods have recently achieved a remarkable success due to the development of deep convolutional neural networks (CNN). Current state-of-the-art CNN methods usually treat the VSR problem as a large number of separate multi-frame super-resolution tasks, at which a batch of low resolution (LR) frames is utilized to generate a single high resolution (HR) frame, and running a slide window to select LR frames over the entire video would obtain a series of HR frames. However, duo to the complex temporal dependency between frames, with the number of LR input frames increase, the performance of the reconstructed HR frames become worse. The reason is in that these methods lack the ability to model complex temporal dependencies and hard to give an accurate motion estimation and compensation for VSR process. Which makes the performance degrade drastically when the motion in frames is complex. In this paper, we propose a Motion-Adaptive Feedback Cell (MAFC), a simple but effective block, which can efficiently capture the motion compensation and feed it back to the network in an adaptive way. Our approach efficiently utilizes the information of the inter-frame motion, the dependence of the network on motion estimation and compensation method can be avoid. In addition, benefiting from the excellent nature of MAFC, the network can achieve better performance in the case of extremely complex motion scenarios. Extensive evaluations and comparisons validate the strengths of our approach, and the experimental results demonstrated that the proposed framework is outperform the state-of-the-art methods.

cs.CV↗

arXiv2020

On the Age of Information for Multicast Transmission with Hard Deadlines in IoT Systems

We consider the multicast transmission of a real-time Internet of Things (IoT) system, where a server transmits time-stamped status updates to multiple IoT devices. We apply a recently proposed metric, named age of information (AoI), to capture the timeliness of the information delivery. The AoI is defined as the time elapsed since the generation of the most recently received status update. Different from the existing studies that considered either multicast transmission without hard deadlines or unicast transmission with hard deadlines, we enforce a hard deadline for the service time of multicast transmission. This is important for many emerging multicast IoT applications, where the outdated status updates are useless for IoT devices. Specifically, the transmission of a status update is terminated when either the hard deadline expires or a sufficient number of IoT devices successfully receive the status update. We first calculate the distributions of the service time for all possible reception outcomes at IoT devices, and then derive a closed-form expression of the average AoI. Simulations validate the performance analysis, which reveals that: 1) the multicast transmission with hard deadlines achieves a lower average AoI than that without hard deadlines; and 2) there exists an optimal value of the hard deadline that minimizes the average AoI.

cs.IT↗

arXiv2020

Depth Based Semantic Scene Completion with Position Importance Aware Loss

Semantic Scene Completion (SSC) refers to the task of inferring the 3D semantic segmentation of a scene while simultaneously completing the 3D shapes. We propose PALNet, a novel hybrid network for SSC based on single depth. PALNet utilizes a two-stream network to extract both 2D and 3D features from multi-stages using fine-grained depth information to efficiently captures the context, as well as the geometric cues of the scene. Current methods for SSC treat all parts of the scene equally causing unnecessary attention to the interior of objects. To address this problem, we propose Position Aware Loss(PA-Loss) which is position importance aware while training the network. Specifically, PA-Loss considers Local Geometric Anisotropy to determine the importance of different positions within the scene. It is beneficial for recovering key details like the boundaries of objects and the corners of the scene. Comprehensive experiments on two benchmark datasets demonstrate the effectiveness of the proposed method and its superior performance. Models and Video demo can be found at: https://github.com/UniLauX/PALNet.

cs.CV↗

arXiv2020

Transition-metal phthalocyanine monolayers as new Chern insulators

To explore new materials for the realization of the quantum anomalous Hall effect (QAHE), we studied electronic, magnetic and topological properties of transition-metal phthalocyanine (TMPc) monolayers in a square lattice. Many of them have large topologically nontrivial band gaps and integer Chern numbers. In particular, the Fermi level of MoPc lies right in the band gap (Eg=8.1 meV) and has large magnetic anisotropy energy (MAE=2.0 meV per Mo atom) and a Curie temperature of 16 K, showing its usefulness for applications. The presence of topologically protected edge state in a MoPc nanoribbon further confirms it as a new two-dimensional topological insulator. This much widens the search of QAHE materials for the design of quantum devices.

cond-mat.mtrl-sci↗

arXiv2020

Probabilistic 3D Multi-Object Tracking for Autonomous Driving

3D multi-object tracking is a key module in autonomous driving applications that provides a reliable dynamic representation of the world to the planning module. In this paper, we present our on-line tracking method, which made the first place in the NuScenes Tracking Challenge, held at the AI Driving Olympics Workshop at NeurIPS 2019. Our method estimates the object states by adopting a Kalman Filter. We initialize the state covariance as well as the process and observation noise covariance with statistics from the training set. We also use the stochastic information from the Kalman Filter in the data association step by measuring the Mahalanobis distance between the predicted object states and current object detections. Our experimental results on the NuScenes validation and test set show that our method outperforms the AB3DMOT baseline method by a large margin in the Average Multi-Object Tracking Accuracy (AMOTA) metric.

cs.CV↗

arXiv2020

QoE-driven Coupled Uplink and Downlink Rate Adaptation for 360-degree Video Live Streaming

360-degree video provides an immersive 360-degree viewing experience and has been widely used in many areas. The 360-degree video live streaming systems involve capturing, compression, uplink (camera to video server) and downlink (video server to user) transmissions. However, few studies have jointly investigated such complex systems, especially the rate adaptation for the coupled uplink and downlink in the 360-degree video streaming under limited bandwidth constraints. In this letter, we propose a quality of experience (QoE)-driven 360-degree video live streaming system, in which a video server performs rate adaptation based on the uplink and downlink bandwidths and information concerning each user's real-time field-of-view (FOV). We formulate it as a nonlinear integer programming problem and propose an algorithm, which combines the Karush-Kuhn-Tucker (KKT) condition and branch and bound method, to solve it. The numerical results show that the proposed optimization model can improve users' QoE significantly in comparison with other baseline schemes.

cs.MM↗

arXiv2020

Speaker-aware speech-transformer

Recently, end-to-end (E2E) models become a competitive alternative to the conventional hybrid automatic speech recognition (ASR) systems. However, they still suffer from speaker mismatch in training and testing condition. In this paper, we use Speech-Transformer (ST) as the study platform to investigate speaker aware training of E2E models. We propose a model called Speaker-Aware Speech-Transformer (SAST), which is a standard ST equipped with a speaker attention module (SAM). The SAM has a static speaker knowledge block (SKB) that is made of i-vectors. At each time step, the encoder output attends to the i-vectors in the block, and generates a weighted combined speaker embedding vector, which helps the model to normalize the speaker variations. The SAST model trained in this way becomes independent of specific training speakers and thus generalizes better to unseen testing speakers. We investigate different factors of SAM. Experimental results on the AISHELL-1 task show that SAST achieves a relative 6.5% CER reduction (CERR) over the speaker-independent (SI) baseline. Moreover, we demonstrate that SAST still works quite well even if the i-vectors in SKB all come from a different data source other than the acoustic training set.

cs.CL↗

arXiv2019

Mission Oriented Miniature Fixed-wing UAV Swarms: A Multi-layered and Distributed Architecture

UAV swarms have triggered wide concern due to their potential application values in recent years. While there are studies proposed in terms of the architecture design for UAV swarms, two main challenges still exist: (1) Scalability, supporting a large scale of vehicles; (2) Versatility, integrating diversified missions. To this end, a multi-layered and distributed architecture for mission oriented miniature fixed-wing UAV swarms is presented in this paper. The proposed architecture is built on the concept of modularity. It divides the overall system to five layers: low-level control, high-level control, coordination, communication and human interaction layers, and many modules that can be viewed as black boxes with interfaces of inputs and outputs. In this way, not only the complexity of developing a large system can be reduced, but also the versatility of supporting diversified missions can be ensured. Furthermore, the proposed architecture is fully distributed that each UAV performs the decision-making procedure autonomously so as to achieve better scalability. Moreover, different kinds of aerial platforms can be feasibly extended by using the control allocation matrices and the integrated hardware box. A prototype swarm system based on the proposed architecture is built and the proposed architecture is evaluated through field experiments with a scale of 21 fixed-wing UAVs. Particularly, to the best of our knowledge, this paper is the first work which successfully demonstrates formation flight, target recognition and tracking missions within an integrated architecture for fixed-wing UAV swarms through field experiments.

cs.MA↗

arXiv2019

Noise robustness of synchronization of two nanomechanical resonators coupled to the same cavity field

We study synchronization of a room temperature optomechanical system formed by two resonators coupled via radiation pressure to the same driven optical cavity mode. By using stochastic Langevin equations and effective slowly-varying amplitude equations, we explore the long-time dynamics of the system. We see that thermal noise can induce significant non-Gaussian dynamical properties, including the coexistence of multi-stable synchronized limit cycles and phase diffusion. Synchronization in this optomechanical system is very robust with respect to thermal noise: in fact, even though each oscillator phase progressively diffuses over the whole limit cycle, their phase difference is locked, and such a phase correlation remains strong in the presence of thermal noise.

physics.optics↗

arXiv2019

Accurate Prediction of Chemical Shifts for Aqueous Protein Structure for "Real World" Cases using Machine Learning

Accurate prediction of NMR chemical shifts can in principle help refine aqueous solution structure of proteins to the quality of X-ray structures. We report a new machine learning algorithm for protein chemical shift prediction that outperforms existing chemical shift calculators on realistic NMR solution data. Our UCBShift predictor implements two modules: a transfer prediction module that employs both sequence and structural alignment to select reference candidates for experimental chemical shift replication, and a redesigned machine learning module based on random forest regression which utilizes more, and more carefully curated, feature extracted data. When combined together, this new predictor achieves state of the art accuracy for predicting chemical shifts on a "real-world" dataset, with root-mean-square errors of 0.31 ppm for amide hydrogens, 0.19 ppm for Halpha, 0.87 ppm for C, 0.81 ppm for Calpha, 1.01 ppm for Cbeta, and 1.83 ppm for N, exceeding current prediction accuracy of popular chemical shift predictors such as SPARTA+ and SHIFTX2.

physics.chem-ph↗

arXiv2019

Boosting Real-Time Driving Scene Parsing with Shared Semantics

Real-time scene parsing is a fundamental feature for autonomous driving vehicles with multiple cameras. In this letter we demonstrate that sharing semantics between cameras with different perspectives and overlapped views can boost the parsing performance when compared with traditional methods, which individually process the frames from each camera. Our framework is based on a deep neural network for semantic segmentation but with two kinds of additional modules for sharing and fusing semantics. On the one hand, a semantics sharing module is designed to establish the pixel-wise mapping between the input images. Features as well as semantics are shared by the map to reduce duplicated workload which leads to more efficient computation. On the other hand, feature fusion modules are designed to combine different modal of semantic features, which leverage the information from both inputs for better accuracy. To evaluate the effectiveness of the proposed framework, we have applied our network to a dual-camera vision system for driving scene parsing. Experimental results show that our network outperforms the baseline method on the parsing accuracy with comparable computations.

cs.CV↗

arXiv2019

Two Stream Networks for Self-Supervised Ego-Motion Estimation

Learning depth and camera ego-motion from raw unlabeled RGB video streams is seeing exciting progress through self-supervision from strong geometric cues. To leverage not only appearance but also scene geometry, we propose a novel self-supervised two-stream network using RGB and inferred depth information for accurate visual odometry. In addition, we introduce a sparsity-inducing data augmentation policy for ego-motion learning that effectively regularizes the pose network to enable stronger generalization performance. As a result, we show that our proposed two-stream pose network achieves state-of-the-art results among learning-based methods on the KITTI odometry benchmark, and is especially suited for self-supervision at scale. Our experiments on a large-scale urban driving dataset of 1 million frames indicate that the performance of our proposed architecture does indeed scale progressively with more data.

cs.CV↗

arXiv2019

Robust Semi-Supervised Monocular Depth Estimation with Reprojected Distances

Dense depth estimation from a single image is a key problem in computer vision, with exciting applications in a multitude of robotic tasks. Initially viewed as a direct regression problem, requiring annotated labels as supervision at training time, in the past few years a substantial amount of work has been done in self-supervised depth training based on strong geometric cues, both from stereo cameras and more recently from monocular video sequences. In this paper we investigate how these two approaches (supervised & self-supervised) can be effectively combined, so that a depth model can learn to encode true scale from sparse supervision while achieving high fidelity local accuracy by leveraging geometric cues. To this end, we propose a novel supervised loss term that complements the widely used photometric loss, and show how it can be used to train robust semi-supervised monocular depth estimation models. Furthermore, we evaluate how much supervision is actually necessary to train accurate scale-aware monocular depth models, showing that with our proposed framework, very sparse LiDAR information, with as few as 4 beams (less than 100 valid depth values per image), is enough to achieve results competitive with the current state-of-the-art.

cs.CV↗

arXiv2019

AIM 2019 Challenge on Real-World Image Super-Resolution: Methods and Results

This paper reviews the AIM 2019 challenge on real world super-resolution. It focuses on the participating methods and final results. The challenge addresses the real world setting, where paired true high and low-resolution images are unavailable. For training, only one set of source input images is therefore provided in the challenge. In Track 1: Source Domain the aim is to super-resolve such images while preserving the low level image characteristics of the source input domain. In Track 2: Target Domain a set of high-quality images is also provided for training, that defines the output domain and desired quality of the super-resolved images. To allow for quantitative evaluation, the source input images in both tracks are constructed using artificial, but realistic, image degradations. The challenge is the first of its kind, aiming to advance the state-of-the-art and provide a standard benchmark for this newly emerging task. In total 7 teams competed in the final testing phase, demonstrating new and innovative solutions to the problem.

cs.CV↗

arXiv2019

HighEr-Resolution Network for Image Demosaicing and Enhancing

Neural-networks based image restoration methods tend to use low-resolution image patches for training. Although higher-resolution image patches can provide more global information, state-of-the-art methods cannot utilize them due to their huge GPU memory usage, as well as the instable training process. However, plenty of studies have shown that global information is crucial for image restoration tasks like image demosaicing and enhancing. In this work, we propose a HighEr-Resolution Network (HERN) to fully learning global information in high-resolution image patches. To achieve this, the HERN employs two parallel paths to learn image features in two different resolutions, respectively. By combining global-aware features and multi-scale features, our HERN is able to learn global information with feasible GPU memory usage. Besides, we introduce a progressive training method to solve the instability issue and accelerate model convergence. On the task of image demosaicing and enhancing, our HERN achieves state-of-the-art performance on the AIM2019 RAW to RGB mapping challenge. The source code of our implementation is available at https://github.com/MKFMIKU/RAW2RGBNet.

eess.IV↗

arXiv2019

DeLottery: A Novel Decentralized Lottery System Based on Blockchain Technology

In this paper, we design DeLottery, a decentralized lottery system based on block chain technology and smart contracts. Lottery is a classical form of entertainment and charity for centuries. Facing the bottleneck of the combination between lottery and information technology, we use smart contracts and blockchain in decentralized, intelligent, and secure systems for lottery industries. Moreover, we are inspired by the algorithm of RANDAO, an outstanding way of random number generation in blockchain scenario. The components and the functions of the novel system are described in details. We implement DeLottery in a blockchain network and show functioning procedure and security of the proposed lottery system.

cs.DC↗

arXiv2019

MoB2:a new multifunctional transition metal diboride monolayer

Several layered transition metal borides can now be realized by a simple and general fabrication method [Fokwa et al.,Adv. Mater. 2018, 30, 1704181],inspiring our interest to transition metal borides monolayer. Here, we predict a new two-dimensional (2D) transition metal diboride MoB2 monolayer (ML) and study its intrinsic mechanical, thermal, electronic, and transport properties. The MoB2 ML has isotropic mechanic properties along the zigzag and armchair directions with a large Young's stiffness, and has an ultralow room-temperature thermal conductivity. The Mo atoms dominate the metallic nature of MoB2 ML. It shows an obvious electrical anisotropy and a current-limiting behavior. Our findings suggest that MoB2 ML is a promising multifunctional material used in ultrathin high-strength mechanical materials, heat insulating materials, electrical-anisotropy-based materials, and current limiters. It is helpful for the experimentalists to further prepare and utilize the transition metal diboride 2D materials.

cond-mat.mtrl-sci↗

arXiv2019

Reconstructing Perceived Images from Brain Activity by Visually-guided Cognitive Representation and Adversarial Learning

Reconstructing visual stimulus (image) only from human brain activity measured with functional Magnetic Resonance Imaging (fMRI) is a significant and meaningful task in Human-AI collaboration. However, the inconsistent distribution and representation between fMRI signals and visual images cause the heterogeneity gap. Moreover, the fMRI data is often extremely high-dimensional and contains a lot of visually-irrelevant information. Existing methods generally suffer from these issues so that a satisfactory reconstruction is still challenging. In this paper, we show that it is possible to overcome these challenges by learning visually-guided cognitive latent representations from the fMRI signals, and inversely decoding them to the image stimuli. The resulting framework is called Dual-Variational Autoencoder/ Generative Adversarial Network (D-VAE/GAN), which combines the advantages of adversarial representation learning with knowledge distillation. In addition, we introduce a novel three-stage learning approach which enables the (cognitive) encoder to gradually distill useful knowledge from the paired (visual) encoder during the learning process. Extensive experimental results on both artificial and natural images have demonstrated that our method could achieve surprisingly good results and outperform all other alternatives.

cs.CV↗