arXiv subjects

Zhi Liu

Publications and source records attributed to Zhi Liu.

At least 163 records · Page 9Linked to original sources

arXiv2021

Referring Segmentation in Images and Videos with Cross-Modal Self-Attention Network

We consider the problem of referring segmentation in images and videos with natural language. Given an input image (or video) and a referring expression, the goal is to segment the entity referred by the expression in the image or video. In this paper, we propose a cross-modal self-attention (CMSA) module to utilize fine details of individual words and the input image or video, which effectively captures the long-range dependencies between linguistic and visual features. Our model can adaptively focus on informative words in the referring expression and important regions in the visual input. We further propose a gated multi-level fusion (GMLF) module to selectively integrate self-attentive cross-modal features corresponding to different levels of visual features. This module controls the feature fusion of information flow of features at different levels with high-level and low-level semantic information related to different attentive words. Besides, we introduce cross-frame self-attention (CFSA) module to effectively integrate temporal information in consecutive frames which extends our method in the case of referring segmentation in videos. Experiments on benchmark datasets of four referring image datasets and two actor and action video segmentation datasets consistently demonstrate that our proposed approach outperforms existing state-of-the-art methods.

cs.CV↗

arXiv2021

Personal Fixations-Based Object Segmentation with Object Localization and Boundary Preservation

As a natural way for human-computer interaction, fixation provides a promising solution for interactive image segmentation. In this paper, we focus on Personal Fixations-based Object Segmentation (PFOS) to address issues in previous studies, such as the lack of appropriate dataset and the ambiguity in fixations-based interaction. In particular, we first construct a new PFOS dataset by carefully collecting pixel-level binary annotation data over an existing fixation prediction dataset, such dataset is expected to greatly facilitate the study along the line. Then, considering characteristics of personal fixations, we propose a novel network based on Object Localization and Boundary Preservation (OLBP) to segment the gazed objects. Specifically, the OLBP network utilizes an Object Localization Module (OLM) to analyze personal fixations and locates the gazed objects based on the interpretation. Then, a Boundary Preservation Module (BPM) is designed to introduce additional boundary information to guard the completeness of the gazed objects. Moreover, OLBP is organized in the mixed bottom-up and top-down manner with multiple types of deep supervision. Extensive experiments on the constructed PFOS dataset show the superiority of the proposed OLBP network over 17 state-of-the-art methods, and demonstrate the effectiveness of the proposed OLM and BPM components. The constructed PFOS dataset and the proposed OLBP network are available at https://github.com/MathLee/OLBPNet4PFOS.

cs.CV↗

arXiv2021

DRLE: Decentralized Reinforcement Learning at the Edge for Traffic Light Control in the IoV

The Internet of Vehicles (IoV) enables real-time data exchange among vehicles and roadside units and thus provides a promising solution to alleviate traffic jams in the urban area. Meanwhile, better traffic management via efficient traffic light control can benefit the IoV as well by enabling a better communication environment and decreasing the network load. As such, IoV and efficient traffic light control can formulate a virtuous cycle. Edge computing, an emerging technology to provide low-latency computation capabilities at the edge of the network, can further improve the performance of this cycle. However, while the collected information is valuable, an efficient solution for better utilization and faster feedback has yet to be developed for edge-empowered IoV. To this end, we propose a Decentralized Reinforcement Learning at the Edge for traffic light control in the IoV (DRLE). DRLE exploits the ubiquity of the IoV to accelerate the collection of traffic data and its interpretation towards alleviating congestion and providing better traffic light control. DRLE operates within the coverage of the edge servers and uses aggregated data from neighboring edge servers to provide city-scale traffic light control. DRLE decomposes the highly complex problem of large area control. into a decentralized multi-agent problem. We prove its global optima with concrete mathematical reasoning. The proposed decentralized reinforcement learning algorithm running at each edge node adapts the traffic lights in real time. We conduct extensive evaluations and demonstrate the superiority of this approach over several state-of-the-art algorithms.

cs.MA↗

arXiv2020

WiFE: WiFi and Vision based Intelligent Facial-Gesture Emotion Recognition

Emotion is an essential part of Artificial Intelligence (AI) and human mental health. Current emotion recognition research mainly focuses on single modality (e.g., facial expression), while human emotion expressions are multi-modal in nature. In this paper, we propose a hybrid emotion recognition system leveraging two emotion-rich and tightly-coupled modalities, i.e., facial expression and body gesture. However, unbiased and fine-grained facial expression and gesture recognition remain a major problem. To this end, unlike our rivals relying on contact or even invasive sensors, we explore the commodity WiFi signal for device-free and contactless gesture recognition, while adopting a vision-based facial expression. However, there exist two design challenges, i.e., how to improve the sensitivity of WiFi signals and how to process the large-volume, heterogeneous, and non-synchronous data contributed by the two-modalities. For the former, we propose a signal sensitivity enhancement method based on the Rician K factor theory; for the latter, we combine CNN and RNN to mine the high-level features of bi-modal data, and perform a score-level fusion for fine-grained recognition. To evaluate the proposed method, we build a first-of-its-kind Vision-CSI Emotion Database (VCED) and conduct extensive experiments. Empirical results show the superiority of the bi-modality by achieving 83.24\% recognition accuracy for seven emotions, as compared with 66.48% and 66.67% recognition accuracy by gesture-only based solution and facial-only based solution, respectively. The VCED database download link is https://github.com/purpleleaves007/WIFE-Dataset.

cs.HC↗

arXiv2020

Observables in quantum field theory on a collapsing black hole background

We study the quantum radiation from a curved background using scalar observables constructed from the (1+1) dimensional renormalized stress-energy tensor (RSET). We compute energy density $\cal U$, flux $\cal F$ and pressure $\cal P$ for an arbitrary collapse scenario and an arbitrary observer. The results show a clear structure: $\cal F - \cal U$ represents the collapse-independent ingoing modes, $\cal F + \cal U$ represents the collapse-dependent outgoing modes and $\cal P - \cal U$ represents the trace anomaly. We also compute the observables constructed from the perception renormalized stress-energy tensor (PeRSET). We find they have the same collapse-dependent contribution comparing to their RSET-related counterparts. For free-falling observers, in particular, the PeRSET-related observables are the same as the corresponding RSET-related ones, except the energy densities differ by the trace anomaly.

gr-qc↗

arXiv2020

Large Deviation principles of Realized Laplace Transform of Volatility

Under scenario of high frequency data, consistent estimator of realized Laplace transform of volatility is proposed by \citet{TT2012a} and related central limit theorem has been well established. In this paper, we investigate the asymptotic tail behaviour of the empirical realized Laplace transform of volatility (ERLTV). We establish both large deviation principle and moderate deviation principle for the ERLTV. The good rate function for the large deviation principle is well defined in the whole real space, which indicates a limit for the normalized logarithmic tail probability of the ERLTV. Moreover, we also derive the function-level large and moderate deviation principles for ERLTV.

math.ST↗

arXiv2020

Heteroscedasticity test of high-frequency data with jumps and microstructure noise

In this paper, we are interested in testing if the volatility process is constant or not during a given time span by using high-frequency data with the presence of jumps and microstructure noise. Based on estimators of integrated volatility and spot volatility, we propose a nonparametric way to depict the discrepancy between local variation and global variation. We show that our proposed test estimator converges to a standard normal distribution if the volatility is constant, otherwise it diverges to infinity. Simulation studies verify the theoretical results and show a good finite sample performance of the test procedure. We also apply our test procedure to do the heteroscedasticity test for some real high-frequency financial data. We observe that in almost half of the days tested, the assumption of constant volatility within a day is violated. And this is due to that the stock prices during opening and closing periods are highly volatile and account for a relative large proportion of intraday variation.

econ.EM↗

arXiv2020

Semi-Global Weighted Least Squares in Image Filtering

Solving the global method of Weighted Least Squares (WLS) model in image filtering is both time- and memory-consuming. In this paper, we present an alternative approximation in a time- and memory- efficient manner which is denoted as Semi-Global Weighed Least Squares (SG-WLS). Instead of solving a large linear system, we propose to iteratively solve a sequence of subsystems which are one-dimensional WLS models. Although each subsystem is one-dimensional, it can take two-dimensional neighborhood information into account due to the proposed special neighborhood construction. We show such a desirable property makes our SG-WLS achieve close performance to the original two-dimensional WLS model but with much less time and memory cost. While previous related methods mainly focus on the 4-connected/8-connected neighborhood system, our SG-WLS can handle a more general and larger neighborhood system thanks to the proposed fast solution. We show such a generalization can achieve better performance than the 4-connected/8-connected neighborhood system in some applications. Our SG-WLS is $\sim20$ times faster than the WLS model. For an image of $M\times N$, the memory cost of SG-WLS is at most at the magnitude of $max\{\frac{1}{M}, \frac{1}{N}\}$ of that of the WLS model. We show the effectiveness and efficiency of our SG-WLS in a range of applications. The code is publicly available at: https://github.com/wliusjtu/Semi-Global-Weighted-Least-Squares-in-Image-Filtering.

cs.GR↗

arXiv2020

Optimal Wireless Streaming of Multi-Quality 360 VR Video by Exploiting Natural, Relative Smoothness-enabled and Transcoding-enabled Multicast Opportunities

In this paper, we would like to investigate optimal wireless streaming of a multi-quality tiled 360 virtual reality (VR) video from a server to multiple users. To this end, we propose to maximally exploit potential multicast opportunities by effectively utilizing characteristics of multi-quality tiled 360 VR videos and computation resources at the users' side. In particular, we consider two requirements for quality variation in one field-of-view (FoV), i.e., the absolute smoothness requirement and the relative smoothness requirement, and two video playback modes, i.e., the direct-playback mode (without user transcoding) and transcode-playback mode (with user transcoding). Besides natural multicast opportunities, we introduce two new types of multicast opportunities, namely, relative smoothness-enabled multicast opportunities, which allow flexible tradeoff between viewing quality and communications resource consumption, and transcoding-enabled multicast opportunities, which allow flexible tradeoff between computation and communications resource consumptions. Then, we establish a novel mathematical model that reflects the impacts of natural, relative smoothness-enabled and transcoding-enabled multicast opportunities on the average transmission energy and transcoding energy. Based on this model, we optimize the transmission resource allocation, playback quality level selection and transmission quality level selection to minimize the energy consumption in the four cases with different requirements for quality variation and video playback modes. By comparing the optimal values in the four cases, we prove that the energy consumption reduces when more multicast opportunities can be utilized. Finally, numerical results show substantial gains of the proposed solutions over existing schemes, and demonstrate the importance of effective exploitation of the three types of multicast opportunities.

cs.IT↗

arXiv2020

Optimal Streaming of 360 VR Videos with Perfect, Imperfect and Unknown FoV Viewing Probabilities

In this paper, we investigate wireless streaming of multi-quality tiled 360 virtual reality (VR) videos from a multi-antenna server to multiple single-antenna users in a multi-carrier system. To capture the impact of field-of-view (FoV) prediction, we consider three cases of FoV viewing probability distributions, i.e., perfect, imperfect and unknown FoV viewing probability distributions, and use the average total utility, worst average total utility and worst total utility as the respective performance metrics. We adopt rate splitting with successive decoding for efficient transmission of multiple sets of tiles of different 360 VR videos to their requesting users. In each case, we optimize the encoding rates of the tiles, minimum encoding rates of the FoVs, rates of the common and private messages and transmission beamforming vectors to maximize the total utility. The problems in the three cases are all challenging nonconvex optimization problems. We successfully transform the problem in each case into a difference of convex (DC) programming problem with a differentiable objective function, and obtain a suboptimal solution using concave-convex procedure (CCCP). Finally, numerical results demonstrate the proposed solutions achieve notable gains over existing schemes in all three cases. To the best of our knowledge, this is the first work revealing the impact of FoV prediction and its accuracy on the performance of streaming of multi-quality tiled 360 VR videos.

cs.IT↗

arXiv2020

Edgeworth corrections for spot volatility estimator

We develop Edgeworth expansion theory for spot volatility estimator under general assumptions on the log-price process that allow for drift and leverage effect. The result is based on further estimation of skewness and kurtosis, when compared with existing second order asymptotic normality result. Thus our theory can provide with a refinement result for the finite sample distribution of spot volatility. We also construct feasible confidence intervals (one-sided and two-sided) for spot volatility by using Edgeworth expansion. The Monte Carlo simulation study we conduct shows that the intervals based on Edgeworth expansion perform better than the conventional intervals based on normal approximation, which justifies the correctness of our theoretical conclusion.

math.ST↗

arXiv2020

Computation Offloading in Beyond 5G Networks: A Distributed Learning Framework and Applications

Facing the trend of merging wireless communications and multi-access edge computing (MEC), this article studies computation offloading in the beyond fifth-generation networks. To address the technical challenges originating from the uncertainties and the sharing of limited resource in an MEC system, we formulate the computation offloading problem as a multi-agent Markov decision process, for which a distributed learning framework is proposed. We present a case study on resource orchestration in computation offloading to showcase the potentials of an online distributed reinforcement learning algorithm developed under the proposed framework. Experimental results demonstrate that our learning algorithm outperforms the benchmark resource orchestration algorithms. Furthermore, we outline the research directions worth in-depth investigation to minimize the time cost, which is one of the main practical issues that prevent the implementation of the proposed distributed learning framework.

cs.DC↗

arXiv2020

Information Freshness-Aware Task Offloading in Air-Ground Integrated Edge Computing Systems

This paper studies the problem of information freshness-aware task offloading in an air-ground integrated multi-access edge computing system, which is deployed by an infrastructure provider (InP). A third-party real-time application service provider provides computing services to the subscribed mobile users (MUs) with the limited communication and computation resources from the InP based on a long-term business agreement. Due to the dynamic characteristics, the interactions among the MUs are modelled by a non-cooperative stochastic game, in which the control policies are coupled and each MU aims to selfishly maximize its own expected long-term payoff. To address the Nash equilibrium solutions, we propose that each MU behaves in accordance with the local system states and conjectures, based on which the stochastic game is transformed into a single-agent Markov decision process. Moreover, we derive a novel online deep reinforcement learning (RL) scheme that adopts two separate double deep Q-networks for each MU to approximate the Q-factor and the post-decision Q-factor. Using the proposed deep RL scheme, each MU in the system is able to make decisions without a priori statistical knowledge of dynamics. Numerical experiments examine the potentials of the proposed scheme in balancing the age of information and the energy consumption.

eess.SP↗

arXiv2020

Cross-Modal Weighting Network for RGB-D Salient Object Detection

Depth maps contain geometric clues for assisting Salient Object Detection (SOD). In this paper, we propose a novel Cross-Modal Weighting (CMW) strategy to encourage comprehensive interactions between RGB and depth channels for RGB-D SOD. Specifically, three RGB-depth interaction modules, named CMW-L, CMW-M and CMW-H, are developed to deal with respectively low-, middle- and high-level cross-modal information fusion. These modules use Depth-to-RGB Weighing (DW) and RGB-to-RGB Weighting (RW) to allow rich cross-modal and cross-scale interactions among feature layers generated by different network blocks. To effectively train the proposed Cross-Modal Weighting Network (CMWNet), we design a composite loss function that summarizes the errors between intermediate predictions and ground truth over different scales. With all these novel components working together, CMWNet effectively fuses information from RGB and depth channels, and meanwhile explores object localization and details across scales. Thorough evaluations demonstrate CMWNet consistently outperforms 15 state-of-the-art RGB-D SOD methods on seven popular benchmarks.

cs.CV↗

arXiv2020

Robust Q-tunable topological induced transparency in metasurface

In this study, we demonstrate first, to the best of our knowledge, robust and dynamically polarization-controlled tunable-high-Q PIT in designed nanostructures metasurface, whose sharp resonance is guaranteed by design and protected against large geometrical imperfections. By employing the explicit analysis of near-field characteristic in the reciprocal-space based on the momentum matching, and the far-field radiation features with point-scattering approach in real-space sparked from Huygens's principles, the physics of interference resonance in the spectra for plane-wave optical transmission and reflection of the metasurface is theoretically and thoroughly investigated. The experimental results verify the theory prediction. The distinctive polarization-selective and Q-tunable PIT shows robust features to performance degradations in traditional PIT system caused by inadvertent fabrication flaws or geometry asymmetry-variations, which paves way for the development of reconfigurable and flexible metasurface and, additionally, opens new avenues in robust and multifunctional nanophotonics device design and applications.

physics.optics↗

arXiv2020

Electronic structure of a Si-containing topological Dirac semimetal CaAl2Si2

There has been an upsurge in the discovery of topological quantum materials, where various topological insulators and semimetals have been theoretically predicted and experimentally observed. However, only very few of them contains silicon, the most widely used element in electronic industry. Recently, ternary compound CaAl2Si2 has been predicted to be a topological Dirac semimetal, hosting Lorentz-symmetry-violating quasiparticles with a strongly tilted conical band dispersion. In this work, by using high-resolution angle-resolved photoemission spectroscopy (ARPES), we investigated the comprehensive electronic structure of CaAl2Si2. A pair of topological Dirac crossings is observed along the kz direction, in good agreement with the ab initio calculations, confirming the topological Dirac semimetal nature of the compound. Our study expands the topological material family on Si-containing compounds, which have great application potential in realizing low-cost, nontoxic electronic device with topological quantum states.

cond-mat.mtrl-sci↗

arXiv2020

Evolution Features and Behavior Characters of Friendship Networks on Campus Life

Analyzing and mining students' behaviors and interactions from big data is an essential part of education data mining. Based on the data of campus smart cards, which include not only static demographic information but also dynamic behavioral data from more than 30000 anonymous students, in this paper, the evolution features of friendship and the relations between behavior characters and student interactions are investigated. On the one hand, four different evolving friendship networks are constructed by means of the friend ties proposed in this paper, which are extracted from monthly consumption records. In addition, the features of the giant connected components (GCCs) of friendship networks are analyzed via social network analysis (SNA) and percolation theory. On the other hand, two high-level behavior characters, orderliness and diligence, are adopted to analyze their associations with student interactions. Our experiment/empirical results indicate that the sizes of friendship networks have declined with time growth and both the small-world effect and power-law degree distribution are found in friendship networks. Second, the results of the assortativity coefficient of both orderliness and diligence verify that there are strong peer effects among students. Finally, the percolation analysis of orderliness on friendship networks shows that a phase transition exists, which is enlightening in that swarm intelligence can be realized by intervening the key students near the transition point.

cs.SI↗

arXiv2020

Large-scale Real-time Personalized Similar Product Recommendations

Similar product recommendation is one of the most common scenes in e-commerce. Many recommendation algorithms such as item-to-item Collaborative Filtering are working on measuring item similarities. In this paper, we introduce our real-time personalized algorithm to model product similarity and real-time user interests. We also introduce several other baseline algorithms including an image-similarity-based method, item-to-item collaborative filtering, and item2vec, and compare them on our large-scale real-world e-commerce dataset. The algorithms which achieve good offline results are also tested on the online e-commerce website. Our personalized method achieves a 10% improvement on the add-cart number in the real-world e-commerce scenario.

cs.IR↗