arXiv subjects

Chao Yu

Publications and source records attributed to Chao Yu.

At least 163 records · Page 9Linked to original sources

arXiv2023

Free-running 4H-SiC single-photon detector with ultralow afterpulse probability at 266 nm

Ultraviolet single-photon detector (UVSPD) provides a key tool for the applications requiring ultraweak light detection in the wavelength band. Here, we report a 4H-SiC single-photon avalanche diode (SPAD) based free-running UVSPD with ultralow afterpulse probability. We design and fabricate the 4H-SiC SPAD with a beveled mesa structure, which exhibits the characteristic of ultralow dark current. We further develop a readout circuit of passive quenching and active reset with tunable hold-off time setting to considerably suppress the afterpulsing effect. The nonuniformity of photon detection efficiency (PDE) across the SPAD active area with a diameter of $\sim$ 180 $μ$m is investigated for performance optimization. The compact UVSPD is then characterized, exhibiting a typical performance of 10.3% PDE, 133 kcps dark count rate and 0.3% afterpulse probability at 266 nm. Such performance indicates that the compact UVSPD could be used for practical ultraviolet photon-counting applications

physics.ins-det↗

arXiv2023

Learning Graph-Enhanced Commander-Executor for Multi-Agent Navigation

This paper investigates the multi-agent navigation problem, which requires multiple agents to reach the target goals in a limited time. Multi-agent reinforcement learning (MARL) has shown promising results for solving this issue. However, it is inefficient for MARL to directly explore the (nearly) optimal policy in the large search space, which is exacerbated as the agent number increases (e.g., 10+ agents) or the environment is more complex (e.g., 3D simulator). Goal-conditioned hierarchical reinforcement learning (HRL) provides a promising direction to tackle this challenge by introducing a hierarchical structure to decompose the search space, where the low-level policy predicts primitive actions in the guidance of the goals derived from the high-level policy. In this paper, we propose Multi-Agent Graph-Enhanced Commander-Executor (MAGE-X), a graph-based goal-conditioned hierarchical method for multi-agent navigation tasks. MAGE-X comprises a high-level Goal Commander and a low-level Action Executor. The Goal Commander predicts the probability distribution of goals and leverages them to assign each agent the most appropriate final target. The Action Executor utilizes graph neural networks (GNN) to construct a subgraph for each agent that only contains crucial partners to improve cooperation. Additionally, the Goal Encoder in the Action Executor captures the relationship between the agent and the designated goal to encourage the agent to reach the final target. The results show that MAGE-X outperforms the state-of-the-art MARL baselines with a 100% success rate with only 3 million training steps in multi-agent particle environments (MPE) with 50 agents, and at least a 12% higher success rate and 2x higher data efficiency in a more complicated quadrotor 3D navigation task.

cs.RO↗

arXiv2023

Learning Zero-Shot Cooperation with Humans, Assuming Humans Are Biased

There is a recent trend of applying multi-agent reinforcement learning (MARL) to train an agent that can cooperate with humans in a zero-shot fashion without using any human data. The typical workflow is to first repeatedly run self-play (SP) to build a policy pool and then train the final adaptive policy against this pool. A crucial limitation of this framework is that every policy in the pool is optimized w.r.t. the environment reward function, which implicitly assumes that the testing partners of the adaptive policy will be precisely optimizing the same reward function as well. However, human objectives are often substantially biased according to their own preferences, which can differ greatly from the environment reward. We propose a more general framework, Hidden-Utility Self-Play (HSP), which explicitly models human biases as hidden reward functions in the self-play objective. By approximating the reward space as linear functions, HSP adopts an effective technique to generate an augmented policy pool with biased policies. We evaluate HSP on the Overcooked benchmark. Empirical results show that our HSP method produces higher rewards than baselines when cooperating with learned human models, manually scripted policies, and real humans. The HSP policy is also rated as the most assistive policy based on human feedback.

cs.AI↗

arXiv2023

Plan To Predict: Learning an Uncertainty-Foreseeing Model for Model-Based Reinforcement Learning

In Model-based Reinforcement Learning (MBRL), model learning is critical since an inaccurate model can bias policy learning via generating misleading samples. However, learning an accurate model can be difficult since the policy is continually updated and the induced distribution over visited states used for model learning shifts accordingly. Prior methods alleviate this issue by quantifying the uncertainty of model-generated samples. However, these methods only quantify the uncertainty passively after the samples were generated, rather than foreseeing the uncertainty before model trajectories fall into those highly uncertain regions. The resulting low-quality samples can induce unstable learning targets and hinder the optimization of the policy. Moreover, while being learned to minimize one-step prediction errors, the model is generally used to predict for multiple steps, leading to a mismatch between the objectives of model learning and model usage. To this end, we propose \emph{Plan To Predict} (P2P), an MBRL framework that treats the model rollout process as a sequential decision making problem by reversely considering the model as a decision maker and the current policy as the dynamics. In this way, the model can quickly adapt to the current policy and foresee the multi-step future uncertainty when generating trajectories. Theoretically, we show that the performance of P2P can be guaranteed by approximately optimizing a lower bound of the true environment return. Empirical results demonstrate that P2P achieves state-of-the-art performance on several challenging benchmark tasks.

cs.LG↗

arXiv2022

Learning Efficient Multi-Agent Cooperative Visual Exploration

We tackle the problem of cooperative visual exploration where multiple agents need to jointly explore unseen regions as fast as possible based on visual signals. Classical planning-based methods often suffer from expensive computation overhead at each step and a limited expressiveness of complex cooperation strategy. By contrast, reinforcement learning (RL) has recently become a popular paradigm for tackling this challenge due to its modeling capability of arbitrarily complex strategies and minimal inference overhead. In this paper, we extend the state-of-the-art single-agent visual navigation method, Active Neural SLAM (ANS), to the multi-agent setting by introducing a novel RL-based planning module, Multi-agent Spatial Planner (MSP).MSP leverages a transformer-based architecture, Spatial-TeamFormer, which effectively captures spatial relations and intra-agent interactions via hierarchical spatial self-attentions. In addition, we also implement a few multi-agent enhancements to process local information from each agent for an aligned spatial representation and more precise planning. Finally, we perform policy distillation to extract a meta policy to significantly improve the generalization capability of final policy. We call this overall solution, Multi-Agent Active Neural SLAM (MAANS). MAANS substantially outperforms classical planning-based baselines for the first time in a photo-realistic 3D simulator, Habitat. Code and videos can be found at https://sites.google.com/view/maans.

cs.CV↗

arXiv2022

The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games

Proximal Policy Optimization (PPO) is a ubiquitous on-policy reinforcement learning algorithm but is significantly less utilized than off-policy learning algorithms in multi-agent settings. This is often due to the belief that PPO is significantly less sample efficient than off-policy methods in multi-agent systems. In this work, we carefully study the performance of PPO in cooperative multi-agent settings. We show that PPO-based multi-agent algorithms achieve surprisingly strong performance in four popular multi-agent testbeds: the particle-world environments, the StarCraft multi-agent challenge, Google Research Football, and the Hanabi challenge, with minimal hyperparameter tuning and without any domain-specific algorithmic modifications or architectures. Importantly, compared to competitive off-policy methods, PPO often achieves competitive or superior results in both final returns and sample efficiency. Finally, through ablation studies, we analyze implementation and hyperparameter factors that are critical to PPO's empirical performance, and give concrete practical suggestions regarding these factors. Our results show that when using these practices, simple PPO-based methods can be a strong baseline in cooperative multi-agent reinforcement learning. Source code is released at \url{https://github.com/marlbenchmark/on-policy}.

cs.LG↗

arXiv2022

DeltaFS: Pursuing Zero Update Overhead via Metadata-Enabled Delta Compression for Log-structured File System on Mobile Devices

Data compression has been widely adopted to release mobile devices from intensive write pressure. Delta compression is particularly promising for its high compression efficacy over conventional compression methods. However, this method suffers from non-trivial system overheads incurred by delta maintenance and read penalty, which prevents its applicability on mobile devices. To this end, this paper proposes DeltaFS, a metadata-enabled Delta compression on log-structured File System for mobile devices, to achieve utmost compressing efficiency and zero hardware costs. DeltaFS smartly exploits the out-of-place updating ability of Log-structured File System (LFS) to alleviate the problems of write amplification, which is the key bottleneck for delta compression implementation. Specifically, DeltaFS utilizes the inline area in file inodes for delta maintenance with zero hardware cost, and integrates an inline area manage strategy to improve the utilization of constrained inline area. Moreover, a complimentary delta maintenance strategy is incorporated, which selectively maintains delta chunks in the main data area to break through the limitation of constrained inline area. Experimental results show that DeltaFS substantially reduces write traffics by up to 64.8\%, and improves the I/O performance by up to 37.3\%.

cs.DC↗

arXiv2022

Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learning

Many advances in cooperative multi-agent reinforcement learning (MARL) are based on two common design principles: value decomposition and parameter sharing. A typical MARL algorithm of this fashion decomposes a centralized Q-function into local Q-networks with parameters shared across agents. Such an algorithmic paradigm enables centralized training and decentralized execution (CTDE) and leads to efficient learning in practice. Despite all the advantages, we revisit these two principles and show that in certain scenarios, e.g., environments with a highly multi-modal reward landscape, value decomposition, and parameter sharing can be problematic and lead to undesired outcomes. In contrast, policy gradient (PG) methods with individual policies provably converge to an optimal solution in these cases, which partially supports some recent empirical observations that PG can be effective in many MARL testbeds. Inspired by our theoretical analysis, we present practical suggestions on implementing multi-agent PG algorithms for either high rewards or diverse emergent behaviors and empirically validate our findings on a variety of domains, ranging from the simplified matrix and grid-world games to complex benchmarks such as StarCraft Multi-Agent Challenge and Google Research Football. We hope our insights could benefit the community towards developing more general and more powerful MARL algorithms. Check our project website at https://sites.google.com/view/revisiting-marl.

cs.AI↗

arXiv2022

Robust optimal investment and risk control for an insurer with general insider information

In this paper, we study the robust optimal investment and risk control problem for an insurer who owns the insider information about the financial market and the insurance market under model uncertainty. Both financial risky asset process and insurance risk process are assumed to be very general jump diffusion processes. The insider information is of the most general form rather than the initial enlargement type. We use the theory of forward integrals to give the first half characterization of the robust optimal strategy and transform the anticipating stochastic differential game problem into the nonanticipative stochastic differential game problem. Then we adopt the stochastic maximum principle to obtain the total characterization of the robust strategy. We discuss the two typical situations when the insurer is `small' and `large' by Malliavin calculus. For the `small' insurer, we obtain the closed-form solution in the continuous case and the half closed-form solution in the case with jumps. For the `large' insurer, we reduce the problem to the quadratic backward stochastic differential equation (BSDE) and obtain the closed-form solution in the continuous case without model uncertainty. We discuss some impacts of the model uncertainty, insider information and the `large' insurer on the optimal strategy.

math.NA↗

arXiv2022

Malliavin calculus and its application to robust optimal portfolio for an insider

Insider information and model uncertainty are two unavoidable problems for the portfolio selection theory in reality. This paper studies the robust optimal portfolio strategy for an investor who owns general insider information under model uncertainty. On the aspect of the mathematical theory, we improve some properties of the forward integral and use Malliavin calculus to derive the anticipating Itô formula . Then we use forward integrals to formulate the insider-trading problem with model uncertainty. We give the half characterization of the robust optimal portfolio and obtain the semimartingale decomposition of the driving noise $W$ with respect to the insider information filtration, which turns the problem turns to the nonanticipative stochastic differential game problem. We give the total characterization by the stochastic maximum principle. When considering two typical situations where the insider is `small' and `large', we give the corresponding BSDEs to characterize the robust optimal portfolio strategy, and derive the closed form of the portfolio and the value function in the case of the small insider by the Donsker $δ$ functional. We present the simulation result and give the economic analysis of optimal strategies under different situations.

math.NA↗

arXiv2022

ESCM$^2$: Entire Space Counterfactual Multi-Task Model for Post-Click Conversion Rate Estimation

Accurate estimation of post-click conversion rate is critical for building recommender systems, which has long been confronted with sample selection bias and data sparsity issues. Methods in the Entire Space Multi-task Model (ESMM) family leverage the sequential pattern of user actions, i.e. $impression\rightarrow click \rightarrow conversion$ to address data sparsity issue. However, they still fail to ensure the unbiasedness of CVR estimates. In this paper, we theoretically demonstrate that ESMM suffers from the following two problems: (1) Inherent Estimation Bias (IEB), where the estimated CVR of ESMM is inherently higher than the ground truth; (2) Potential Independence Priority (PIP) for CTCVR estimation, where there is a risk that the ESMM overlooks the causality from click to conversion. To this end, we devise a principled approach named Entire Space Counterfactual Multi-task Modelling (ESCM$^2$), which employs a counterfactual risk miminizer as a regularizer in ESMM to address both IEB and PIP issues simultaneously. Extensive experiments on offline datasets and online environments demonstrate that our proposed ESCM$^2$ can largely mitigate the inherent IEB and PIP issues and achieve better performance than baseline models.

cs.AI↗

arXiv2022

Constrained Sequence-to-Tree Generation for Hierarchical Text Classification

Hierarchical Text Classification (HTC) is a challenging task where a document can be assigned to multiple hierarchically structured categories within a taxonomy. The majority of prior studies consider HTC as a flat multi-label classification problem, which inevitably leads to "label inconsistency" problem. In this paper, we formulate HTC as a sequence generation task and introduce a sequence-to-tree framework (Seq2Tree) for modeling the hierarchical label structure. Moreover, we design a constrained decoding strategy with dynamic vocabulary to secure the label consistency of the results. Compared with previous works, the proposed approach achieves significant and consistent improvements on three benchmark datasets.

cs.CL↗

arXiv2022

Passive Motion Detection via mmWave Communication System

In this paper, an integrated passive sensing and communication system working in 60 GHz band is elaborated, and the sensing performance is investigated in an application of hand gesture recognition. Specifically, in this integrated system, there are two radio frequency (RF) chains at the receiver and one at the transmitter. Each RF chain is connected with one phased array for analog beamforming. To facilitate simultaneous sensing and communication, the transmitter delivers one stream of information-bearing signals via two beam lobes, one is aligned with the main signal propagation path and the other is directed to the sensing target. Signals from the two lobes are received by the two RF chains at the receiver, respectively. By cross ambiguity coherent processing, the time-Doppler spectrograms of hand gestures can be obtained. Relying on the passive sensing system, a dataset of received signals, where three types of hand gestures are sensed, is collected by using Line-of-Sight (LoS) and Non-Line-of-Sight (NLoS) paths as the reference channel respectively. Then a neural network is trained by the dataset for motion detection. It is shown that the classification accuracy rate is high as long as sufficient sensing time is assured. Finally, an empirical model characterizing the relation between the classification accuracy and sensing duration is derived analytically.

eess.SP↗

arXiv2022

Quasi-Monte Carlo-Based Conditional Malliavin Method for Continuous-Time Asian Option Greeks

Although many methods for computing the Greeks of discrete-time Asian options are proposed, few methods to calculate the Greeks of continuous-time Asian options are known. In this paper, we develop an integration by parts formula in the multi-dimensional Malliavin calculus, and apply it to obtain the Greeks formulae for continuous-time Asian options in the multi-asset situation. We combine the Malliavin method with the quasi-Monte Carlo method to calculate the Greeks in simulation. We discuss the asymptotic convergence of simulation estimates for the continuous-time Asian option Greeks obtained by Malliavin derivatives. We propose to use the conditional quasi-Monte Carlo method to smooth Malliavin Greeks, and show that the calculation of conditional expectations analytically is viable for many types of Asian options. We prove that the new estimates for Greeks have good smoothness. For binary Asian options, Asian call options and up-and-out Asian call options, for instance, our estimates are infinitely times differentiable. We take the gradient principal component analysis method as a dimension reduction technique in simulation. Numerical experiments demonstrate the large efficiency improvement of the proposed method, especially for Asian options with discontinuous payoff functions.

math.NA↗

arXiv2022

High-harmonic generation approaching the quantum critical point of strongly correlated systems

By employing the exact diagonalization method, we investigate the high-harmonic generation (HHG) of the correlated systems under the strong laser irradiation. For the extended Hubbard model on a periodic chain, HHG close to the quantum critical point (QCP) is more significant compared to two neighboring gapped phases (i.e., charge-density-wave and spin-density wave states), especially in low-frequencies. We confirm that the systems in the vicinity of the QCP are supersensitive to the external field and more optical-transition channels via excited states are responsible for HHG. This feature holds the potential of obtaining high-efficiency harmonics by making use of materials approaching to QCP. Based on two-dimensional Haldane model, we further propose that the even- or odd-order components of generated harmonics can be promisingly regarded as spectral signals to distinguish the topologically ordered phases from locally ordered ones. Our findings in this work pave the way to achieve ultrafast light source from HHG in strongly correlated materials and to study quantum phase transition by nonlinear optics in strong laser fields.

cond-mat.str-el↗

arXiv2022

Note on surface growth approach for bulk reconstruction

In a recent paper, a novel surface growth approach for reconstructing bulk geometry and matter fields was proposed, it was shown that this picture can be explicitly realized by the one-shot entanglement distillation tensor network and the surface state correspondence. In the present paper, we give direct analysis for the growth of the bulk minimal surfaces in asymptotically AdS 3 spacetime and show that bulk geometry can be efficiently reproduced in this way, which provides further support for the surface growth approach in entanglement wedge reconstruction.

hep-th↗

arXiv2022

Solid-like high harmonic generation from rotationally periodic systems

High harmonic generation (HHG) from crystals in strong laser fields has been understood by the band theory of solid, which is based on the periodic boundary condition (PBC) of translational invariant. For systems having PBC of rotational invariant, in principles an analogous Bloch theorem can be developed and applied. Taking a ring-type cluster of cyclo[18]carbon as a representative, we theoretically suggest a quasi-band model and study its HHG by solving time-dependent Liouville-von Neumann equation. Under the irradiation of circularly polarized laser, explicit selection rules for left-handed and right-handed harmonics are observed, while in linearly polarized laser field, cyclo[18]carbon exhibits solid-like HHG originated from intra-band oscillations and inter-band transitions, which in turn is promising to optically detect the symmetry and geometry of controversial structures. In a sense, this work presents a connection linking the high harmonics of gases and solids.

physics.optics↗

arXiv2022

The Role of Shift Vector in High-Harmonic Generation from Non-Centrosymmetric Topological Insulators under Strong Laser Fields

As a promising avenue to obtain new extreme ultraviolet light source and detect electronic properties, high-harmonic generation (HHG) has been actively developed in both theory and experiment. In solids lacking inversion symmetry, when electrons undergo a nonadiabatic transition, a directional charge shift occurs and is characterized by shift vector, which measures the real-space shift of the photoexcited electron and hole. For the first time, we have revealed that shift vector plays prominent roles in the real-space tunneling mechanism of three-step model for electrons under strong laser fields. Since shift vector is determined by the topological properties of related wave functions, we expect HHG with its contribution can provide direct knowledge on the band topology in noncentrosymmetric topological insulators (TIs). In both Kane-Mele model and realistic material BiTeI, we have found that the shift vector reverses when band inversion happens during the topological phase transition between normal and topological insulators. Under oscillating strong laser fields, the reversal of shift vector leads to completely opposite radiation time of high-order harmonics. This makes HHG a feasible all-optical strong-field method to directly identify the band inversion in non-centrosymmetric TIs.

physics.optics↗