arXiv subjects

Ran Cheng

Publications and source records attributed to Ran Cheng.

At least 109 records · Page 6Linked to original sources

arXiv2023

Learning Grounded Vision-Language Representation for Versatile Understanding in Untrimmed Videos

Joint video-language learning has received increasing attention in recent years. However, existing works mainly focus on single or multiple trimmed video clips (events), which makes human-annotated event boundaries necessary during inference. To break away from the ties, we propose a grounded vision-language learning framework for untrimmed videos, which automatically detects informative events and effectively excavates the alignments between multi-sentence descriptions and corresponding event segments. Instead of coarse-level video-language alignments, we present two dual pretext tasks to encourage fine-grained segment-level alignments, i.e., text-to-event grounding (TEG) and event-to-text generation (ETG). TEG learns to adaptively ground the possible event proposals given a set of sentences by estimating the cross-modal distance in a joint semantic space. Meanwhile, ETG aims to reconstruct (generate) the matched texts given event proposals, encouraging the event representation to retain meaningful semantic information. To encourage accurate label assignment between the event set and the text set, we propose a novel semantic-aware cost to mitigate the sub-optimal matching results caused by ambiguous boundary annotations. Our framework is easily extensible to tasks covering visually-grounded language understanding and generation. We achieve state-of-the-art dense video captioning performance on ActivityNet Captions, YouCook2 and YouMakeup, and competitive performance on several other language generation and understanding tasks. Our method also achieved 1st place in both the MTVG and MDVC tasks of the PIC 4th Challenge. Our code is publicly available at https://github.com/zjr2000/GVL.

cs.CV↗

arXiv2023

Rethinking Population-assisted Off-policy Reinforcement Learning

While off-policy reinforcement learning (RL) algorithms are sample efficient due to gradient-based updates and data reuse in the replay buffer, they struggle with convergence to local optima due to limited exploration. On the other hand, population-based algorithms offer a natural exploration strategy, but their heuristic black-box operators are inefficient. Recent algorithms have integrated these two methods, connecting them through a shared replay buffer. However, the effect of using diverse data from population optimization iterations on off-policy RL algorithms has not been thoroughly investigated. In this paper, we first analyze the use of off-policy RL algorithms in combination with population-based algorithms, showing that the use of population data could introduce an overlooked error and harm performance. To test this, we propose a uniform and scalable training design and conduct experiments on our tailored framework in robot locomotion tasks from the OpenAI gym. Our results substantiate that using population data in off-policy RL can cause instability during training and even degrade performance. To remedy this issue, we further propose a double replay buffer design that provides more on-policy data and show its effectiveness through experiments. Our results offer practical insights for training these hybrid methods.

cs.LG↗

arXiv2023

Neural Architecture Search as Multiobjective Optimization Benchmarks: Problem Formulation and Performance Assessment

The ongoing advancements in network architecture design have led to remarkable achievements in deep learning across various challenging computer vision tasks. Meanwhile, the development of neural architecture search (NAS) has provided promising approaches to automating the design of network architectures for lower prediction error. Recently, the emerging application scenarios of deep learning have raised higher demands for network architectures considering multiple design criteria: number of parameters/floating-point operations, and inference latency, among others. From an optimization point of view, the NAS tasks involving multiple design criteria are intrinsically multiobjective optimization problems; hence, it is reasonable to adopt evolutionary multiobjective optimization (EMO) algorithms for tackling them. Nonetheless, there is still a clear gap confining the related research along this pathway: on the one hand, there is a lack of a general problem formulation of NAS tasks from an optimization point of view; on the other hand, there are challenges in conducting benchmark assessments of EMO algorithms on NAS tasks. To bridge the gap: (i) we formulate NAS tasks into general multi-objective optimization problems and analyze the complex characteristics from an optimization point of view; (ii) we present an end-to-end pipeline, dubbed $\texttt{EvoXBench}$, to generate benchmark test problems for EMO algorithms to run efficiently -- without the requirement of GPUs or Pytorch/Tensorflow; (iii) we instantiate two test suites comprehensively covering two datasets, seven search spaces, and three hardware devices, involving up to eight objectives. Based on the above, we validate the proposed test suites using six representative EMO algorithms and provide some empirical analyses. The code of $\texttt{EvoXBench}$ is available from $\href{https://github.com/EMI-Group/EvoXBench}{\rm{here}}$.

cs.NE↗

arXiv2023

Accelerating Vision-Language Pretraining with Free Language Modeling

The state of the arts in vision-language pretraining (VLP) achieves exemplary performance but suffers from high training costs resulting from slow convergence and long training time, especially on large-scale web datasets. An essential obstacle to training efficiency lies in the entangled prediction rate (percentage of tokens for reconstruction) and corruption rate (percentage of corrupted tokens) in masked language modeling (MLM), that is, a proper corruption rate is achieved at the cost of a large portion of output tokens being excluded from prediction loss. To accelerate the convergence of VLP, we propose a new pretraining task, namely, free language modeling (FLM), that enables a 100% prediction rate with arbitrary corruption rates. FLM successfully frees the prediction rate from the tie-up with the corruption rate while allowing the corruption spans to be customized for each token to be predicted. FLM-trained models are encouraged to learn better and faster given the same GPU time by exploiting bidirectional contexts more flexibly. Extensive experiments show FLM could achieve an impressive 2.5x pretraining time reduction in comparison to the MLM-based methods, while keeping competitive performance on both vision-language understanding and generation tasks. Code will be public at https://github.com/TencentARC/FLM.

cs.CV↗

arXiv2023

Identifying Axion Insulator by Quantized Magnetoelectric Effect in Antiferromagnetic ${\mathrm{MnBi}}_{2}{\mathrm{Te}}_{4}$ Tunnel Junction

Intrinsic magnetic topological insulator ${\mathrm{MnBi}}_{2}{\mathrm{Te}}_{4}$ is believed to be an axion insulator in its antiferromagnetic ground state. However, direct identification of axion insulators remains experimentally elusive because the observed vanishing Hall resistance, while indicating the onset of the axion field, is inadequate to distinguish the system from a trivial normal insulator. Using numerical Green's functions, we theoretically demonstrate the quantized magnetoelectric current in a tunnel junction of atomically thin ${\mathrm{MnBi}}_{2}{\mathrm{Te}}_{4}$ sandwiched between two contacts, which is a smoking-gun signal that unambiguously confirms antiferromagnetic ${\mathrm{MnBi}}_{2}{\mathrm{Te}}_{4}$ to be an axion insulator. Our predictions can be verified directly by experiments.

cond-mat.mes-hall↗

arXiv2023

Spotlights: Probing Shapes from Spherical Viewpoints

Recent years have witnessed the surge of learned representations that directly build upon point clouds. Though becoming increasingly expressive, most existing representations still struggle to generate ordered point sets. Inspired by spherical multi-view scanners, we propose a novel sampling model called Spotlights to represent a 3D shape as a compact 1D array of depth values. It simulates the configuration of cameras evenly distributed on a sphere, where each virtual camera casts light rays from its principal point through sample points on a small concentric spherical cap to probe for the possible intersections with the object surrounded by the sphere. The structured point cloud is hence given implicitly as a function of depths. We provide a detailed geometric analysis of this new sampling scheme and prove its effectiveness in the context of the point cloud completion task. Experimental results on both synthetic and real data demonstrate that our method achieves competitive accuracy and consistency while having a significantly reduced computational cost. Furthermore, we show superior performance on the downstream point cloud registration task over state-of-the-art completion methods.

cs.CV↗

arXiv2022

Optically-generated focused ultrasound for noninvasive brain stimulation with ultrahigh precision

High precision neuromodulation is a powerful tool to decipher neurocircuits and treat neurological diseases. Current non-invasive neuromodulation methods offer limited precision at the millimeter level. Here, we report optically-generated focused ultrasound (OFUS) for non-invasive brain stimulation with ultrahigh precision. OFUS is generated by a soft optoacoustic pad (SOAP) fabricated through embedding candle soot nanoparticles in a curved polydimethylsiloxane film. SOAP generates a transcranial ultrasound focus at 15 MHz with an ultrahigh lateral resolution of 83 um, which is two orders of magnitude smaller than that of conventional transcranial-focused ultrasound (tFUS). Here, we show effective OFUS neurostimulation in vitro with a single ultrasound cycle. We demonstrate submillimeter transcranial stimulation of the mouse motor cortex in vivo. An acoustic energy of 0.6 mJ/cm^2, four orders of magnitude less than that of tFUS, is sufficient for successful OFUS neurostimulation. OFUS offers new capabilities for neuroscience studies and disease treatments by delivering a focus with ultrahigh precision non-invasively.

q-bio.NC↗

arXiv2022

Lamarckian Platform: Pushing the Boundaries of Evolutionary Reinforcement Learning towards Asynchronous Commercial Games

Despite the emerging progress of integrating evolutionary computation into reinforcement learning, the absence of a high-performance platform endowing composability and massive parallelism causes non-trivial difficulties for research and applications related to asynchronous commercial games. Here we introduce Lamarckian - an open-source platform featuring support for evolutionary reinforcement learning scalable to distributed computing resources. To improve the training speed and data efficiency, Lamarckian adopts optimized communication methods and an asynchronous evolutionary reinforcement learning workflow. To meet the demand for an asynchronous interface by commercial games and various methods, Lamarckian tailors an asynchronous Markov Decision Process interface and designs an object-oriented software architecture with decoupled modules. In comparison with the state-of-the-art RLlib, we empirically demonstrate the unique advantages of Lamarckian on benchmark tests with up to 6000 CPU cores: i) both the sampling efficiency and training speed are doubled when running PPO on Google football game; ii) the training speed is 13 times faster when running PBT+PPO on Pong game. Moreover, we also present two use cases: i) how Lamarckian is applied to generating behavior-diverse game AI; ii) how Lamarckian is applied to game balancing tests for an asynchronous commercial game.

cs.LG↗

arXiv2022

Quantum interference in a superconductor-${\mathrm{MnBi}}_{2}{\mathrm{Te}}_{4}$-superconductor Josephson junction

We study the transport properties of a Josephson junction consisting of two identical $s$-wave superconductors separated by an even-layer ${\mathrm{MnBi}}_{2}{\mathrm{Te}}_{4}$ (MBT). Using recursive Green's function method, we calculate the supercurrent in the presence of a perpendicular magnetic field and find that its quantum interference exhibits distinct patterns when the MBT is in different magnetic states. In the antiferromagnetic state, the MBT is an axion insulator supporting an extended "hinge" supercurrent, which leads to a sinusoidal interference pattern decaying with the field strength. In the ferromagnetic state, the MBT is a Chern insulator and the unbalanced chiral supercurrents on opposite edges give rise to a highly asymmetric interference pattern. If the MBT turns into a metal as the Fermi level is tuned into the conduction band, the interference exhibits a Fraunhofer pattern due to the uniformly distributed bulk supercurrent. Our work unravels a strong indicator to identify different phases in the MBT and can be verified directly by experiments.

cond-mat.mes-hall↗

arXiv2022

Field-Assisted Sub-Terahertz Spin Pumping and Auto-Oscillation in NiO

Spin pumping converting sub-terahertz electromagnetic waves to DC spin currents has recently been demonstrated in antiferromagnets (AFMs) with easy-axis magnetic anisotropy. However, easy-plane AFMs such as NiO, which are easier to prepare experimentally, are considered to be bad candidates for spin pumping because the Néel vector oscillation is linearly polarized, placing a major restriction on the material choice for practical applications. Through a case study of NiO, we show that an applied magnetic field below the spin-flop transition can substantially modify the polarization of the resonance eigenmodes, which enables coherent sub-terahertz spin pumping as strong as that in easy-axis AFMs. In addition, we find that an applied magnetic field can significantly reduce the threshold of Néel vector auto-oscillation triggered by spin-transfer torques. These prominent field-assisted effects can greatly facilitate spintronic device engineering in the sub-terahertz frequency regime.

cond-mat.mes-hall↗

arXiv2022

Multiobjective Test Problems with Degenerate Pareto Fronts

In multiobjective optimisation, a set of scalable test problems with a variety of features allow researchers to investigate and evaluate the abilities of different optimisation algorithms, and thus can help them to design and develop more effective and efficient approaches. Existing test problem suites mainly focus on situations where all the objectives are fully conflicting with each other. In such cases, an m-objective optimisation problem has an (m-1)-dimensional Pareto front in the objective space. However, in some optimisation problems, there may be unexpected characteristics among objectives, e.g., redundancy. The redundancy of some objectives can lead to the multiobjective problem having a degenerate Pareto front, i.e., the dimension of the Pareto front of the $m$-objective problem be less than (m-1). In this paper, we systematically study degenerate multiobjective problems. We abstract three general characteristics of degenerate problems, which are not formulated and systematically investigated in the literature. Based on these characteristics, we present a set of test problems to support the investigation of multiobjective optimisation algorithms under situations with redundant objectives. To the best of our knowledge, this work is the first one that explicitly formulates these three characteristics of degenerate problems, thus allowing the resulting test problems to be featured by their generality, in contrast to existing test problems designed for specific purposes (e.g., visualisation).

cs.NE↗

arXiv2022

Surrogate-assisted Multi-objective Neural Architecture Search for Real-time Semantic Segmentation

The architectural advancements in deep neural networks have led to remarkable leap-forwards across a broad array of computer vision tasks. Instead of relying on human expertise, neural architecture search (NAS) has emerged as a promising avenue toward automating the design of architectures. While recent achievements in image classification have suggested opportunities, the promises of NAS have yet to be thoroughly assessed on more challenging tasks of semantic segmentation. The main challenges of applying NAS to semantic segmentation arise from two aspects: (i) high-resolution images to be processed; (ii) additional requirement of real-time inference speed (i.e., real-time semantic segmentation) for applications such as autonomous driving. To meet such challenges, we propose a surrogate-assisted multi-objective method in this paper. Through a series of customized prediction models, our method effectively transforms the original NAS task into an ordinary multi-objective optimization problem. Followed by a hierarchical pre-screening criterion for in-fill selection, our method progressively achieves a set of efficient architectures trading-off between segmentation accuracy and inference speed. Empirical evaluations on three benchmark datasets together with an application using Huawei Atlas 200 DK suggest that our method can identify architectures significantly outperforming existing state-of-the-art architectures designed both manually by human experts and automatically by other NAS methods.

cs.CV↗

arXiv2022

Voltage-driven exchange resonance achieving 100\% mechanical efficiency

Magnetic resonances driven by current-induced torques are crucial tools to study magnetic materials but are very limited in frequency and mechanical efficiency. We propose an alternative mechanism, voltage-induced torque, to realize high efficiency in generating high-frequency magnetization dynamics. When a ferromagnet-topological insulator-ferromagnet trilayer heterostructure is operated as an adiabatic quantum motor, voltage-induced torque arises from the adiabatic motion of gapped topological electrons on the two interfaces and act oppositely on the two ferromagnetic layers, which can excite the exchange mode where the two ferromagnetic layers precess with a $π$-phase difference. The exchange mode resonance, bearing a much higher frequency than the ferromagnetic resonance, is accompanied by topological charge pumping, leading to a sharp peak in electrical admittance at the resonance point. Because the output current is purely adiabatic while dissipative current vanishes identically, the proposed voltage-driven exchange resonance entails a remarkably high mechanical efficiency close to unity, which is impossible in any current-driven systems.

cond-mat.mes-hall↗

arXiv2022

Unidirectional Spin Hall Magnetoresistance in Antiferromagnetic Heterostructures

Unidirectional spin Hall magnetoresistance (USMR) has been widely reported in the heavy metal / ferromagnet (HM/FM) bilayer systems. We observe the USMR in the Pt/α-Fe2O3 bilayers where the α-Fe2O3 is an antiferromagnetic (AFM) insulator. Systematic field and temperature dependent measurements confirm the magnonic origin of the USMR. The appearance of AFM-USMR is driven by the imbalance of creation and annihilation of AFM magnons by spin orbit torque due to thermal random field. However, unlike its ferromagnetic counterpart, theoretical modeling reveals that the USMR in Pt/α-Fe2O3 is determined by the antiferromagtic magnon number, and with a non-monotonic field dependence. Our findings extend the generality of the USMR which pave the ways for the highly sensitive detection of AF spin state.

cond-mat.mes-hall↗

arXiv2022

Bi-fidelity Evolutionary Multiobjective Search for Adversarially Robust Deep Neural Architectures

Deep neural networks have been found vulnerable to adversarial attacks, thus raising potentially concerns in security-sensitive contexts. To address this problem, recent research has investigated the adversarial robustness of deep neural networks from the architectural point of view. However, searching for architectures of deep neural networks is computationally expensive, particularly when coupled with adversarial training process. To meet the above challenge, this paper proposes a bi-fidelity multiobjective neural architecture search approach. First, we formulate the NAS problem for enhancing adversarial robustness of deep neural networks into a multiobjective optimization problem. Specifically, in addition to a low-fidelity performance predictor as the first objective, we leverage an auxiliary-objective -- the value of which is the output of a surrogate model trained with high-fidelity evaluations. Secondly, we reduce the computational cost by combining three performance estimation methods, i.e., parameter sharing, low-fidelity evaluation, and surrogate-based predictor. The effectiveness of the proposed approach is confirmed by extensive experiments conducted on CIFAR-10, CIFAR-100 and SVHN datasets.

cs.LG↗

arXiv2022

Exploiting Context Information for Generic Event Boundary Captioning

Generic Event Boundary Captioning (GEBC) aims to generate three sentences describing the status change for a given time boundary. Previous methods only process the information of a single boundary at a time, which lacks utilization of video context information. To tackle this issue, we design a model that directly takes the whole video as input and generates captions for all boundaries parallelly. The model could learn the context information for each time boundary by modeling the boundary-boundary interactions. Experiments demonstrate the effectiveness of context information. The proposed method achieved a 72.84 score on the test set, and we reached the $2^{nd}$ place in this challenge. Our code is available at: \url{https://github.com/zjr2000/Context-GEBC}

cs.CV↗

arXiv2022

SoloGAN: Multi-domain Multimodal Unpaired Image-to-Image Translation via a Single Generative Adversarial Network

Despite significant advances in image-to-image (I2I) translation with generative adversarial networks (GANs), it remains challenging to effectively translate an image to a set of diverse images in multiple target domains using a single pair of generator and discriminator. Existing I2I translation methods adopt multiple domain-specific content encoders for different domains, where each domain-specific content encoder is trained with images from the same domain only. Nevertheless, we argue that the content (domain-invariance) features should be learned from images among all of the domains. Consequently, each domain-specific content encoder of existing schemes fails to extract the domain-invariant features efficiently. To address this issue, we present a flexible and general SoloGAN model for efficient multimodal I2I translation among multiple domains with unpaired data. In contrast to existing methods, the SoloGAN algorithm uses a single projection discriminator with an additional auxiliary classifier and shares the encoder and generator for all domains. Consequently, the SoloGAN can be trained effectively with images from all domains such that the domain-invariance content representation can be efficiently extracted. Qualitative and quantitative results over a wide range of datasets against several counterparts and variants of the SoloGAN demonstrate the merits of the method, especially for challenging I2I translation datasets, i.e., datasets involving extreme shape variations or need to keep the complex backgrounds unchanged after translations. Furthermore, we demonstrate the contribution of each component in SoloGAN by ablation studies.

cs.CV↗

arXiv2022

VLMixer: Unpaired Vision-Language Pre-training via Cross-Modal CutMix

Existing vision-language pre-training (VLP) methods primarily rely on paired image-text datasets, which are either annotated by enormous human labors, or crawled from the internet followed by elaborate data cleaning techniques. To reduce the dependency on well-aligned image-text pairs, it is promising to directly leverage the large-scale text-only and image-only corpora. This paper proposes a data augmentation method, namely cross-modal CutMix (CMC), for implicit cross-modal alignment learning in unpaired VLP. Specifically, CMC transforms natural sentences from the textual view into a multi-modal view, where visually-grounded words in a sentence are randomly replaced by diverse image patches with similar semantics. There are several appealing proprieties of the proposed CMC. First, it enhances the data diversity while keeping the semantic meaning intact for tackling problems where the aligned data are scarce; Second, by attaching cross-modal noise on uni-modal data, it guides models to learn token-level interactions across modalities for better denoising. Furthermore, we present a new unpaired VLP method, dubbed as VLMixer, that integrates CMC with contrastive learning to pull together the uni-modal and multi-modal views for better instance-level alignments among different modalities. Extensive experiments on five downstream tasks show that VLMixer could surpass previous state-of-the-art unpaired VLP methods.

cs.CV↗