arXiv subjects

Bo Liu

Publications and source records attributed to Bo Liu.

At least 433 records · Page 24Linked to original sources

arXiv2023

Metric Residual Networks for Sample Efficient Goal-Conditioned Reinforcement Learning

Goal-conditioned reinforcement learning (GCRL) has a wide range of potential real-world applications, including manipulation and navigation problems in robotics. Especially in such robotics tasks, sample efficiency is of the utmost importance for GCRL since, by default, the agent is only rewarded when it reaches its goal. While several methods have been proposed to improve the sample efficiency of GCRL, one relatively under-studied approach is the design of neural architectures to support sample efficiency. In this work, we introduce a novel neural architecture for GCRL that achieves significantly better sample efficiency than the commonly-used monolithic network architecture. The key insight is that the optimal action-value function Q^*(s, a, g) must satisfy the triangle inequality in a specific sense. Furthermore, we introduce the metric residual network (MRN) that deliberately decomposes the action-value function Q(s,a,g) into the negated summation of a metric plus a residual asymmetric component. MRN provably approximates any optimal action-value function Q^*(s,a,g), thus making it a fitting neural architecture for GCRL. We conduct comprehensive experiments across 12 standard benchmark environments in GCRL. The empirical results demonstrate that MRN uniformly outperforms other state-of-the-art GCRL neural architectures in terms of sample efficiency.

cs.LG↗

arXiv2023

Principal Stratification with Time-to-Event Outcomes

Post-randomization events, also known as intercurrent events, such as treatment noncompliance and censoring due to a terminal event, are common in clinical trials. Principal stratification is a framework for causal inference in the presence of intercurrent events. Despite the extensive existing literature, there lacks generally applicable and accessible methods for principal stratification analysis with time-to-event outcomes. In this paper, we specify two causal estimands for time-to-event outcomes in principal stratification. For estimation, we adopt the general strategy of latent mixture modeling and derive the corresponding likelihood function. For computational convenience, we illustrate the general strategy with a mixture of Bayesian parametric Weibull-Cox proportional model for the outcome. We utilize the Stan programming language to obtain automatic posterior sampling of the model parameters via the Hamiltonian Monte Carlo. We provide the analytical forms of the causal estimands as functions of the model parameters and an alternative numerical method when analytical forms are not available. We apply the proposed method to the ADAPTABLE trial to evaluate the causal effect of taking 81 mg versus 325 mg aspirin on the risk of major adverse cardiovascular events.

stat.ME↗

arXiv2023

Async-fork: Mitigating Query Latency Spikes Incurred by the Fork-based Snapshot Mechanism from the OS Level

In-memory key-value stores (IMKVSes) serve many online applications because of their efficiency. To support data backup, popular industrial IMKVSes periodically take a point-in-time snapshot of the in-memory data with the system call fork. However, this mechanism can result in latency spikes for queries arriving during the snapshot period because fork leads the engine into the kernel mode in which the engine is out-of-service for queries. In contrast to existing research focusing on optimizing snapshot algorithms, we optimize the fork operation to address the latency spikes problem from the operating system (OS) level, while keeping the data persistent mechanism in IMKVSes unchanged. Specifically, we first conduct an in-depth study to reveal the impact of the fork operation as well as the optimization techniques on query latency. Based on findings in the study, we propose Async-fork to offload the work of copying the page table from the engine (the parent process) to the child process as copying the page table dominates the execution time of fork. To keep data consistent between the parent and the child, we design the proactive synchronization strategy. Async-fork is implemented in the Linux kernel and deployed into the online Redis database in public clouds. Our experiment results show that compared with the default fork method in OS, Async-fork reduces the tail latency of queries arriving during the snapshot period by 81.76% on an 8GB instance and 99.84% on a 64GB instance.

cs.DB↗

arXiv2023

Self-supervised Hypergraph Representation Learning for Sociological Analysis

Modern sociology has profoundly uncovered many convincing social criteria for behavioural analysis. Unfortunately, many of them are too subjective to be measured and presented in online social networks. On the other hand, data mining techniques can better find data patterns but many of them leave behind unnatural understanding. In this paper, we propose a fundamental methodology to support the further fusion of data mining techniques and sociological behavioral criteria. Our highlights are three-fold: First, we propose an effective hypergraph awareness and a fast line graph construction framework. The hypergraph can more profoundly indicate the interactions between individuals and their environments because each edge in the hypergraph (a.k.a hyperedge) contains more than two nodes, which is perfect to describe social environments. A line graph treats each social environment as a super node with the underlying influence between different environments. In this way, we go beyond traditional pair-wise relations and explore richer patterns under various sociological criteria; Second, we propose a novel hypergraph-based neural network to learn social influence flowing from users to users, users to environments, environment to users, and environments to environments. The neural network can be learned via a task-free method, making our model very flexible to support various data mining tasks and sociological analysis; Third, we propose both qualitative and quantitive solutions to effectively evaluate the most common sociological criteria like social conformity, social equivalence, environmental evolving and social polarization. Our extensive experiments show that our framework can better support both data mining tasks for online user behaviours and sociological analysis.

cs.SI↗

arXiv2022

Tunable van Hove singularity without structural instability in Kagome metal CsTi$_3$Bi$_5$

In Kagome metal CsV$_3$Sb$_5$, multiple intertwined orders are accompanied by both electronic and structural instabilities. These exotic orders have attracted much recent attention, but their origins remain elusive. The newly discovered CsTi$_3$Bi$_5$ is a Ti-based Kagome metal to parallel CsV$_3$Sb$_5$. Here, we report angle-resolved photoemission experiments and first-principles calculations on pristine and Cs-doped CsTi$_3$Bi$_5$ samples. Our results reveal that the van Hove singularity (vHS) in CsTi$_3$Bi$_5$ can be tuned in a large energy range without structural instability, different from that in CsV$_3$Sb$_5$. As such, CsTi$_3$Bi$_5$ provides a complementary platform to disentangle and investigate the electronic instability with a tunable vHS in Kagome metals.

cond-mat.str-el↗

arXiv2022

Complete cavity map of the C. elegans connectome

Network structure or topology is the basis for understanding complex systems. Recently, higher-order structures have been considered as a new research direction that can provide new perspectives and phenomena. However, most existing studies have focused on simple 1-cycles and triangles, and few have explored the higher-order and non-trivial cycles. The current study focused on the cavity, which is a non-trivial cycle. We proposed a method to compute cavities with different orders based on pruning in the boundary matrix and calculated all cavities of the neural network of Caenorhabditis elegans (\emph{C. elegans}) neural network. This study reports for the first time a complete cavity map of C. elegans neural network, developing a new method for mining higher-order structures that can be applied by researchers in neuroscience, network science and other interdisciplinary fields to explore higher-order structural markers of complex systems.

q-bio.NC↗

arXiv2022

The Extreme Super-Eddington NLS1 RX J0134.2-4258 -- II. A Weak-Line Seyfert Linking to the Weak-Line Quasar

RX J0134.2-4258 is one of the most super-Eddington narrow-line Seyfert 1 (NLS1) galaxies, on which we conducted a monitoring campaign from radio to X-rays. In this paper, we present a detailed analysis of its optical/UV spectra and broadband spectral energy distribution (SED). Our study shows that the preferred black hole mass of RX J0134.2-4258 is $M_{\rm BH} \sim 2 \times 10^{7}~M_{\odot}$, giving a mass accretion rate through the outer disc of $\dot{m}_{\rm out} \sim 20$ (assuming zero spin), compared to the observed luminosity ratio $L_{\rm bol}/L_{\rm Edd} \sim 6$. This reduction in radiative efficiency is expected for super-Eddington flows, as power can be lost via advection and/or disc winds. We find that the optical/UV lines of RX J0134.2-4258 resemble those from weak-like quasars (WLQs), as it has notably weak C IV and N V emission lines. It also has drastic X-ray variability, again similar to that recently observed in some other WLQs. However, WLQs have systematically higher masses ($\gtrsim 10^8~M_{\odot}$), and lower Eddington ratios ($\dot{m}_{\rm out} \sim 1$) than RX J0134.2-4258. We compare instead to the most extreme NLS1s, with similarly large $\dot{m}_{\rm out}$ but smaller masses. These show similarly large reductions in radiative efficiency but their UV lines are not similarly wind-dominated. We suggest a new category of weak-line Seyfert (WLS) galaxies to describe sources like RX J0134.2-4258, and interpret its (so far unique) properties in a model, where the lower-disc-temperature in the higher-mass black holes leads to the UV-line-driving mechanism, which enhances the super-Eddington radiation-pressure-driven wind.

astro-ph.HE↗

arXiv2022

Intelligent Computing: The Latest Advances, Challenges and Future

Computing is a critical driving force in the development of human civilization. In recent years, we have witnessed the emergence of intelligent computing, a new computing paradigm that is reshaping traditional computing and promoting digital revolution in the era of big data, artificial intelligence and internet-of-things with new computing theories, architectures, methods, systems, and applications. Intelligent computing has greatly broadened the scope of computing, extending it from traditional computing on data to increasingly diverse computing paradigms such as perceptual intelligence, cognitive intelligence, autonomous intelligence, and human-computer fusion intelligence. Intelligence and computing have undergone paths of different evolution and development for a long time but have become increasingly intertwined in recent years: intelligent computing is not only intelligence-oriented but also intelligence-driven. Such cross-fertilization has prompted the emergence and rapid advancement of intelligent computing. Intelligent computing is still in its infancy and an abundance of innovations in the theories, systems, and applications of intelligent computing are expected to occur soon. We present the first comprehensive survey of literature on intelligent computing, covering its theory fundamentals, the technological fusion of intelligence and computing, important applications, challenges, and future perspectives. We believe that this survey is highly timely and will provide a comprehensive reference and cast valuable insights into intelligent computing for academic and industrial researchers and practitioners.

cs.AI↗

arXiv2022

Multi-timescale Event Detection in Nonintrusive Load Monitoring based on MDL Principle

Load event detection is the fundamental step for the event-based non-intrusive load monitoring (NILM). However, existing event detection methods with fixed parameters may fail in coping with the inherent multi-timescale characteristics of events and their event detection accuracy is easily affected by the load fluctuation. In this regard, this paper extends our previously designed two-stage event detection framework, and proposes a novel multi-timescale event detection method based on the principle of minimum description length (MDL). Following the completion of step-like event detection in the first stage, a long-transient event detection scheme with variable-length sliding window is designed for the second stage, which is intended to provide the observation and characterization of the same event at different time scales. In that, the context information in the aggregated load data is mined by motif discovery, and then based on the MDL principle, the proper observation scales are selected for different events and the corresponding detection results are determined. In the post-processing step, a load fluctuation location method based on voice activity detection (VAD) is proposed to identify and remove the unreasonable events caused by fluctuations. Based on newly proposed evaluation metrics, the comparison tests on public and private datasets demonstrate that our method achieves higher detection accuracy and integrity for events of various appliances across different scenarios.

eess.SP↗

arXiv2022

AlignVE: Visual Entailment Recognition Based on Alignment Relations

Visual entailment (VE) is to recognize whether the semantics of a hypothesis text can be inferred from the given premise image, which is one special task among recent emerged vision and language understanding tasks. Currently, most of the existing VE approaches are derived from the methods of visual question answering. They recognize visual entailment by quantifying the similarity between the hypothesis and premise in the content semantic features from multi modalities. Such approaches, however, ignore the VE's unique nature of relation inference between the premise and hypothesis. Therefore, in this paper, a new architecture called AlignVE is proposed to solve the visual entailment problem with a relation interaction method. It models the relation between the premise and hypothesis as an alignment matrix. Then it introduces a pooling operation to get feature vectors with a fixed size. Finally, it goes through the fully-connected layer and normalization layer to complete the classification. Experiments show that our alignment-based architecture reaches 72.45\% accuracy on SNLI-VE dataset, outperforming previous content-based models under the same settings.

cs.CV↗

arXiv2022

TorchOpt: An Efficient Library for Differentiable Optimization

Recent years have witnessed the booming of various differentiable optimization algorithms. These algorithms exhibit different execution patterns, and their execution needs massive computational resources that go beyond a single CPU and GPU. Existing differentiable optimization libraries, however, cannot support efficient algorithm development and multi-CPU/GPU execution, making the development of differentiable optimization algorithms often cumbersome and expensive. This paper introduces TorchOpt, a PyTorch-based efficient library for differentiable optimization. TorchOpt provides a unified and expressive differentiable optimization programming abstraction. This abstraction allows users to efficiently declare and analyze various differentiable optimization programs with explicit gradients, implicit gradients, and zero-order gradients. TorchOpt further provides a high-performance distributed execution runtime. This runtime can fully parallelize computation-intensive differentiation operations (e.g. tensor tree flattening) on CPUs / GPUs and automatically distribute computation to distributed devices. Experimental results show that TorchOpt achieves $5.2\times$ training time speedup on an 8-GPU server. TorchOpt is available at: https://github.com/metaopt/torchopt/.

cs.MS↗

arXiv2022

Spatial Temporal Graph Convolution with Graph Structure Self-learning for Early MCI Detection

Graph neural networks (GNNs) have been successfully applied to early mild cognitive impairment (EMCI) detection, with the usage of elaborately designed features constructed from blood oxygen level-dependent (BOLD) time series. However, few works explored the feasibility of using BOLD signals directly as features. Meanwhile, existing GNN-based methods primarily rely on hand-crafted explicit brain topology as the adjacency matrix, which is not optimal and ignores the implicit topological organization of the brain. In this paper, we propose a spatial temporal graph convolutional network with a novel graph structure self-learning mechanism for EMCI detection. The proposed spatial temporal graph convolution block directly exploits BOLD time series as input features, which provides an interesting view for rsfMRI-based preclinical AD diagnosis. Moreover, our model can adaptively learn the optimal topological structure and refine edge weights with the graph structure self-learning mechanism. Results on the Alzheimer's Disease Neuroimaging Initiative (ADNI) database show that our method outperforms state-of-the-art approaches. Biomarkers consistent with previous studies can be extracted from the model, proving the reliable interpretability of our method.

cs.LG↗

arXiv2022

Comparison of two equivariant $η$-forms

In this paper, we first define the equivariant infinitesimal $η$-form, then we compare it with the equivariant $η$-form, modulo exact forms, by a locally computable form. As a consequence, we obtain the singular behavior of the equivariant $η$-form, modulo exact forms, as a function on the acting Lie group. This result extends a result of Goette and it plays an important role in our recent work on the localization of $η$-invariants and on the differential $K$-theory.

math.DG↗

arXiv2022

EnvPool: A Highly Parallel Reinforcement Learning Environment Execution Engine

There has been significant progress in developing reinforcement learning (RL) training systems. Past works such as IMPALA, Apex, Seed RL, Sample Factory, and others, aim to improve the system's overall throughput. In this paper, we aim to address a common bottleneck in the RL training system, i.e., parallel environment execution, which is often the slowest part of the whole system but receives little attention. With a curated design for paralleling RL environments, we have improved the RL environment simulation speed across different hardware setups, ranging from a laptop and a modest workstation, to a high-end machine such as NVIDIA DGX-A100. On a high-end machine, EnvPool achieves one million frames per second for the environment execution on Atari environments and three million frames per second on MuJoCo environments. When running EnvPool on a laptop, the speed is 2.8x that of the Python subprocess. Moreover, great compatibility with existing RL training libraries has been demonstrated in the open-sourced community, including CleanRL, rl_games, DeepMind Acme, etc. Finally, EnvPool allows researchers to iterate their ideas at a much faster pace and has great potential to become the de facto RL environment execution engine. Example runs show that it only takes five minutes to train agents to play Atari Pong and MuJoCo Ant on a laptop. EnvPool is open-sourced at https://github.com/sail-sg/envpool.

cs.LG↗

arXiv2022

Non-equilibrium dynamics of ultracold lattice bosons inside a cavity

We study the non-equilibrium quench dynamics crossing a continuous phase transition between the charge density wave (CDW) and supersolid (SS) phases of a bosonic lattice gas with cavity-mediated interactions. When changing the hopping amplitude in the Hamiltonian as a function of time, we investigate the scaling behavior of the correlation length and vortex density with respect to the quench time and find that there is a threshold of the quench rate separating two distinct scaling regimes. When slowly varying the system below that threshold, we find a power-law scaling as predicted by the Kibble-Zurek mechanism (KZM). While considering fast quench above that threshold, a deviation from the KZM prediction occurs, manifested by a saturation of the defect density. We further show that such distinct scaling behaviors during different dynamic procedures can be understood through comparing the relaxation time and the quench rate.

cond-mat.quant-gas↗

arXiv2022

BOME! Bilevel Optimization Made Easy: A Simple First-Order Approach

Bilevel optimization (BO) is useful for solving a variety of important machine learning problems including but not limited to hyperparameter optimization, meta-learning, continual learning, and reinforcement learning. Conventional BO methods need to differentiate through the low-level optimization process with implicit differentiation, which requires expensive calculations related to the Hessian matrix. There has been a recent quest for first-order methods for BO, but the methods proposed to date tend to be complicated and impractical for large-scale deep learning applications. In this work, we propose a simple first-order BO algorithm that depends only on first-order gradient information, requires no implicit differentiation, and is practical and efficient for large-scale non-convex functions in deep learning. We provide non-asymptotic convergence analysis of the proposed method to stationary points for non-convex objectives and present empirical results that show its superior practical performance.

cs.LG↗

arXiv2022

Current-driven magnetization dynamics and their correlation with magnetization configurations in perpendicularly magnetized tunnel junctions

We study spin-transfer-torque driven magnetization dynamics of a perpendicular magnetic tunnel junction (MTJ) nanopillar. Based on the combination of spin-torque ferromagnetic resonance and microwave spectroscopy techniques, we demonstrate that the free layer (FL) and the weak pinned reference layer (RL) exhibit distinct dynamic behaviors with opposite frequency vs. field dispersion relations. The FL can support a single coherent spin-wave (SW) mode for both parallel and antiparallel configurations, while the RL exhibits spin-wave excitation only for the antiparallel state. These two SW modes corresponding to the FL and RL coexist at an antiparallel state and exhibit a crossover phenomenon of oscillation frequency with increasing the external magnetic field, which could be helpful in the mutual synchronization of auto-oscillations for SW-based neuromorphic computing.

cond-mat.mes-hall↗

arXiv2022

Bismut-Cheeger eta form and higher spectral flow

In this paper, using the equivariant version of the Dai-Zhang higher spectral flow, we generalize the variation formula, embedding formula and the adiabatic limit formula for the Atiyah-Patodi-Singer eta invariants to the equivariant Bismut-Cheeger eta forms for a fiberwise compact Lie group action when the kernels of corresponding fiberwise Dirac operators form equivariant vector bundles.

math.DG↗