arXiv subjects

Dong Wang

Publications and source records attributed to Dong Wang.

At least 361 records · Page 20Linked to original sources

arXiv2024

Learning Manipulation by Predicting Interaction

Representation learning approaches for robotic manipulation have boomed in recent years. Due to the scarcity of in-domain robot data, prevailing methodologies tend to leverage large-scale human video datasets to extract generalizable features for visuomotor policy learning. Despite the progress achieved, prior endeavors disregard the interactive dynamics that capture behavior patterns and physical interaction during the manipulation process, resulting in an inadequate understanding of the relationship between objects and the environment. To this end, we propose a general pre-training pipeline that learns Manipulation by Predicting the Interaction (MPI) and enhances the visual representation.Given a pair of keyframes representing the initial and final states, along with language instructions, our algorithm predicts the transition frame and detects the interaction object, respectively. These two learning objectives achieve superior comprehension towards "how-to-interact" and "where-to-interact". We conduct a comprehensive evaluation of several challenging robotic tasks.The experimental results demonstrate that MPI exhibits remarkable improvement by 10% to 64% compared with previous state-of-the-art in real-world robot platforms as well as simulation environments. Code and checkpoints are publicly shared at https://github.com/OpenDriveLab/MPI.

cs.RO↗

arXiv2024

Subspace-Configurable Networks

While the deployment of deep learning models on edge devices is increasing, these models often lack robustness when faced with dynamic changes in sensed data. This can be attributed to sensor drift, or variations in the data compared to what was used during offline training due to factors such as specific sensor placement or naturally changing sensing conditions. Hence, achieving the desired robustness necessitates the utilization of either an invariant architecture or specialized training approaches, like data augmentation techniques. Alternatively, input transformations can be treated as a domain shift problem, and solved by post-deployment model adaptation. In this paper, we train a parameterized subspace of configurable networks, where an optimal network for a particular parameter setting is part of this subspace. The obtained subspace is low-dimensional and has a surprisingly simple structure even for complex, non-invertible transformations of the input, leading to an exceptionally high efficiency of subspace-configurable networks (SCNs) when limited storage and computing resources are at stake.

cs.LG↗

arXiv2024

Iterative Thresholding Methods for Longest Minimal Length Partitions

In this paper, we introduce two iterative methods for longest minimal length partition problem, which asks whether the disc (ball) is the set maximizing the total perimeter of the shortest partition that divides the total region into sub-regions with given volume proportions, under a volume constraint. The objective functional is approximated by a short-time heat flow using indicator functions of regions and Gaussian convolution. The problem is then represented as a constrained max-min optimization problem. Auction dynamics is used to find the shortest partition in a fixed region, and threshold dynamics is used to update the region. Numerical experiments in two-dimensional and three-dimensional cases are shown with different numbers of partitions, unequal volume proportions, and different initial shapes. The results of both methods are consistent with the conjecture that the disc in two dimensions and the ball in three dimensions are the solution of the longest minimal length partition problem.

math.NA↗

arXiv2024

Suppression of Coherent Synchrotron Radiation-Induced Emittance Growth in a Multi-Bend Deflection Line

Preserving beam quality during the transportation of high-brightness electron beams is a significant and widespread challenge in the design of modern accelerators. The importance of this issue stems from the fact that the quality of the beam at the accelerator's output is crucial for various applications, including particle colliders, free-electron lasers, and synchrotron radiation sources. The coherent synchrotron radiation (CSR) effect can degrade beam quality when a bunch is deflected. Therefore, developing a structure that effectively suppresses the CSR effect, especially for the short bunches is critically important. This involves protecting both the transverse emittance and the longitudinal profile to ensure the production of a high-quality beam. In this study, an optimization based on the reverse lattice of the beamline is proposed. This method can simplify the optimization process. Based on this approach, the Quadruple Bend Achromat (QBA) deflection structure has been designed and optimized. Then we have derived a general solution to completely suppress the impact of steady-state CSR on the transverse plane for different topologies of QBA. Furthermore, a general condition is proposed for suppressing displacements caused by CSR in sequence drifts for isochronous structures. Simultaneously, QBA has proven to be the simplest structure that can simultaneously suppress both types of CSR effects. Simulation results for bunches with a peak current of up to $3000A$ show almost no change in transverse emittance for a large angle deflection.

physics.acc-ph↗

arXiv2024

Formation and Evolution of Transient Prominence Bubbles Driven by Erupting Mini-filaments

Prominence bubbles, the dark arch-shaped "voids" below quiescent prominences, are generally believed to be caused by the interaction between the prominences and the slowly-emerging or quasi-stable underlying magnetic loops. However, this scenario could not explain some short-lived bubbles with extremely dynamic properties of evolution. Based on high-resolution H$α$ observations, here we propose that bubbles should be classified into two categories according to their dynamic properties: quasi-steady (Type-I) bubbles and transient (Type-II) bubbles. Type-I bubbles could remain relatively stable and last for several hours, indicating the existence of a quasi-stable magnetic topology, while Type-II bubbles grow and collapse quickly within one hour without stability duration, which are usually associated with erupting mini-filaments. Analysis of several typical Type-II bubbles from different views, especially including an on-disk event, reveals that Type-II bubbles quickly appear and expand at a velocity of $\thicksim$5--25 km s$^{-1}$ accompanied by an erupting mini-filament below. The mini-filament's rising velocity is slightly larger than that of the Type-II bubbles' boundary, which will lead to the collision with each other in a short time, subsequent collapse of Type-II bubbles, and formation of a large plume into the above prominence. We also speculate that only if the angle between the axis of the erupting mini-filament and the line-of-sight is large enough, the interaction between the erupting mini-filament and the overlying prominence could trigger a Type-II bubble with a typical arch-shaped but quickly-expanding bright boundary.

astro-ph.SR↗

arXiv2024

Task Aligned Meta-learning based Augmented Graph for Cold-Start Recommendation

The cold-start problem is a long-standing challenge in recommender systems due to the lack of user-item interactions, which significantly hurts the recommendation effect over new users and items. Recently, meta-learning based methods attempt to learn globally shared prior knowledge across all users, which can be rapidly adapted to new users and items with very few interactions. Though with significant performance improvement, the globally shared parameter may lead to local optimum. Besides, they are oblivious to the inherent information and feature interactions existing in the new users and items, which are critical in cold-start scenarios. In this paper, we propose a Task aligned Meta-learning based Augmented Graph (TMAG) to address cold-start recommendation. Specifically, a fine-grained task aligned constructor is proposed to cluster similar users and divide tasks for meta-learning, enabling consistent optimization direction. Besides, an augmented graph neural network with two graph enhanced approaches is designed to alleviate data sparsity and capture the high-order user-item interactions. We validate our approach on three real-world datasets in various cold-start scenarios, showing the superiority of TMAG over state-of-the-art methods for cold-start recommendation.

cs.IR↗

arXiv2024

Other Tokens Matter: Exploring Global and Local Features of Vision Transformers for Object Re-Identification

Object Re-Identification (Re-ID) aims to identify and retrieve specific objects from images captured at different places and times. Recently, object Re-ID has achieved great success with the advances of Vision Transformers (ViT). However, the effects of the global-local relation have not been fully explored in Transformers for object Re-ID. In this work, we first explore the influence of global and local features of ViT and then further propose a novel Global-Local Transformer (GLTrans) for high-performance object Re-ID. We find that the features from last few layers of ViT already have a strong representational ability, and the global and local information can mutually enhance each other. Based on this fact, we propose a Global Aggregation Encoder (GAE) to utilize the class tokens of the last few Transformer layers and learn comprehensive global features effectively. Meanwhile, we propose the Local Multi-layer Fusion (LMF) which leverages both the global cues from GAE and multi-layer patch tokens to explore the discriminative local representations. Extensive experiments demonstrate that our proposed method achieves superior performance on four object Re-ID benchmarks.

cs.CV↗

arXiv2024

Odd dipole screening in disordered matter

Disordered solids, straddling the solid-fluid boundary, lack a comprehensive continuum mechanical description. They exhibit a complex microstructure wherein multiple meta-stable states exist. Deforming disordered solids induces particles rearrangements enabling the system to transition between meta-stable states. A dramatic consequence of these transitions is that quasistatic deformation cycles modify the reference state, facilitating the storage and release of mechanical energy. Here we develop a continuum mechanical theory of disordered solids, which accounts for the absence of a reference state and the lack of conserved potential energy. Our theory, which introduces a new modulus describing non-conservative mechanical screening, reduces to classical elasticity in the absence of screening. We analytically derive predictions for the deformation field for various perturbations and geometries. While our theory applies to general disordered solids, we focus on a two-dimensional disordered granular system and predict accurately the non-affine displacement fields observed in experiments for both small and large deformations, along with the observable vanishing shear modulus. The new proposed moduli satisfy universal relations that are independent of the specific experimental realization. Our work thus forms the basis of an entirely new family of continuum descriptions of the mechanics of disordered solids.

cond-mat.soft↗

arXiv2024

Off-Policy Primal-Dual Safe Reinforcement Learning

Primal-dual safe RL methods commonly perform iterations between the primal update of the policy and the dual update of the Lagrange Multiplier. Such a training paradigm is highly susceptible to the error in cumulative cost estimation since this estimation serves as the key bond connecting the primal and dual update processes. We show that this problem causes significant underestimation of cost when using off-policy methods, leading to the failure to satisfy the safety constraint. To address this issue, we propose conservative policy optimization, which learns a policy in a constraint-satisfying area by considering the uncertainty in cost estimation. This improves constraint satisfaction but also potentially hinders reward maximization. We then introduce local policy convexification to help eliminate such suboptimality by gradually reducing the estimation uncertainty. We provide theoretical interpretations of the joint coupling effect of these two ingredients and further verify them by extensive experiments. Results on benchmark tasks show that our method not only achieves an asymptotic performance comparable to state-of-the-art on-policy methods while using much fewer samples, but also significantly reduces constraint violation during training. Our code is available at https://github.com/ZifanWu/CAL.

cs.LG↗

arXiv2024

Nonreciprocal Photon-Phonon Entanglement in Kerr-Modified Spinning Cavity Magnomechanics

Cavity magnomechanics has shown great potential in studying macroscopic quantum effects, especially for quantum entanglement, which is a key resource for quantum information science. Here we propose to realize magnon mediated nonreciprocal photon-phonon entanglement, which exhibits asymmetry when opposite magnetic or driving fields are respectively applied to the magnons with the Kerr effect or the photons with the Sagnac effect. We find that the mean magnon number can selectively exhibit nonreciprocal linear or nonlinear (bistable) behavior with the strength of the strong driving field on the cavity. Assisted by this driving field, the magnon-phonon coupling is greatly enhanced, leading to the nonreciprocal photon-phonon entanglement via the swapping interaction between the magnons and photons. This nonreciprocal entanglement can be significantly enhanced with the magnon Kerr and Sagnac effects. Given the available parameters, the nonreciprocal photon-phonon entanglement can be preserved at $\sim3$ K, showing remarkable resilience against the bath temperature. The result reveals that our paper holds promise in developing various nonreciprocal devices with both the magnon Kerr and Sagnac effects in cavity magnomechanics.

quant-ph↗

arXiv2024

GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting

In this paper, we introduce \textbf{GS-SLAM} that first utilizes 3D Gaussian representation in the Simultaneous Localization and Mapping (SLAM) system. It facilitates a better balance between efficiency and accuracy. Compared to recent SLAM methods employing neural implicit representations, our method utilizes a real-time differentiable splatting rendering pipeline that offers significant speedup to map optimization and RGB-D rendering. Specifically, we propose an adaptive expansion strategy that adds new or deletes noisy 3D Gaussians in order to efficiently reconstruct new observed scene geometry and improve the mapping of previously observed areas. This strategy is essential to extend 3D Gaussian representation to reconstruct the whole scene rather than synthesize a static object in existing methods. Moreover, in the pose tracking process, an effective coarse-to-fine technique is designed to select reliable 3D Gaussian representations to optimize camera pose, resulting in runtime reduction and robust estimation. Our method achieves competitive performance compared with existing state-of-the-art real-time methods on the Replica, TUM-RGBD datasets. Project page: https://gs-slam.github.io/.

cs.CV↗

arXiv2024

Deployable Reinforcement Learning with Variable Control Rate

Deploying controllers trained with Reinforcement Learning (RL) on real robots can be challenging: RL relies on agents' policies being modeled as Markov Decision Processes (MDPs), which assume an inherently discrete passage of time. The use of MDPs results in that nearly all RL-based control systems employ a fixed-rate control strategy with a period (or time step) typically chosen based on the developer's experience or specific characteristics of the application environment. Unfortunately, the system should be controlled at the highest, worst-case frequency to ensure stability, which can demand significant computational and energy resources and hinder the deployability of the controller on onboard hardware. Adhering to the principles of reactive programming, we surmise that applying control actions only when necessary enables the use of simpler hardware and helps reduce energy consumption. We challenge the fixed frequency assumption by proposing a variant of RL with variable control rate. In this approach, the policy decides the action the agent should take as well as the duration of the time step associated with that action. In our new setting, we expand Soft Actor-Critic (SAC) to compute the optimal policy with a variable control rate, introducing the Soft Elastic Actor-Critic (SEAC) algorithm. We show the efficacy of SEAC through a proof-of-concept simulation driving an agent with Newtonian kinematics. Our experiments show higher average returns, shorter task completion times, and reduced computational resources when compared to fixed rate policies.

cs.RO↗

arXiv2024

Open-Vocabulary Federated Learning with Multimodal Prototyping

Existing federated learning (FL) studies usually assume the training label space and test label space are identical. However, in real-world applications, this assumption is too ideal to be true. A new user could come up with queries that involve data from unseen classes, and such open-vocabulary queries would directly defect such FL systems. Therefore, in this work, we explicitly focus on the under-explored open-vocabulary challenge in FL. That is, for a new user, the global server shall understand her/his query that involves arbitrary unknown classes. To address this problem, we leverage the pre-trained vision-language models (VLMs). In particular, we present a novel adaptation framework tailored for VLMs in the context of FL, named as Federated Multimodal Prototyping (Fed-MP). Fed-MP adaptively aggregates the local model weights based on light-weight client residuals, and makes predictions based on a novel multimodal prototyping mechanism. Fed-MP exploits the knowledge learned from the seen classes, and robustifies the adapted VLM to unseen categories. Our empirical evaluation on various datasets validates the effectiveness of Fed-MP.

cs.CL↗

arXiv2024

Unified Sequence-to-Sequence Learning for Single- and Multi-Modal Visual Object Tracking

In this paper, we introduce a new sequence-to-sequence learning framework for RGB-based and multi-modal object tracking. First, we present SeqTrack for RGB-based tracking. It casts visual tracking as a sequence generation task, forecasting object bounding boxes in an autoregressive manner. This differs from previous trackers, which depend on the design of intricate head networks, such as classification and regression heads. SeqTrack employs a basic encoder-decoder transformer architecture. The encoder utilizes a bidirectional transformer for feature extraction, while the decoder generates bounding box sequences autoregressively using a causal transformer. The loss function is a plain cross-entropy. Second, we introduce SeqTrackv2, a unified sequence-to-sequence framework for multi-modal tracking tasks. Expanding upon SeqTrack, SeqTrackv2 integrates a unified interface for auxiliary modalities and a set of task-prompt tokens to specify the task. This enables it to manage multi-modal tracking tasks using a unified model and parameter set. This sequence learning paradigm not only simplifies the tracking framework, but also showcases superior performance across 14 challenging benchmarks spanning five single- and multi-modal tracking tasks. The code and models are available at https://github.com/chenxin-dlut/SeqTrackv2.

cs.CV↗

arXiv2024

HPL-ESS: Hybrid Pseudo-Labeling for Unsupervised Event-based Semantic Segmentation

Event-based semantic segmentation has gained popularity due to its capability to deal with scenarios under high-speed motion and extreme lighting conditions, which cannot be addressed by conventional RGB cameras. Since it is hard to annotate event data, previous approaches rely on event-to-image reconstruction to obtain pseudo labels for training. However, this will inevitably introduce noise, and learning from noisy pseudo labels, especially when generated from a single source, may reinforce the errors. This drawback is also called confirmation bias in pseudo-labeling. In this paper, we propose a novel hybrid pseudo-labeling framework for unsupervised event-based semantic segmentation, HPL-ESS, to alleviate the influence of noisy pseudo labels. In particular, we first employ a plain unsupervised domain adaptation framework as our baseline, which can generate a set of pseudo labels through self-training. Then, we incorporate offline event-to-image reconstruction into the framework, and obtain another set of pseudo labels by predicting segmentation maps on the reconstructed images. A noisy label learning strategy is designed to mix the two sets of pseudo labels and enhance the quality. Moreover, we propose a soft prototypical alignment module to further improve the consistency of target domain features. Extensive experiments show that our proposed method outperforms existing state-of-the-art methods by a large margin on the DSEC-Semantic dataset (+5.88% accuracy, +10.32% mIoU), which even surpasses several supervised methods.

cs.CV↗

arXiv2024

Evidence-Driven Retrieval Augmented Response Generation for Online Misinformation

The proliferation of online misinformation has posed significant threats to public interest. While numerous online users actively participate in the combat against misinformation, many of such responses can be characterized by the lack of politeness and supporting facts. As a solution, text generation approaches are proposed to automatically produce counter-misinformation responses. Nevertheless, existing methods are often trained end-to-end without leveraging external knowledge, resulting in subpar text quality and excessively repetitive responses. In this paper, we propose retrieval augmented response generation for online misinformation (RARG), which collects supporting evidence from scientific sources and generates counter-misinformation responses based on the evidences. In particular, our RARG consists of two stages: (1) evidence collection, where we design a retrieval pipeline to retrieve and rerank evidence documents using a database comprising over 1M academic articles; (2) response generation, in which we align large language models (LLMs) to generate evidence-based responses via reinforcement learning from human feedback (RLHF). We propose a reward function to maximize the utilization of the retrieved evidence while maintaining the quality of the generated text, which yields polite and factual responses that clearly refutes misinformation. To demonstrate the effectiveness of our method, we study the case of COVID-19 and perform extensive experiments with both in- and cross-domain datasets, where RARG consistently outperforms baselines by generating high-quality counter-misinformation responses.

cs.CL↗

arXiv2024

Computational modeling of the physical features that influence breast cancer invasion into adipose tissue

Breast cancer invasion into adipose tissue strongly influences disease progression and metastasis. The degree of cancer cell invasion into adipose tissue depends on numerous biochemical and physical properties of cancer cells, adipocytes, and other key components of adipose tissue. We model breast cancer invasion into adipose tissue as a physical process by carrying out simulations of active, cohesive spherical particles (cancer cells) invading into confluent packings of deformable polyhedra (adipocytes). We quantify the degree of invasion by calculating the interfacial area $A_t$ between cancer cells and adipocytes. We determine the long-time value of $A_t$ versus the activity and strength of the cohesion between cancer cells, as well as mechanical properties of the adipocytes and extracellular matrix (ECM) in which the adipocytes are embedded. We show that the degree of invasion collapses onto a master curve by plotting it versus a dimensionless energy scale $E_c$, which grows linearly with mean-square fluctuations and persistence time of the cancer cell velocities, is inversely proportional to the pressure of the system, and has an offset that increases with the cancer cell cohesive energy. The condition, $E_c \gg 1$, indicates that cancer cells will invade the adipose tissue, whereas for $E_c \ll 1$, the cancer cells and adipocytes remain demixed. We also show that constraints on adipocyte positions by the ECM decrease $A_t$ relative to that obtained for unconstrained adipocytes. Finally, spatial heterogeneity in structural and mechanical properties of the adipocytes in the presence of ECM impedes invasion relative to adipose tissue with uniform properties.

physics.bio-ph↗

arXiv2024

Implicit Event-RGBD Neural SLAM

Implicit neural SLAM has achieved remarkable progress recently. Nevertheless, existing methods face significant challenges in non-ideal scenarios, such as motion blur or lighting variation, which often leads to issues like convergence failures, localization drifts, and distorted mapping. To address these challenges, we propose EN-SLAM, the first event-RGBD implicit neural SLAM framework, which effectively leverages the high rate and high dynamic range advantages of event data for tracking and mapping. Specifically, EN-SLAM proposes a differentiable CRF (Camera Response Function) rendering technique to generate distinct RGB and event camera data via a shared radiance field, which is optimized by learning a unified implicit representation with the captured event and RGBD supervision. Moreover, based on the temporal difference property of events, we propose a temporal aggregating optimization strategy for the event joint tracking and global bundle adjustment, capitalizing on the consecutive difference constraints of events, significantly enhancing tracking accuracy and robustness. Finally, we construct the simulated dataset DEV-Indoors and real captured dataset DEV-Reals containing 6 scenes, 17 sequences with practical motion blur and lighting changes for evaluations. Experimental results show that our method outperforms the SOTA methods in both tracking ATE and mapping ACC with a real-time 17 FPS in various challenging environments. Project page: https://delinqu.github.io/EN-SLAM.

cs.CV↗