arXiv subjects

Xinyu Liu

Publications and source records attributed to Xinyu Liu.

At least 199 records · Page 11Linked to original sources

arXiv2024

UN-SAM: Universal Prompt-Free Segmentation for Generalized Nuclei Images

In digital pathology, precise nuclei segmentation is pivotal yet challenged by the diversity of tissue types, staining protocols, and imaging conditions. Recently, the segment anything model (SAM) revealed overwhelming performance in natural scenarios and impressive adaptation to medical imaging. Despite these advantages, the reliance of labor-intensive manual annotation as segmentation prompts severely hinders their clinical applicability, especially for nuclei image analysis containing massive cells where dense manual prompts are impractical. To overcome the limitations of current SAM methods while retaining the advantages, we propose the Universal prompt-free SAM framework for Nuclei segmentation (UN-SAM), by providing a fully automated solution with remarkable generalization capabilities. Specifically, to eliminate the labor-intensive requirement of per-nuclei annotations for prompt, we devise a multi-scale Self-Prompt Generation (SPGen) module to revolutionize clinical workflow by automatically generating high-quality mask hints to guide the segmentation tasks. Moreover, to unleash the generalization capability of SAM across a variety of nuclei images, we devise a Domain-adaptive Tuning Encoder (DT-Encoder) to seamlessly harmonize visual features with domain-common and domain-specific knowledge, and further devise a Domain Query-enhanced Decoder (DQ-Decoder) by leveraging learnable domain queries for segmentation decoding in different nuclei domains. Extensive experiments prove that UN-SAM with exceptional performance surpasses state-of-the-arts in nuclei instance and semantic segmentation, especially the generalization capability in zero-shot scenarios. The source code is available at https://github.com/CUHK-AIM-Group/UN-SAM.

eess.IV↗

arXiv2024

S2R-ViT for Multi-Agent Cooperative Perception: Bridging the Gap from Simulation to Reality

Due to the lack of enough real multi-agent data and time-consuming of labeling, existing multi-agent cooperative perception algorithms usually select the simulated sensor data for training and validating. However, the perception performance is degraded when these simulation-trained models are deployed to the real world, due to the significant domain gap between the simulated and real data. In this paper, we propose the first Simulation-to-Reality transfer learning framework for multi-agent cooperative perception using a novel Vision Transformer, named as S2R-ViT, which considers both the Deployment Gap and Feature Gap between simulated and real data. We investigate the effects of these two types of domain gaps and propose a novel uncertainty-aware vision transformer to effectively relief the Deployment Gap and an agent-based feature adaptation module with inter-agent and ego-agent discriminators to reduce the Feature Gap. Our intensive experiments on the public multi-agent cooperative perception datasets OPV2V and V2V4Real demonstrate that the proposed S2R-ViT can effectively bridge the gap from simulation to reality and outperform other methods significantly for point cloud-based 3D object detection.

cs.CV↗

arXiv2024

AdvGPS: Adversarial GPS for Multi-Agent Perception Attack

The multi-agent perception system collects visual data from sensors located on various agents and leverages their relative poses determined by GPS signals to effectively fuse information, mitigating the limitations of single-agent sensing, such as occlusion. However, the precision of GPS signals can be influenced by a range of factors, including wireless transmission and obstructions like buildings. Given the pivotal role of GPS signals in perception fusion and the potential for various interference, it becomes imperative to investigate whether specific GPS signals can easily mislead the multi-agent perception system. To address this concern, we frame the task as an adversarial attack challenge and introduce \textsc{AdvGPS}, a method capable of generating adversarial GPS signals which are also stealthy for individual agents within the system, significantly reducing object detection accuracy. To enhance the success rates of these attacks in a black-box scenario, we introduce three types of statistically sensitive natural discrepancies: appearance-based discrepancy, distribution-based discrepancy, and task-aware discrepancy. Our extensive experiments on the OPV2V dataset demonstrate that these attacks substantially undermine the performance of state-of-the-art methods, showcasing remarkable transferability across different point cloud based 3D detection systems. This alarming revelation underscores the pressing need to address security implications within multi-agent perception systems, thereby underscoring a critical area of research.

cs.CV↗

arXiv2024

Breaking Data Silos: Cross-Domain Learning for Multi-Agent Perception from Independent Private Sources

The diverse agents in multi-agent perception systems may be from different companies. Each company might use the identical classic neural network architecture based encoder for feature extraction. However, the data source to train the various agents is independent and private in each company, leading to the Distribution Gap of different private data for training distinct agents in multi-agent perception system. The data silos by the above Distribution Gap could result in a significant performance decline in multi-agent perception. In this paper, we thoroughly examine the impact of the distribution gap on existing multi-agent perception systems. To break the data silos, we introduce the Feature Distribution-aware Aggregation (FDA) framework for cross-domain learning to mitigate the above Distribution Gap in multi-agent perception. FDA comprises two key components: Learnable Feature Compensation Module and Distribution-aware Statistical Consistency Module, both aimed at enhancing intermediate features to minimize the distribution gap among multi-agent features. Intensive experiments on the public OPV2V and V2XSet datasets underscore FDA's effectiveness in point cloud-based 3D object detection, presenting it as an invaluable augmentation to existing multi-agent perception systems.

cs.CV↗

arXiv2023

Single-Input Polarization-Sensitive Optical Coherence Tomography Through a Catheter

Intravascular polarimetry with catheter-based polarization-sensitive optical coherence tomography (PS-OCT) complements the high-resolution structural tomograms of OCT with morphological contrast available through polarimetry. Its clinical translation has been complicated by the need for modification of conventional OCT hardware to enable polarimetric measurements. Here, we present a signal processing method to reconstruct polarization properties of tissue from measurements with a single input polarization state, bypassing the need for modulation or multiplexing of input states. Our method relies on a polarization symmetry intrinsic to round-trip measurements and uses the residual spectral variation of the polarization states incident on the tissue to avoid measurement ambiguities. We demonstrate depth-resolved birefringence and optic axis orientation maps reconstructed from in-vivo data of human coronary arteries. We validate our method through comparison with conventional dual-input state measurements and find a mean cumulative retardance error of 13.2deg without observable bias. The 95% limit of agreement between depth-resolved birefringence is 2.80 x 10^(-4), which is less than the agreement between two repeat pullbacks of conventional PS-OCT (3.14 x 10^(-4)), indicating that the two methods can be used interchangeably. The hardware simplification arising from using a single input state may be decisive in realizing the potential of polarimetric measurements for assessing coronary atherosclerosis in clinical practice.

physics.optics↗

arXiv2023

Evidence of Indium impurity band in superconducting (Sn,In)Te thin films

Sn1-xInxTe has been synthesized and studied recently as a candidate topological superconductor. Its superconducting critical temperature increases with Indium concentration. However, the role of Indium in altering the normal state band structure and generating superconductivity is not well-understood. Here, we explore this question in Sn1-xInxTe (0<x<0.3) thin films, characterized by magneto-transport, infrared transmission and photoemission spectroscopy measurement. We show that Indium is forming an impurity band below the valence band edge which pins the Fermi energy and effectively generates electron doping. An enhanced density-of-states due to this impurity band leads to the enhancement of superconducting transition temperature measured in multiple previous studies. The existence of the In impurity band and the role of In as a resonant impurity should be more carefully considered when discussing the topological nature of Sn1-xInxTe.

cond-mat.supr-con↗

arXiv2023

Neural-Optic Co-Designed Polarization-Multiplexed Metalens for Compact Computational Spectral Imaging

As the realm of spectral imaging applications extends its reach into the domains of mobile technology and augmented reality, the demands for compact yet high-fidelity systems become increasingly pronounced. Conventional methodologies, exemplified by coded aperture snapshot spectral imaging systems, are significantly limited by their cumbersome physical dimensions and form factors. To address this inherent challenge, diffractive optical elements (DOEs) have been repeatedly employed as a means to mitigate issues related to the bulky nature of these systems. Nonetheless, it's essential to note that the capabilities of DOEs primarily revolve around the modulation of the phase of light. Here, we introduce an end-to-end computational spectral imaging framework based on a polarization-multiplexed metalens. A distinguishing feature of this approach lies in its capacity to simultaneously modulate orthogonal polarization channels. When harnessed in conjunction with a neural network, it facilitates the attainment of high-fidelity spectral reconstruction. Importantly, the framework is intrinsically fully differentiable, a feature that permits the joint optimization of both the metalens structure and the parameters governing the neural network. The experimental results presented herein validate the exceptional spatial-spectral reconstruction performance, underscoring the efficacy of this system in practical, real-world scenarios. This innovative approach transcends the traditional boundaries separating hardware and software in the realm of computational imaging and holds the promise of substantially propelling the miniaturization of spectral imaging systems.

physics.optics↗

arXiv2023

Statistical ensembles in integrable Hamiltonian systems with almost periodic transitions

We study the long-term average evolution of the random ensemble along integrable Hamiltonian systems with time $T$-periodic transitions. More precisely, for any observable $G$, it is demonstrated that the ensemble under $G$ in long time average converges to that over one time period $T$, and that the probability measure induced by the probability density function describing the ensemble at time $t$ weakly converges to the average of the probability measures over time $T$. And we extend the result to almost periodic cases. The key to the proof is based on the {\it {Riemann-Lebesgue lemma in time-average form}} generalized in the paper. %This work contributes to the comprehension of the statistical mechanics of Hamiltonian systems subject to disturbances.

math.DS↗

arXiv2023

Broadband CPW-based impedance-transformed Josephson parametric amplifier

Quantum-limited Josephson parametric amplifiers play a pivotal role in advancing the field of circuit quantum electrodynamics by enabling the fast and high-fidelity measurement of weak microwave signals. Therefore, it is necessary to develop robust parametric amplifiers with low noise, broad bandwidth, and reduced design complexity for microwave detection. However, current broadband parametric amplifiers either have degraded noise performance or rely on complex designs. Here, we present a device based on the broadband impedance-transformed Josephson parametric amplifier (IMPA) that integrates a horn-like coplanar waveguide (CPW) transmission line, which significantly decreases the design and fabrication complexity, while keeping comparable performance. The device shows an instantaneous bandwidth of 700(200) MHz for 15(20) dB gain with an average saturation power of -110 dBm and near quantum-limited added noise. The operating frequency can be tuned over 1.4 GHz using an external flux bias. We further demonstrate the negligible back-action from our device on a transmon qubit. The amplification performance and simplicity of our device promise its wide adaptation in quantum metrology, quantum communication, and quantum information processing.

quant-ph↗

arXiv2023

Deep Transfer Learning for Intelligent Vehicle Perception: a Survey

Deep learning-based intelligent vehicle perception has been developing prominently in recent years to provide a reliable source for motion planning and decision making in autonomous driving. A large number of powerful deep learning-based methods can achieve excellent performance in solving various perception problems of autonomous driving. However, these deep learning methods still have several limitations, for example, the assumption that lab-training (source domain) and real-testing (target domain) data follow the same feature distribution may not be practical in the real world. There is often a dramatic domain gap between them in many real-world cases. As a solution to this challenge, deep transfer learning can handle situations excellently by transferring the knowledge from one domain to another. Deep transfer learning aims to improve task performance in a new domain by leveraging the knowledge of similar tasks learned in another domain before. Nevertheless, there are currently no survey papers on the topic of deep transfer learning for intelligent vehicle perception. To the best of our knowledge, this paper represents the first comprehensive survey on the topic of the deep transfer learning for intelligent vehicle perception. This paper discusses the domain gaps related to the differences of sensor, data, and model for the intelligent vehicle perception. The recent applications, challenges, future researches in intelligent vehicle perception are also explored.

cs.CV↗

arXiv2023

Energy gap of topological surface states in proximity to a magnetic insulator

Topological surface-states can acquire an energy gap when time-reversal symmetry is broken by interfacing with a magnetic insulator. This gap has yet to be measured. Such topological-magnetic insulator heterostructures can host a quantized anomalous Hall effect and can allow the control of the magnetic state of the insulator in a spintronic device. In this work, we observe the energy gap of topological surface-states in proximity to a magnetic insulator using magnetooptical Landau level spectroscopy. We measure Pb1-xSnxSe/EuSe heterostructures grown by molecular beam epitaxy exhibiting a record mobility and low Fermi energy. Through temperature dependent measurements and theoretical calculations, we show this gap is likely due to quantum confinement and conclude that the magnetic proximity effect is weak in this system. This weakness is disadvantageous for the realization of the quantum anomalous Hall effect, but favorable for spintronic devices which require the preservation of spin-momentum locking at the Fermi level.

cond-mat.mes-hall↗

arXiv2023

MBE growth of axion insulator candidate EuIn2As2

The synthesis of thin films of magnetic topological materials is necessary to achieve novel quantized Hall effects and electrodynamic responses. EuIn2As2 is a recently predicted topological axion insulator that has an antiferromagnetic ground state and an inverted band structure, but that has only been synthesized and studied as a single crystal. We report on the synthesis of c-axis oriented EuIn2As2 films on sapphire substrates by molecular beam epitaxy. By carefully tuning the substrate temperature during growth, we stabilize the Zintl phase of EuIn2As2 expected to be topologically non-trivial. The magnetic properties of these films reproduce those seen in single crystals, but their resistivity is enhanced when grown at lower temperatures. We additionally find that the magnetoresistance of EuIn2As2 is negative even up to fields as high as 31T. while it is highly anisotropic at low fields, it becomes nearly isotropic at high magnetic fields above 5T. Overall, the transport characteristics of EuIn2As2 appear similar to those of chalcogenide topological insulators, motivating the development of devices to gate tune the Fermi energy and reveal topological features in quantum transport.

cond-mat.mtrl-sci↗

arXiv2023

Towards Robust Aspect-based Sentiment Analysis through Non-counterfactual Augmentations

While state-of-the-art NLP models have demonstrated excellent performance for aspect based sentiment analysis (ABSA), substantial evidence has been presented on their lack of robustness. This is especially manifested as significant degradation in performance when faced with out-of-distribution data. Recent solutions that rely on counterfactually augmented datasets show promising results, but they are inherently limited because of the lack of access to explicit causal structure. In this paper, we present an alternative approach that relies on non-counterfactual data augmentation. Our proposal instead relies on using noisy, cost-efficient data augmentations that preserve semantics associated with the target aspect. Our approach then relies on modelling invariances between different versions of the data to improve robustness. A comprehensive suite of experiments shows that our proposal significantly improves upon strong pre-trained baselines on both standard and robustness-specific datasets. Our approach further establishes a new state-of-the-art on the ABSA robustness benchmark and transfers well across domains.

cs.CL↗

arXiv2023

Style-transfer based Speech and Audio-visual Scene Understanding for Robot Action Sequence Acquisition from Videos

To realize human-robot collaboration, robots need to execute actions for new tasks according to human instructions given finite prior knowledge. Human experts can share their knowledge of how to perform a task with a robot through multi-modal instructions in their demonstrations, showing a sequence of short-horizon steps to achieve a long-horizon goal. This paper introduces a method for robot action sequence generation from instruction videos using (1) an audio-visual Transformer that converts audio-visual features and instruction speech to a sequence of robot actions called dynamic movement primitives (DMPs) and (2) style-transfer-based training that employs multi-task learning with video captioning and weakly-supervised learning with a semantic classifier to exploit unpaired video-action data. We built a system that accomplishes various cooking actions, where an arm robot executes a DMP sequence acquired from a cooking video using the audio-visual Transformer. Experiments with Epic-Kitchen-100, YouCookII, QuerYD, and in-house instruction video datasets show that the proposed method improves the quality of DMP sequences by 2.3 times the METEOR score obtained with a baseline video-to-action Transformer. The model achieved 32% of the task success rate with the task knowledge of the object.

cs.CL↗

arXiv2023

A photo-mechanical coupling theory for photoisomerization hydrogel considering the distribution state of molecular chains

Owing to the possibility of controlling its specific mechanical behaviors up taken by irradiated by light at particular wavelengths, the photoisomerization hydrogels have a broad range of potential applications. A theory connecting the optical excitation to mechanical behavior is essential to precisely control the photo-mechanical behaviors of the hydrogel. In this work, a photo-mechanical coupling theory is developed to describe the photo-mechanical responses of photoisomerization hydrogels within the framework of finite deformation continuum thermodynamics. In the model, the deformation gradient is decomposed into two parts to effectively model the light induced deformation and the elastic one. To consider the effect of the optical excitation on mechanical behaviors, we first investigate the transporting mechanism of light in hydrogel, as well as the photochemical reaction process; and we then explore the disturbance of light irradiation on the equilibrium of the thermodynamic system of hydrogel, as well as the relationship of conformational entropy of hydrogel network with the photochemical reaction; finally, based on the entropy elasticity theory, we propose a new free energy function of the photosensitive hydrogel to consider the effect of molecular chain distribution evolution on the stiffness of the hydrogel network. With the implementation of the proposed model, we study the photo-mechanical behaviors and mechanical properties of photoisomerization hydrogels. The present research is helpful for understanding the multi-field coupling behaviors of the photosensitive hydrogel, and then providing guidelines for the application of photoisomerization hydrogel.

physics.chem-ph↗

arXiv2023

EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention

Vision transformers have shown great success due to their high model capabilities. However, their remarkable performance is accompanied by heavy computation costs, which makes them unsuitable for real-time applications. In this paper, we propose a family of high-speed vision transformers named EfficientViT. We find that the speed of existing transformer models is commonly bounded by memory inefficient operations, especially the tensor reshaping and element-wise functions in MHSA. Therefore, we design a new building block with a sandwich layout, i.e., using a single memory-bound MHSA between efficient FFN layers, which improves memory efficiency while enhancing channel communication. Moreover, we discover that the attention maps share high similarities across heads, leading to computational redundancy. To address this, we present a cascaded group attention module feeding attention heads with different splits of the full feature, which not only saves computation cost but also improves attention diversity. Comprehensive experiments demonstrate EfficientViT outperforms existing efficient models, striking a good trade-off between speed and accuracy. For instance, our EfficientViT-M5 surpasses MobileNetV3-Large by 1.9% in accuracy, while getting 40.4% and 45.2% higher throughput on Nvidia V100 GPU and Intel Xeon CPU, respectively. Compared to the recent efficient model MobileViT-XXS, EfficientViT-M2 achieves 1.8% superior accuracy, while running 5.8x/3.7x faster on the GPU/CPU, and 7.4x faster when converted to ONNX format. Code and models are available at https://github.com/microsoft/Cream/tree/main/EfficientViT.

cs.CV↗

arXiv2023

An efficient and robust SAV based algorithm for discrete gradient systems arising from optimizations

We propose in this paper a new minimization algorithm based on a slightly modified version of the scalar auxiliary variable (SAV) approach coupled with a relaxation step and an adaptive strategy. It enjoys several distinct advantages over popular gradient based methods: (i) it is unconditionally energy diminishing with a modified energy which is intrinsically related to the original energy, thus no parameter tuning is needed for stability; (ii) it allows the use of large step-sizes which can effectively accelerate the convergence rate. We also present a convergence analysis for some SAV based algorithms, which include our new algorithm without the relaxation step as a special case. We apply our new algorithm to several illustrative and benchmark problems, and compare its performance with several popular gradient based methods. The numerical results indicate that the new algorithm is very robust, and its adaptive version usually converges significantly faster than those popular gradient descent based methods.

math.NA↗

arXiv2023

Instruction-ViT: Multi-Modal Prompts for Instruction Learning in ViT

Prompts have been proven to play a crucial role in large language models, and in recent years, vision models have also been using prompts to improve scalability for multiple downstream tasks. In this paper, we focus on adapting prompt design based on instruction tuning into a visual transformer model for image classification which we called Instruction-ViT. The key idea is to implement multi-modal prompts (text or image prompt) related to category information to guide the fine-tuning of the model. Based on the experiments of several image captionining tasks, the performance and domain adaptability were improved. Our work provided an innovative strategy to fuse multi-modal prompts with better performance and faster adaptability for visual classification models.

cs.CV↗