arXiv subjects

Yu Liu

Publications and source records attributed to Yu Liu.

At least 541 records · Page 30Linked to original sources

arXiv2024

AnimateLCM: Computation-Efficient Personalized Style Video Generation without Personalized Video Data

This paper introduces an effective method for computation-efficient personalized style video generation without requiring access to any personalized video data. It reduces the necessary generation time of similarly sized video diffusion models from 25 seconds to around 1 second while maintaining the same level of performance. The method's effectiveness lies in its dual-level decoupling learning approach: 1) separating the learning of video style from video generation acceleration, which allows for personalized style video generation without any personalized style video data, and 2) separating the acceleration of image generation from the acceleration of video motion generation, enhancing training efficiency and mitigating the negative effects of low-quality video data.

cs.CV↗

arXiv2024

Instruction-Guided Visual Masking

Instruction following is crucial in contemporary LLM. However, when extended to multimodal setting, it often suffers from misalignment between specific textual instruction and targeted local region of an image. To achieve more accurate and nuanced multimodal instruction following, we introduce Instruction-guided Visual Masking (IVM), a new versatile visual grounding model that is compatible with diverse multimodal models, such as LMM and robot model. By constructing visual masks for instruction-irrelevant regions, IVM-enhanced multimodal models can effectively focus on task-relevant image regions to better align with complex instructions. Specifically, we design a visual masking data generation pipeline and create an IVM-Mix-1M dataset with 1 million image-instruction pairs. We further introduce a new learning technique, Discriminator Weighted Supervised Learning (DWSL) for preferential IVM training that prioritizes high-quality data samples. Experimental results on generic multimodal tasks such as VQA and embodied robotic control demonstrate the versatility of IVM, which as a plug-and-play tool, significantly boosts the performance of diverse multimodal models, yielding new state-of-the-art results across challenging multimodal benchmarks. Code, model and data are available at https://github.com/2toinf/IVM.

cs.CV↗

arXiv2024

LoD-Loc: Aerial Visual Localization using LoD 3D Map with Neural Wireframe Alignment

We propose a new method named LoD-Loc for visual localization in the air. Unlike existing localization algorithms, LoD-Loc does not rely on complex 3D representations and can estimate the pose of an Unmanned Aerial Vehicle (UAV) using a Level-of-Detail (LoD) 3D map. LoD-Loc mainly achieves this goal by aligning the wireframe derived from the LoD projected model with that predicted by the neural network. Specifically, given a coarse pose provided by the UAV sensor, LoD-Loc hierarchically builds a cost volume for uniformly sampled pose hypotheses to describe pose probability distribution and select a pose with maximum probability. Each cost within this volume measures the degree of line alignment between projected and predicted wireframes. LoD-Loc also devises a 6-DoF pose optimization algorithm to refine the previous result with a differentiable Gaussian-Newton method. As no public dataset exists for the studied problem, we collect two datasets with map levels of LoD3.0 and LoD2.0, along with real RGB queries and ground-truth pose annotations. We benchmark our method and demonstrate that LoD-Loc achieves excellent performance, even surpassing current state-of-the-art methods that use textured 3D models for localization. The code and dataset are available at https://victorzoo.github.io/LoD-Loc.github.io/.

cs.CV↗

arXiv2024

Unsupervised Meta-Learning via Dynamic Head and Heterogeneous Task Construction for Few-Shot Classification

Meta-learning has been widely used in recent years in areas such as few-shot learning and reinforcement learning. However, the questions of why and when it is better than other algorithms in few-shot classification remain to be explored. In this paper, we perform pre-experiments by adjusting the proportion of label noise and the degree of task heterogeneity in the dataset. We use the metric of Singular Vector Canonical Correlation Analysis to quantify the representation stability of the neural network and thus to compare the behavior of meta-learning and classical learning algorithms. We find that benefiting from the bi-level optimization strategy, the meta-learning algorithm has better robustness to label noise and heterogeneous tasks. Based on the above conclusion, we argue a promising future for meta-learning in the unsupervised area, and thus propose DHM-UHT, a dynamic head meta-learning algorithm with unsupervised heterogeneous task construction. The core idea of DHM-UHT is to use DBSCAN and dynamic head to achieve heterogeneous task construction and meta-learn the whole process of unsupervised heterogeneous task construction. On several unsupervised zero-shot and few-shot datasets, DHM-UHT obtains state-of-the-art performance. The code is released at https://github.com/tuantuange/DHM-UHT.

cs.LG↗

arXiv2024

GPU Acceleration of Numerical Atomic Orbitals-Based Density Functional Theory Algorithms within the ABACUS package

With the fast developments of high-performance computing, first-principles methods based on quantum mechanics play a significant role in materials research, serving as fundamental tools for predicting and analyzing various properties of materials. However, the inherent complexity and substantial computational demands of first-principles algorithms, such as density functional theory, limit their use in larger systems. The rapid development of heterogeneous computing, particularly General-Purpose Graphics Processing Units (GPGPUs), has heralded new prospects for enhancing the performance and cost-effectiveness of first-principles algorithms. We utilize GPGPUs to accelerate the electronic structure algorithms in Atomic-orbital Based Ab-initio Computation at USTC (ABACUS), a first-principles computational package based on the linear combination of atomic orbitals (LCAO) basis set. We design algorithms on GPGPU to efficiently construct and diagonalize the Hamiltonian of a given system, including the related force and stress calculations. The effectiveness of this computational acceleration has been demonstrated through calculations on twisted bilayer graphene with the system size up to 10,444 atoms.

cond-mat.mtrl-sci↗

arXiv2024

Data-Driven Parametrization of Molecular Mechanics Force Fields for Expansive Chemical Space Coverage

A force field is a critical component in molecular dynamics simulations for computational drug discovery. It must achieve high accuracy within the constraints of molecular mechanics' (MM) limited functional forms, which offers high computational efficiency. With the rapid expansion of synthetically accessible chemical space, traditional look-up table approaches face significant challenges. In this study, we address this issue using a modern data-driven approach, developing ByteFF, an Amber-compatible force field for drug-like molecules. To create ByteFF, we generated an expansive and highly diverse molecular dataset at the B3LYP-D3(BJ)/DZVP level of theory. This dataset includes 2.4 million optimized molecular fragment geometries with analytical Hessian matrices, along with 3.2 million torsion profiles. We then trained an edge-augmented, symmetry-preserving molecular graph neural network (GNN) on this dataset, employing a carefully optimized training strategy. Our model predicts all bonded and non-bonded MM force field parameters for drug-like molecules simultaneously across a broad chemical space. ByteFF demonstrates state-of-the-art performance on various benchmark datasets, excelling in predicting relaxed geometries, torsional energy profiles, and conformational energies and forces. Its exceptional accuracy and expansive chemical space coverage make ByteFF a valuable tool for multiple stages of computational drug discovery.

cs.LG↗

arXiv2024

Discovering Cyclists' Visual Preferences Through Shared Bike Trajectories and Street View Images Using Inverse Reinforcement Learning

Cycling has gained global popularity for its health benefits and positive urban impacts. To effectively promote cycling, early studies have extensively investigated the relationship between cycling behaviors and environmental factors, especially cyclists' preferences when making route decisions. However, these studies often struggle to comprehensively describe detailed cycling procedures at a large scale due to data limitations, and they tend to overlook the complex nature of cyclists' preferences. To address these issues, we propose a novel framework aimed to quantify and interpret cyclists' complicated visual preferences by leveraging maximum entropy deep inverse reinforcement learning(MEDIRL)and explainable artificial intelligence(XAI). Implemented in Bantian Sub-district, Shenzhen, we adapt MEDIRL model for efficient estimation of cycling reward function by integrating dockless-bike-sharing(DBS) trajectory and street view images(SVIs), which serves as a representation of cyclists' preferences for street visual environments during routing. In addition, we demonstrate the feasibility and reliability of MEDIRL in discovering cyclists' visual preferences. We find that cyclists focus on specific street visual elements when making route decisions, which can be summarized as their attention to safety, street enclosure, and cycling comfort. Further analysis reveals the complex nonlinear effects of street visual elements on cyclists' preferences, offering a cost-effective perspective on streetscapes design. Our proposed framework advances the understanding of individual cycling behaviors and provides actionable insights for urban planners to design bicycle-friendly streetscapes that prioritize cyclists' preferences.

cs.LG↗

arXiv2024

GLRT-Based Metric Learning for Remote Sensing Object Retrieval

With the improvement in the quantity and quality of remote sensing images, content-based remote sensing object retrieval (CBRSOR) has become an increasingly important topic. However, existing CBRSOR methods neglect the utilization of global statistical information during both training and test stages, which leads to the overfitting of neural networks to simple sample pairs of samples during training and suboptimal metric performance. Inspired by the Neyman-Pearson theorem, we propose a generalized likelihood ratio test-based metric learning (GLRTML) approach, which can estimate the relative difficulty of sample pairs by incorporating global data distribution information during training and test phases. This guides the network to focus more on difficult samples during the training process, thereby encourages the network to learn more discriminative feature embeddings. In addition, GLRT is a more effective than traditional metric space due to the utilization of global data distribution information. Accurately estimating the distribution of embeddings is critical for GLRTML. However, in real-world applications, there is often a distribution shift between the training and target domains, which diminishes the effectiveness of directly using the distribution estimated on training data. To address this issue, we propose the clustering pseudo-labels-based fast parameter adaptation (CPLFPA) method. CPLFPA efficiently estimates the distribution of embeddings in the target domain by clustering target domain instances and re-estimating the distribution parameters for GLRTML. We reorganize datasets for CBRSOR tasks based on fine-grained ship remote sensing image slices (FGSRSI-23) and military aircraft recognition (MAR20) datasets. Extensive experiments on these datasets demonstrate the effectiveness of our proposed GLRTML and CPLFPA.

cs.CV↗

arXiv2024

Measurements with Noise: Bayesian Optimization for Co-optimizing Noise and Property Discovery in Automated Experiments

We have developed a Bayesian optimization (BO) workflow that integrates intra-step noise optimization into automated experimental cycles. Traditional BO approaches in automated experiments focus on optimizing experimental trajectories but often overlook the impact of measurement noise on data quality and cost. Our proposed framework simultaneously optimizes both the target property and the associated measurement noise by introducing time as an additional input parameter, thereby balancing the signal-to-noise ratio and experimental duration. Two approaches are explored: a reward-driven noise optimization and a double-optimization acquisition function, both enhancing the efficiency of automated workflows by considering noise and cost within the optimization process. We validate our method through simulations and real-world experiments using Piezoresponse Force Microscopy (PFM), demonstrating the successful optimization of measurement duration and property exploration. Our approach offers a scalable solution for optimizing multiple variables in automated experimental workflows, improving data quality, and reducing resource expenditure in materials science and beyond.

cond-mat.mtrl-sci↗

arXiv2024

Robo-MUTUAL: Robotic Multimodal Task Specification via Unimodal Learning

Multimodal task specification is essential for enhanced robotic performance, where \textit{Cross-modality Alignment} enables the robot to holistically understand complex task instructions. Directly annotating multimodal instructions for model training proves impractical, due to the sparsity of paired multimodal data. In this study, we demonstrate that by leveraging unimodal instructions abundant in real data, we can effectively teach robots to learn multimodal task specifications. First, we endow the robot with strong \textit{Cross-modality Alignment} capabilities, by pretraining a robotic multimodal encoder using extensive out-of-domain data. Then, we employ two Collapse and Corrupt operations to further bridge the remaining modality gap in the learned multimodal representation. This approach projects different modalities of identical task goal as interchangeable representations, thus enabling accurate robotic operations within a well-aligned multimodal latent space. Evaluation across more than 130 tasks and 4000 evaluations on both simulated LIBERO benchmark and real robot platforms showcases the superior capabilities of our proposed framework, demonstrating significant advantage in overcoming data constraints in robotic learning. Website: zh1hao.wang/Robo_MUTUAL

cs.RO↗

arXiv2024

Several functional capacities and Carleson type embeddings of fractional Sobolev sapces on stratified Lie groups

In this paper, we focus on the functional and geometrical aspects of the fractional Sobolev capacity, the Besov capacity and the Riesz capacity on stratified lie groups, respectively. Firstly, we provide a new Carleson characterization of the extension of fractional Sobolev spaces to $L^{q}(\X\times\mathbb{R}_{+},μ)$ with $q\in\mathbb{R}_{+}$ using the fractional heat semigroup and the Caffarelli-Silvestre type extension on stratified Lie groups $\X$. Secondly, a characterization of $ν$ on $\X$ which ensures the continuity of the fractional Sobolev space belonging to $L^{q}(\X,ν)$ is also obtained via taking $t\rightarrow 0$. Finally, with the help of inequalities related to the Besov capacity and its properties, we also obtain a characterization of $ν$ on $\X$ which ensures the continuity of the Besov type space belonging to $L^{q}(\X,ν)$.

math.AP↗

arXiv2024

Sharp asymptotics of disconnection time of large cylinders by simple and biased random walks

We investigate the asymptotic disconnection time of a large discrete cylinder $(\mathbb{Z}/N\mathbb{Z})^{d}\times \mathbb{Z}$, $d\geq 2$, by simple and biased random walks. For simple random walk, we derive a sharp asymptotic lower bound that matches the upper bound from [Sznitman, Ann. Probab., 2009]. For biased walks, we obtain bounds that asymptotically match in the principal order when the bias is not too strong, which greatly improves non-matching bounds from [Windisch, Ann. Appl. Probab., 2008]. As a crucial tool in the proof, we also obtain a "very strong" coupling between the trace of random walk on the cylinder and random interlacements, which is of independent interest.

math.PR↗

arXiv2024

CLSP: High-Fidelity Contrastive Language-State Pre-training for Agent State Representation

With the rapid development of artificial intelligence, multimodal learning has become an important research area. For intelligent agents, the state is a crucial modality to convey precise information alongside common modalities like images, videos, and language. This becomes especially clear with the broad adoption of reinforcement learning and multimodal large language models. Nevertheless, the representation of state modality still lags in development. To this end, we propose a High-Fidelity Contrastive Language-State Pre-training (CLSP) method, which can accurately encode state information into general representations for both reinforcement learning and multimodal large language models. Specifically, we first design a pre-training task based on the classification to train an encoder with coarse-grained information. Next, we construct data pairs of states and language descriptions, utilizing the pre-trained encoder to initialize the CLSP encoder. Then, we deploy contrastive learning to train the CLSP encoder to effectively represent precise state information. Additionally, we enhance the representation of numerical information using the Random Fourier Features (RFF) method for high-fidelity mapping. Extensive experiments demonstrate the superior precision and generalization capabilities of our representation, achieving outstanding results in text-state retrieval, reinforcement learning navigation tasks, and multimodal large language model understanding.

cs.AI↗

arXiv2024

ControlEdit: A MultiModal Local Clothing Image Editing Method

Multimodal clothing image editing refers to the precise adjustment and modification of clothing images using data such as textual descriptions and visual images as control conditions, which effectively improves the work efficiency of designers and reduces the threshold for user design. In this paper, we propose a new image editing method ControlEdit, which transfers clothing image editing to multimodal-guided local inpainting of clothing images. We address the difficulty of collecting real image datasets by leveraging the self-supervised learning approach. Based on this learning approach, we extend the channels of the feature extraction network to ensure consistent clothing image style before and after editing, and we design an inverse latent loss function to achieve soft control over the content of non-edited areas. In addition, we adopt Blended Latent Diffusion as the sampling method to make the editing boundaries transition naturally and enforce consistency of non-edited area content. Extensive experiments demonstrate that ControlEdit surpasses baseline algorithms in both qualitative and quantitative evaluations.

cs.CV↗

arXiv2024

MapCraft: Dissecting and Designing Custom Geo-Infographics

Geographic infographics are increasingly utilized across various domains to convey spatially relevant information effectively. However, creating these infographics typically requires substantial expertise in design and visualization, as well as proficiency with specialized tools, which can deter many potential creators. To address this barrier, our research analyzed and categorized 118 geographic infographics and sketches designed by 8 experts, leading to the development of a structured design space encompassing four critical dimensions: basic map representations, encoding channels, label design and placement, and highlighting techniques. Based on this design space, we developed a web-based authoring tool that allows users to explore and apply these dimensions interactively. The tool's effectiveness was evaluated through a user study involving 12 participants without prior design experience. Participants were first required manually to create geographic infographics using provided datasets, then utilize our authoring tool to recreate and refine their initial drafts. We also conducted pre- and post-use assessments of the participants' knowledge of geographic infographic design. The findings revealed significant improvements in understanding and applying information encoding channels, highlighting techniques, and label design and placement strategies. These results demonstrate the tool's dual capacity to assist users in creating geographics while educating them on key visualization strategies. Our tool, therefore, empowers a broader audience, including those with limited design and visualization backgrounds, to effectively create and utilize geo-infographics.

cs.HC↗

arXiv2024

A Comparative Study of Table Sized Physicalization and Digital Visualization

Data physicalization is gaining popularity in public and educational contexts due to its potential to make abstract data more tangible and understandable. Despite its growing use, there remains a significant gap in our understanding of how large-size physical visualizations compare to their digital counterparts in terms of user comprehension and memory retention. This study aims to bridge this knowledge gap by comparing the effectiveness of visualizing school building history data on large digital screens versus large physical models. Our experimental approach involved 32 participants who were exposed to one of the visualization mediums. We assessed their user experience and immediate understanding of the content, measured through tests after exposure, and evaluated memory retention with follow-up tests seven days later. The results revealed notable differences between the two forms of visualization: physicalization not only facilitated better initial comprehension but also significantly enhanced long-term memory retention. Furthermore, user feedback on usability was also higher on physicalization. These findings underscore the substantial impact of physicalization in improving information comprehension and retention. This study contributes crucial insights into future visualization media selection in educational and public settings.

cs.HC↗

arXiv2024

Experimental Analysis of Freehand Multi-Object Selection Techniques in Virtual Reality Head-Mounted Displays

Object selection is essential in virtual reality (VR) head-mounted displays (HMDs). Prior work mainly focuses on enhancing and evaluating techniques for selecting a single object in VR, leaving a gap in the techniques for multi-object selection, a more complex but common selection scenario. To enable multi-object selection, the interaction technique should support group selection in addition to the default pointing selection mode for acquiring a single target. This composite interaction could be particularly challenging when using freehand gestural input. In this work, we present an empirical comparison of six freehand techniques, which are comprised of three mode-switching gestures (Finger Segment, Multi-Finger, and Wrist Orientation) and two group selection techniques (Cone-casting Selection and Crossing Selection) derived from prior work. Our results demonstrate the performance, user experience, and preference of each technique. The findings derive three design implications that can guide the design of freehand techniques for multi-object selection in VR HMDs.

cs.HC↗

arXiv2024

Relative cluster tilting theory and $τ$-tilting theory

Let $\mathcal C$ be a Krull-Schmidt triangulated category with shift functor $[1]$ and $\mathcal R$ be a rigid subcategory of $\mathcal C$. We are concerned with the mutation of two-term weak $\mathcal R[1]$-cluster tilting subcategories. We show that any almost complete two-term weak $\mathcal R[1]$-cluster tilting subcategory has exactly two completions. Then we apply the results on relative cluster tilting subcategories to the domain of $τ$-tilting theory in functor categories and abelian categories.

math.RT↗