arXiv subjects

Chang Liu

Publications and source records attributed to Chang Liu.

At least 541 records · Page 30Linked to original sources

arXiv2024

A Novel Framework Using Deep Reinforcement Learning for Join Order Selection

Join order selection is a sub-field of query optimization that aims to find the optimal join order for an SQL query with the minimum cost. The challenge lies in the exponentially growing search space as the number of tables increases, making exhaustive enumeration impractical. Traditional optimizers use static heuristics to prune the search space, but they often fail to adapt to changes or improve based on feedback from the DBMS. Recent research addresses these limitations with Deep Reinforcement Learning (DRL), allowing models to use feedback to dynamically search for better join orders and enhance performance over time. Existing research primarily focuses on capturing join order sequences and their representations at various levels, with limited comparative analysis of reinforcement learning methods. In this paper, we propose GTDD, a novel framework that integrates Graph Neural Networks (GNN), Treestructured Long Short-Term Memory (Tree LSTM), and DuelingDQN. We conduct a series of experiments that demonstrate a clear advantage of GTDD over state-of the-art techniques.

cs.DB↗

arXiv2024

Falcon-UI: Understanding GUI Before Following User Instructions

Pursuing human-like interaction for Graphical User Interface (GUI) agents requires understanding the GUI context and following user instructions. However, existing works typically couple these two aspects and focus more on instruct-following abilities, while ignoring the importance of understanding the GUI context. In this paper, we introduce an instruction-free GUI navigation dataset, termed Insight-UI Dataset, to enhance model comprehension of GUI environments. Insight-UI Dataset is automatically generated from the Common Crawl corpus, simulating various platforms -- including iOS, Android, Windows, and Linux -- across multiple resolutions on 312K domains. Although GUI interactions vary by context, diverse interfaces share common internal patterns, such as clicking an item to view its details. It implies the feasibility of independent GUI operation learning, followed by joint optimization with instruction tuning. Thereby, we develop the GUI agent model Falcon-UI, which is initially pretrained on Insight-UI Dataset and subsequently fine-tuned on Android and Web GUI datasets, including AITW, AITZ, Android Control, and Mind2Web. With 7 billion parameters, Falcon-UI achieves accuracy comparable to the 72 billion-parameter Qwen2VL on AITZ, validating the alignment between GUI context comprehension and agent performance. Our code and dataset will be open-sourced.

cs.CL↗

arXiv2024

High SNR 3D Imaging from Millimeter-scale Thick Tissues to Cellular Dynamics via Structured Illumination Microscopy

Three-dimensional (3D) fluorescence imaging provides a vital approach for study of biological tissues with intricate structures, and optical sectioning structured illumination microscopy (OS-SIM) stands out for its high imaging speed, low phototoxicity and high spatial resolution. However, OS-SIM faces the problem of low signal-to-noise ratio (SNR) when using traditional decoding algorithms, especially in thick tissues. Here we propose a Hilbert-transform decoding and space domain based high-low (HT-SHiLo) algorithm for noise suppression in OS-SIM. We demonstrate HT-SHiLo algorithm can significantly improve the SNR of optical sectioning images at rapid processing speed, and double the imaging depth in thick tissues. With our OS-SIM system, we achieve high quality 3D images of various biological samples including mouse brains, Drosophila clock neurons, organoids, and live cells. We anticipate that this approach will render OS-SIM a powerful technique for research of cellular organelles or thick tissues in 3D morphology.

physics.optics↗

arXiv2024

MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Models

Despite the superior capabilities of Multimodal Large Language Models (MLLMs) across diverse tasks, they still face significant trustworthiness challenges. Yet, current literature on the assessment of trustworthy MLLMs remains limited, lacking a holistic evaluation to offer thorough insights into future improvements. In this work, we establish MultiTrust, the first comprehensive and unified benchmark on the trustworthiness of MLLMs across five primary aspects: truthfulness, safety, robustness, fairness, and privacy. Our benchmark employs a rigorous evaluation strategy that addresses both multimodal risks and cross-modal impacts, encompassing 32 diverse tasks with self-curated datasets. Extensive experiments with 21 modern MLLMs reveal some previously unexplored trustworthiness issues and risks, highlighting the complexities introduced by the multimodality and underscoring the necessity for advanced methodologies to enhance their reliability. For instance, typical proprietary models still struggle with the perception of visually confusing images and are vulnerable to multimodal jailbreaking and adversarial attacks; MLLMs are more inclined to disclose privacy in text and reveal ideological and cultural biases even when paired with irrelevant images in inference, indicating that the multimodality amplifies the internal risks from base LLMs. Additionally, we release a scalable toolbox for standardized trustworthiness research, aiming to facilitate future advancements in this important field. Code and resources are publicly available at: https://multi-trust.github.io/.

cs.CL↗

arXiv2024

Ultra precise determination of Cs($nS_{1/2}$) and Cs($nD_J$) quantum defects for sensing and computing: Evaluation of core contributions

We make absolute frequency measurements of Cs Rydberg transitions, $\vert 6S_{1/2}, F=3 \rangle \rightarrow \vert nS_{1/2}~(n=23\rm{-}90)\rangle$ and $\vert nD_{3/2,5/2}~(n=21\rm{-}90)\rangle$, with an accuracy of less than $ 72\,\rm kHz$. The quantum defect parameters for the measured Rydberg series are the most precise obtained to date. The quantum defect series is terminated at $δ_4$, showing that prior fits requiring higher order quantum defects reflect uncertainties in the observations. The precision of the measured quantum defects allow for the calculation of Rydberg electric-dipole transitions and fine-structure intervals extrapolated from high principal quantum numbers, to rival that of sophisticated many-body relativistic calculations carried out at low Rydberg principal quantum numbers. We quantitatively predict the contributions to the quantum defect parameters from core polarization and core penetration of Cs inner shell electrons. A new value for the ionization energy, consistent across the $ nS_{1/2}$ and $ nD_{3/2,5/2}$ Rydberg series, is reported at $31406.467 751 48 (14)~\rm{cm}^{-1}$.

physics.atom-ph↗

arXiv2024

Rapid Parameter Estimation for Merging Massive Black Hole Binaries Using Continuous Normalizing Flows

Detecting the coalescences of massive black hole binaries (MBHBs) is one of the primary targets for space-based gravitational wave observatories such as LISA, Taiji, and Tianqin. The fast and accurate parameter estimation of merging MBHBs is of great significance for the global fitting of all resolvable sources, as well as the astrophysical interpretation of gravitational wave signals. However, such analyses usually entail significant computational costs. To address these challenges, inspired by the latest progress in generative models, we explore the application of continuous normalizing flows (CNFs) on the parameter estimation of MBHBs. Specifically, we employ linear interpolation and trig interpolation methods to construct transport paths for training CNFs. Additionally, we creatively introduce a parameter transformation method based on the symmetry in the detector's response function. This transformation is integrated within CNFs, allowing us to train the model using a simplified dataset, and then perform parameter estimation on more general data, hence also acting as a crucial factor in improving the training speed. In conclusion, for the first time, within a comprehensive and reasonable parameter range, we have achieved a complete and unbiased 11-dimensional rapid inference for MBHBs in the presence of astrophysical confusion noise using CNFs. In the experiments based on simulated data, our model produces posterior distributions comparable to those obtained by nested sampling.

gr-qc↗

arXiv2024

Advancing Myopia To Holism: Fully Contrastive Language-Image Pre-training

In rapidly evolving field of vision-language models (VLMs), contrastive language-image pre-training (CLIP) has made significant strides, becoming foundation for various downstream tasks. However, relying on one-to-one (image, text) contrastive paradigm to learn alignment from large-scale messy web data, CLIP faces a serious myopic dilemma, resulting in biases towards monotonous short texts and shallow visual expressivity. To overcome these issues, this paper advances CLIP into one novel holistic paradigm, by updating both diverse data and alignment optimization. To obtain colorful data with low cost, we use image-to-text captioning to generate multi-texts for each image, from multiple perspectives, granularities, and hierarchies. Two gadgets are proposed to encourage textual diversity. To match such (image, multi-texts) pairs, we modify the CLIP image encoder into multi-branch, and propose multi-to-multi contrastive optimization for image-text part-to-part matching. As a result, diverse visual embeddings are learned for each image, bringing good interpretability and generalization. Extensive experiments and ablations across over ten benchmarks indicate that our holistic CLIP significantly outperforms existing myopic CLIP, including image-text retrieval, open-vocabulary classification, and dense visual tasks.

cs.CV↗

arXiv2024

Neural refractive index field: Unlocking the Potential of Background-oriented Schlieren Tomography in Volumetric Flow Visualization

Background-oriented Schlieren tomography (BOST) is a prevalent method for visualizing intricate turbulent flows, valued for its ease of implementation and capacity to capture three-dimensional distributions of a multitude of flow parameters. However, the voxel-based meshing scheme leads to significant challenges, such as inadequate spatial resolution, substantial discretization errors, poor noise immunity, and excessive computational costs. This work presents an innovative reconstruction approach termed neural refractive index field (NeRIF) which implicitly represents the flow field with a neural network, which is trained with tailored strategies. Both numerical simulations and experimental demonstrations on turbulent Bunsen flames suggest that our approach can significantly improve the reconstruction accuracy and spatial resolution while concurrently reducing computational expenses. Although showcased in the context of background-oriented schlieren tomography here, the key idea embedded in the NeRIF can be readily adapted to various other tomographic modalities including tomographic absorption spectroscopy and tomographic particle imaging velocimetry, broadening its potential impact across different domains of flow visualization and analysis.

physics.flu-dyn↗

arXiv2024

NeRF Inpainting with Geometric Diffusion Prior and Balanced Score Distillation

Recent advances in NeRF inpainting have leveraged pretrained diffusion models to enhance performance. However, these methods often yield suboptimal results due to their ineffective utilization of 2D diffusion priors. The limitations manifest in two critical aspects: the inadequate capture of geometric information by pretrained diffusion models and the suboptimal guidance provided by existing Score Distillation Sampling (SDS) methods. To address these problems, we introduce GB-NeRF, a novel framework that enhances NeRF inpainting through improved utilization of 2D diffusion priors. Our approach incorporates two key innovations: a fine-tuning strategy that simultaneously learns appearance and geometric priors and a specialized normal distillation loss that integrates these geometric priors into NeRF inpainting. We propose a technique called Balanced Score Distillation (BSD) that surpasses existing methods such as Score Distillation (SDS) and the improved version, Conditional Score Distillation (CSD). BSD offers improved inpainting quality in appearance and geometric aspects. Extensive experiments show that our method provides superior appearance fidelity and geometric consistency compared to existing approaches.

cs.CV↗

arXiv2024

MatchTime: Towards Automatic Soccer Game Commentary Generation

Soccer is a globally popular sport with a vast audience, in this paper, we consider constructing an automatic soccer game commentary model to improve the audiences' viewing experience. In general, we make the following contributions: First, observing the prevalent video-text misalignment in existing datasets, we manually annotate timestamps for 49 matches, establishing a more robust benchmark for soccer game commentary generation, termed as SN-Caption-test-align; Second, we propose a multi-modal temporal alignment pipeline to automatically correct and filter the existing dataset at scale, creating a higher-quality soccer game commentary dataset for training, denoted as MatchTime; Third, based on our curated dataset, we train an automatic commentary generation model, named MatchVoice. Extensive experiments and ablation studies have demonstrated the effectiveness of our alignment pipeline, and training model on the curated dataset achieves state-of-the-art performance for commentary generation, showcasing that better alignment can lead to significant performance improvements in downstream tasks.

cs.CV↗

arXiv2024

Robust State Estimation for Legged Robots with Dual Beta Kalman Filter

Existing state estimation algorithms for legged robots that rely on proprioceptive sensors often overlook foot slippage and leg deformation in the physical world, leading to large estimation errors. To address this limitation, we propose a comprehensive measurement model that accounts for both foot slippage and variable leg length by analyzing the relative motion between foot contact points and the robot's body center. We show that leg length is an observable quantity, meaning that its value can be explicitly inferred by designing an auxiliary filter. To this end, we introduce a dual estimation framework that iteratively employs a parameter filter to estimate the leg length parameters and a state filter to estimate the robot's state. To prevent error accumulation in this iterative framework, we construct a partial measurement model for the parameter filter using the leg static equation. This approach ensures that leg length estimation relies solely on joint torques and foot contact forces, avoiding the influence of state estimation errors on the parameter estimation. Unlike leg length which can be directly estimated, foot slippage cannot be measured directly with the current sensor configuration. However, since foot slippage occurs at a low frequency, it can be treated as outliers in the measurement data. To mitigate the impact of these outliers, we propose the beta Kalman filter (beta KF), which redefines the estimation loss in canonical Kalman filtering using beta divergence. This divergence can assign low weights to outliers in an adaptive manner, thereby enhancing the robustness of the estimation algorithm. These techniques together form the dual beta-Kalman filter (Dual beta KF), a novel algorithm for robust state estimation in legged robots. Experimental results on the Unitree GO2 robot demonstrate that the Dual beta KF significantly outperforms state-of-the-art methods.

cs.RO↗

arXiv2024

StableV2V: Stablizing Shape Consistency in Video-to-Video Editing

Recent advancements of generative AI have significantly promoted content creation and editing, where prevailing studies further extend this exciting progress to video editing. In doing so, these studies mainly transfer the inherent motion patterns from the source videos to the edited ones, where results with inferior consistency to user prompts are often observed, due to the lack of particular alignments between the delivered motions and edited contents. To address this limitation, we present a shape-consistent video editing method, namely StableV2V, in this paper. Our method decomposes the entire editing pipeline into several sequential procedures, where it edits the first video frame, then establishes an alignment between the delivered motions and user prompts, and eventually propagates the edited contents to all other frames based on such alignment. Furthermore, we curate a testing benchmark, namely DAVIS-Edit, for a comprehensive evaluation of video editing, considering various types of prompts and difficulties. Experimental results and analyses illustrate the outperforming performance, visual consistency, and inference efficiency of our method compared to existing state-of-the-art studies.

cs.CV↗

arXiv2024

SAN: Structure-Aware Network for Complex and Long-tailed Chinese Text Recognition

In text recognition, complex glyphs and tail classes have always been factors affecting model performance. Specifically for Chinese text recognition, the lack of shape-awareness can lead to confusion among close complex characters. Since such characters are often tail classes that appear less frequently in the training-set, making it harder for the model to capture its shape information. Hence in this work, we propose a structure-aware network utilizing the hierarchical composition information to improve the recognition performance of complex characters. Implementation-wise, we first propose an auxiliary radical branch and integrate it into the base recognition network as a regularization term, which distills hierarchical composition information into the feature extractor. A Tree-Similarity-based weighting mechanism is then proposed to further utilize the depth information in the hierarchical representation. Experiments demonstrate that the proposed approach can significantly improve the performances of complex characters and tail characters, yielding a better overall performance. Code is available at https://github.com/Levi-ZJY/SAN.

cs.CV↗

arXiv2024

Assemblage: Automatic Binary Dataset Construction for Machine Learning

Binary code is pervasive, and binary analysis is a key task in reverse engineering, malware classification, and vulnerability discovery. Unfortunately, while there exist large corpora of malicious binaries, obtaining high-quality corpora of benign binaries for modern systems has proven challenging (e.g., due to licensing issues). Consequently, machine learning based pipelines for binary analysis utilize either costly commercial corpora (e.g., VirusTotal) or open-source binaries (e.g., coreutils) available in limited quantities. To address these issues, we present Assemblage: an extensible cloud-based distributed system that crawls, configures, and builds Windows PE binaries to obtain high-quality binary corpuses suitable for training state-of-the-art models in binary analysis. We have run Assemblage on AWS over the past year, producing 890k Windows PE and 428k Linux ELF binaries across 29 configurations. Assemblage is designed to be both reproducible and extensible, enabling users to publish "recipes" for their datasets, and facilitating the extraction of a wide array of features. We evaluated Assemblage by using its data to train modern learning-based pipelines for compiler provenance and binary function similarity. Our results illustrate the practical need for robust corpora of high-quality Windows PE binaries in training modern learning-based binary analyses. Assemblage code is open sourced under the MIT license, and the dataset can be downloaded from https://assemblage-dataset.net

cs.CR↗

arXiv2024

Is Function Similarity Over-Engineered? Building a Benchmark

Binary analysis is a core component of many critical security tasks, including reverse engineering, malware analysis, and vulnerability detection. Manual analysis is often time-consuming, but identifying commonly-used or previously-seen functions can reduce the time it takes to understand a new file. However, given the complexity of assembly, and the NP-hard nature of determining function equivalence, this task is extremely difficult. Common approaches often use sophisticated disassembly and decompilation tools, graph analysis, and other expensive pre-processing steps to perform function similarity searches over some corpus. In this work, we identify a number of discrepancies between the current research environment and the underlying application need. To remedy this, we build a new benchmark, REFuSE-Bench, for binary function similarity detection consisting of high-quality datasets and tests that better reflect real-world use cases. In doing so, we address issues like data duplication and accurate labeling, experiment with real malware, and perform the first serious evaluation of ML binary function similarity models on Windows data. Our benchmark reveals that a new, simple basline, one which looks at only the raw bytes of a function, and requires no disassembly or other pre-processing, is able to achieve state-of-the-art performance in multiple settings. Our findings challenge conventional assumptions that complex models with highly-engineered features are being used to their full potential, and demonstrate that simpler approaches can provide significant value.

cs.LG↗

arXiv2024

Multimodal Structure Preservation Learning

When selecting data to build machine learning models in practical applications, factors such as availability, acquisition cost, and discriminatory power are crucial considerations. Different data modalities often capture unique aspects of the underlying phenomenon, making their utilities complementary. On the other hand, some sources of data host structural information that is key to their value. Hence, the utility of one data type can sometimes be enhanced by matching the structure of another. We propose Multimodal Structure Preservation Learning (MSPL) as a novel method of learning data representations that leverages the clustering structure provided by one data modality to enhance the utility of data from another modality. We demonstrate the effectiveness of MSPL in uncovering latent structures in synthetic time series data and recovering clusters from whole genome sequencing and antimicrobial resistance data using mass spectrometry data in support of epidemiology applications. The results show that MSPL can imbue the learned features with external structures and help reap the beneficial synergies occurring across disparate data modalities.

cs.LG↗

arXiv2024

Second harmonic generation induced by gate voltage oscillation in few layer MnBi2Te4

Nonlinear charge transport, such as nonreciprocal longitudinal resistance and nonlinear Hall effect, has attracted considerable interest in probing the symmetries and topological properties of new materials. Recent research has revealed significant nonreciprocal longitudinal resistance and nonlinear Hall effect in MnBi2Te4, an intrinsic magnetic topological insulator, induced by the quantum metric dipole. However, the inconsistent response with charge density and conflicting C3z symmetry requirement necessitate a thorough understanding of factors affecting the nonlinear transport measurement. This study uncovers an experimental factor leading to significant nonlinear transport signals in MnBi2Te4, attributed to gate voltage oscillation from the application of large alternating current. Additionally, a methodology is proposed to suppress this effect by individually grounding the voltage electrodes during second-harmonic measurements. The investigation underscores the critical importance of assessing the impact of gate voltage oscillation before determining the intrinsic nature of nonlinear transport in 2D material devices with an electrically connected operative gate electrode.

cond-mat.mtrl-sci↗

arXiv2024

Evolution of pairing symmetry in FeSe$_{1-x}$S$_x$ as probed by uniaxial-strain tuning of $T_c$

In iron-based superconductors (FeSCs), the interplay between electronic nematicity and superconductivity is essential for understanding the exotic superconducting ground state. In the nematic regime, uniaxial-strain ($\varepsilon$) tuning of the superconducting transition temperature $T_c$ [$ΔT_c(\varepsilon)=α\varepsilon+β\varepsilon^2$] offers a unique approach to investigating the evolution of pairing symmetry if both $s$ and $d$ wave pairing instabilities are relevant. Here, we employ uniaxial strain to tune the $T_c$ of FeSe$_{1-x}$S$_x$, in which both nematicity and superconductivity undergo significant changes with doping. While $T_c$ is usually suppressed quadratically with $\varepsilon$ in optimally doped BaFe$_2$As$_2$, $ΔT_c(\varepsilon)$ in FeSe$_{1-x}$S$_x$ dominated by $ΔT_c(\varepsilon)=β\varepsilon^2$ changes its sign from $β$ < $0$ in FeSe to $β$ > $0$ in FeSe$_{1-x}$S$_x$ ($x\gtrsim0.10$), indicating an evolution of the pairing symmetry from an $s_{\pm}$ state towards an $s+d$ wave state. These findings highlight the $ΔT_c(\varepsilon)$ as a powerful probe for elucidating the superconducting pairing symmetry in the nematic regime of FeSCs and provide new insights into the evolution of pairing symmetry in FeSCs.

cond-mat.supr-con↗