arXiv subjects

Li Zhang

Publications and source records attributed to Li Zhang.

At least 415 records · Page 23Linked to original sources

arXiv2025

Brick-Diffusion: Generating Long Videos with Brick-to-Wall Denoising

Recent advances in diffusion models have greatly improved text-driven video generation. However, training models for long video generation demands significant computational power and extensive data, leading most video diffusion models to be limited to a small number of frames. Existing training-free methods that attempt to generate long videos using pre-trained short video diffusion models often struggle with issues such as insufficient motion dynamics and degraded video fidelity. In this paper, we present Brick-Diffusion, a novel, training-free approach capable of generating long videos of arbitrary length. Our method introduces a brick-to-wall denoising strategy, where the latent is denoised in segments, with a stride applied in subsequent iterations. This process mimics the construction of a staggered brick wall, where each brick represents a denoised segment, enabling communication between frames and improving overall video quality. Through quantitative and qualitative evaluations, we demonstrate that Brick-Diffusion outperforms existing baseline methods in generating high-fidelity videos.

cs.CV↗

arXiv2024

An Ordinary Differential Equation Sampler with Stochastic Start for Diffusion Bridge Models

Diffusion bridge models have demonstrated promising performance in conditional image generation tasks, such as image restoration and translation, by initializing the generative process from corrupted images instead of pure Gaussian noise. However, existing diffusion bridge models often rely on Stochastic Differential Equation (SDE) samplers, which result in slower inference speed compared to diffusion models that employ high-order Ordinary Differential Equation (ODE) solvers for acceleration. To mitigate this gap, we propose a high-order ODE sampler with a stochastic start for diffusion bridge models. To overcome the singular behavior of the probability flow ODE (PF-ODE) at the beginning of the reverse process, a posterior sampling approach was introduced at the first reverse step. The sampling was designed to ensure a smooth transition from corrupted images to the generative trajectory while reducing discretization errors. Following this stochastic start, Heun's second-order solver is applied to solve the PF-ODE, achieving high perceptual quality with significantly reduced neural function evaluations (NFEs). Our method is fully compatible with pretrained diffusion bridge models and requires no additional training. Extensive experiments on image restoration and translation tasks, including super-resolution, JPEG restoration, Edges-to-Handbags, and DIODE-Outdoor, demonstrated that our sampler outperforms state-of-the-art methods in both visual quality and Frechet Inception Distance (FID).

cs.CV↗

arXiv2024

Effective Random Test Generation for Deep Learning Compilers

Deep learning compilers help address the difficulties of deploying deep learning models on diverse types of hardware. Testing deep learning compilers is highly crucial, because they are impacting countless AI applications that use them for model optimization and deployment. To test deep learning compilers, random testing, the testing method popularly used for compiler testing practices, faces the challenge of generating semantically valid test inputs, i.e., deep learning models that satisfy the semantic model specifications (in short as semantic specifications). To tackle this challenge, in this paper, we propose a novel approach named Isra, including a domain-specific constraint solver that resolves the constraints from the semantic specifications without backtracking. We implement and apply our approach to three popular real-world deep learning compilers including TVM, Glow, and a commercial compiler named SophGo. The evaluation results show that Isra is more effective than the state-of-the-art approaches and the baseline approaches on constructing valid test inputs for compiler-bug detection, and Isra successfully finds 24 previously unknown bugs in released versions of the three compilers. These results indicate Isra's effectiveness and practical value.

cs.SE↗

arXiv2024

CURE4Rec: A Benchmark for Recommendation Unlearning with Deeper Influence

With increasing privacy concerns in artificial intelligence, regulations have mandated the right to be forgotten, granting individuals the right to withdraw their data from models. Machine unlearning has emerged as a potential solution to enable selective forgetting in models, particularly in recommender systems where historical data contains sensitive user information. Despite recent advances in recommendation unlearning, evaluating unlearning methods comprehensively remains challenging due to the absence of a unified evaluation framework and overlooked aspects of deeper influence, e.g., fairness. To address these gaps, we propose CURE4Rec, the first comprehensive benchmark for recommendation unlearning evaluation. CURE4Rec covers four aspects, i.e., unlearning Completeness, recommendation Utility, unleaRning efficiency, and recommendation fairnEss, under three data selection strategies, i.e., core data, edge data, and random data. Specifically, we consider the deeper influence of unlearning on recommendation fairness and robustness towards data with varying impact levels. We construct multiple datasets with CURE4Rec evaluation and conduct extensive experiments on existing recommendation unlearning methods. Our code is released at https://github.com/xiye7lai/CURE4Rec.

cs.IR↗

arXiv2024

Generalizable Articulated Object Perception with Superpoints

Manipulating articulated objects with robotic arms is challenging due to the complex kinematic structure, which requires precise part segmentation for efficient manipulation. In this work, we introduce a novel superpoint-based perception method designed to improve part segmentation in 3D point clouds of articulated objects. We propose a learnable, part-aware superpoint generation technique that efficiently groups points based on their geometric and semantic similarities, resulting in clearer part boundaries. Furthermore, by leveraging the segmentation capabilities of the 2D foundation model SAM, we identify the centers of pixel regions and select corresponding superpoints as candidate query points. Integrating a query-based transformer decoder further enhances our method's ability to achieve precise part segmentation. Experimental results on the GAPartNet dataset show that our method outperforms existing state-of-the-art approaches in cross-category part segmentation, achieving AP50 scores of 77.9% for seen categories (4.4% improvement) and $39.3\%$ for unseen categories (11.6% improvement), with superior results in 5 out of 9 part categories for seen objects and outperforming all previous methods across all part categories for unseen objects.

cs.CV↗

arXiv2024

Incorporating Feature Pyramid Tokenization and Open Vocabulary Semantic Segmentation

The visual understanding are often approached from 3 granular levels: image, patch and pixel. Visual Tokenization, trained by self-supervised reconstructive learning, compresses visual data by codebook in patch-level with marginal information loss, but the visual tokens does not have semantic meaning. Open Vocabulary semantic segmentation benefits from the evolving Vision-Language models (VLMs) with strong image zero-shot capability, but transferring image-level to pixel-level understanding remains an imminent challenge. In this paper, we treat segmentation as tokenizing pixels and study a united perceptual and semantic token compression for all granular understanding and consequently facilitate open vocabulary semantic segmentation. Referring to the cognitive process of pretrained VLM where the low-level features are progressively composed to high-level semantics, we propose Feature Pyramid Tokenization (PAT) to cluster and represent multi-resolution feature by learnable codebooks and then decode them by joint learning pixel reconstruction and semantic segmentation. We design loosely coupled pixel and semantic learning branches. The pixel branch simulates bottom-up composition and top-down visualization of codebook tokens, while the semantic branch collectively fuse hierarchical codebooks as auxiliary segmentation guidance. Our experiments show that PAT enhances the semantic intuition of VLM feature pyramid, improves performance over the baseline segmentation model and achieves competitive performance on open vocabulary semantic segmentation benchmark. Our model is parameter-efficient for VLM integration and flexible for the independent tokenization. We hope to give inspiration not only on improving segmentation but also on semantic visual token utilization.

cs.CV↗

arXiv2024

Efficacy of Temporal Interference Electrical Stimulation for Spinal Cord Injury Rehabilitation: A Case Series

Spinal cord injury (SCI) is a debilitating condition that often results in significant motor and sensory deficits, impacting the quality of life. Current rehabilitation methods, including physical therapy and electrical stimulation, offer variable outcomes and often require invasive procedures. Temporal interference (TI) stimulation has emerged as a novel, non-invasive neuromodulation technique capable of targeting deep neural structures with precision, providing a promising alternative for SCI rehabilitation. This study explores the efficacy of TI stimulation as a non-invasive approach for improving motor and sensory function in patients with incomplete SCI. Three male patients with incomplete cervical SCI (AIS D) participated in a two-week intervention consisting of 14 sessions of TI stimulation targeting their injury sites. TI stimulation was delivered using frequencies of 1000 Hz and 1040 Hz, with assessments conducted pre- and post-intervention, including motor and sensory evaluations, functional scales, and imaging studies.All participants demonstrated significant improvements in neurological function, motor strength, sensory perception, and functional independence. Neurological levels of injury shifted upward in all cases, with one patient improving from C5 to C7. Graded Redefined Assessment of Strength, Sensibility and Prehension (GRASSP) results shows additional strength, prehension and sensory outcomes obtained for the arm and hand functions of participants. Motor scores (UEMS and LEMS) increased, sensory scores for light touch and pin prick improved, and functional assessments, such as the Berg Balance Scale (BBS) and Barthel Index (BI), showed marked gains. Pain scores also decreased in two participants, highlighting additional therapeutic benefits.

q-bio.NC↗

arXiv2024

OmniDrag: Enabling Motion Control for Omnidirectional Image-to-Video Generation

As virtual reality gains popularity, the demand for controllable creation of immersive and dynamic omnidirectional videos (ODVs) is increasing. While previous text-to-ODV generation methods achieve impressive results, they struggle with content inaccuracies and inconsistencies due to reliance solely on textual inputs. Although recent motion control techniques provide fine-grained control for video generation, directly applying these methods to ODVs often results in spatial distortion and unsatisfactory performance, especially with complex spherical motions. To tackle these challenges, we propose OmniDrag, the first approach enabling both scene- and object-level motion control for accurate, high-quality omnidirectional image-to-video generation. Building on pretrained video diffusion models, we introduce an omnidirectional control module, which is jointly fine-tuned with temporal attention layers to effectively handle complex spherical motion. In addition, we develop a novel spherical motion estimator that accurately extracts motion-control signals and allows users to perform drag-style ODV generation by simply drawing handle and target points. We also present a new dataset, named Move360, addressing the scarcity of ODV data with large scene and object motions. Experiments demonstrate the significant superiority of OmniDrag in achieving holistic scene-level and fine-grained object-level control for ODV generation. The project page is available at https://lwq20020127.github.io/OmniDrag.

cs.CV↗

arXiv2024

Classification and physical characteristics analysis of Fermi-GBM Gamma-ray bursts based on Deep-learning

The classification of Gamma-Ray Bursts has long been an unresolved problem. Early long and short burst classification based on duration is not convincing due to the significant overlap in duration plot, which leads to different views on the classification results. We propose a new classification method based on Convolutional Neural Networks and adopt a sample including 3774 GRBs observed by Fermi-GBM to address the $T_\text{90}$ overlap problem. By using count maps that incorporate both temporal and spectral features as inputs, we successfully classify 593 overlapping events into two distinct categories, thereby refuting the existence of an intermediate GRB class. Additionally, we apply the optimal model to extract features from the count maps and visualized the extracted GRB features using the t-SNE algorithm, discovering two distinct clusters corresponding to S-type and L-type GRBs. To further investigate the physical properties of these two types of bursts, we conduct a time-integrated spectral analysis and discovered significant differences in their spectral characteristics. The analysis also show that most GRBs associated with kilonovae belong to the S-type, while those associated with supernovae are predominantly L-type, with few exceptions. Additionally, the duration characteristics of short bursts with extended emission suggest that they may manifest as either L-type or S-type GRBs. Compared to traditional classification methods (Amati and EHD methods), the new approach demonstrates significant advantages in classification accuracy and robustness without relying on redshift observations. The deep learning classification strategy proposed in this paper provides a more reliable tool for future GRB research.

astro-ph.HE↗

arXiv2024

A Framework For Image Synthesis Using Supervised Contrastive Learning

Text-to-image (T2I) generation aims at producing realistic images corresponding to text descriptions. Generative Adversarial Network (GAN) has proven to be successful in this task. Typical T2I GANs are 2 phase methods that first pretrain an inter-modal representation from aligned image-text pairs and then use GAN to train image generator on that basis. However, such representation ignores the inner-modal semantic correspondence, e.g. the images with same label. The semantic label in priory describes the inherent distribution pattern with underlying cross-image relationships, which is supplement to the text description for understanding the full characteristics of image. In this paper, we propose a framework leveraging both inter- and inner-modal correspondence by label guided supervised contrastive learning. We extend the T2I GANs to two parameter-sharing contrast branches in both pretraining and generation phases. This integration effectively clusters the semantically similar image-text pair representations, thereby fostering the generation of higher-quality images. We demonstrate our framework on four novel T2I GANs by both single-object dataset CUB and multi-object dataset COCO, achieving significant improvements in the Inception Score (IS) and Frechet Inception Distance (FID) metrics of imagegeneration evaluation. Notably, on more complex multi-object COCO, our framework improves FID by 30.1%, 27.3%, 16.2% and 17.1% for AttnGAN, DM-GAN, SSA-GAN and GALIP, respectively. We also validate our superiority by comparing with other label guided T2I GANs. The results affirm the effectiveness and competitiveness of our approach in advancing the state-of-the-art GAN for T2I generation

cs.CV↗

arXiv2024

Explainable CTR Prediction via LLM Reasoning

Recommendation Systems have become integral to modern user experiences, but lack transparency in their decision-making processes. Existing explainable recommendation methods are hindered by reliance on a post-hoc paradigm, wherein explanation generators are trained independently of the underlying recommender models. This paradigm necessitates substantial human effort in data construction and raises concerns about explanation reliability. In this paper, we present ExpCTR, a novel framework that integrates large language model based explanation generation directly into the CTR prediction process. Inspired by recent advances in reinforcement learning, we employ two carefully designed reward mechanisms, LC alignment, which ensures explanations reflect user intentions, and IC alignment, which maintains consistency with traditional ID-based CTR models. Our approach incorporates an efficient training paradigm with LoRA and a three-stage iterative process. ExpCTR circumvents the need for extensive explanation datasets while fostering synergy between CTR prediction and explanation generation. Experimental results demonstrate that ExpCTR significantly enhances both recommendation accuracy and interpretability across three real-world datasets.

cs.IR↗

arXiv2024

Query Performance Explanation through Large Language Model for HTAP Systems

In hybrid transactional and analytical processing (HTAP) systems, users often struggle to understand why query plans from one engine (OLAP or OLTP) perform significantly slower than those from another. Although optimizers provide plan details via the EXPLAIN function, these explanations are frequently too technical for non-experts and offer limited insights into performance differences across engines. To address this, we propose a novel framework that leverages large language models (LLMs) to explain query performance in HTAP systems. Built on Retrieval-Augmented Generation (RAG), our framework constructs a knowledge base that stores historical query executions and expert-curated explanations. To enable efficient retrieval of relevant knowledge, query plans are embedded using a lightweight tree-CNN classifier. This augmentation allows the LLM to generate clear, context-aware explanations of performance differences between engines. Our approach demonstrates the potential of LLMs in hybrid engine systems, paving the way for further advancements in database optimization and user support.

cs.DB↗

arXiv2024

DroidCall: A Dataset for LLM-powered Android Intent Invocation

The growing capabilities of large language models in natural language understanding significantly strengthen existing agentic systems. To power performant on-device mobile agents for better data privacy, we introduce DroidCall, the first training and testing dataset for accurate Android intent invocation. With a highly flexible and reusable data generation pipeline, we constructed 10k samples in DroidCall. Given a task instruction in natural language, small language models such as Qwen2.5-3B and Gemma2-2B fine-tuned with DroidCall can approach or even surpass the capabilities of GPT-4o for accurate Android intent invocation. We also provide an end-to-end Android app equipped with these fine-tuned models to demonstrate the Android intent invocation process. The code and dataset are available at https://github.com/UbiquitousLearning/DroidCall.

cs.AI↗

arXiv2024

A First Look at GPT Apps: Landscape and Vulnerability

Following OpenAI's introduction of GPTs, a surge in GPT apps has led to the launch of dedicated LLM app stores. Nevertheless, given its debut, there is a lack of sufficient understanding of this new ecosystem. To fill this gap, this paper presents a first comprehensive longitudinal (5-month) study of the evolution, landscape, and vulnerability of the emerging LLM app ecosystem, focusing on two GPT app stores: \textit{GPTStore.AI} and the official \textit{OpenAI GPT Store}. Specifically, we develop two automated tools and a TriLevel configuration extraction strategy to efficiently gather metadata (\ie names, creators, descriptions, \etc) and user feedback for all GPT apps across these two stores, as well as configurations (\ie system prompts, knowledge files, and APIs) for the top 10,000 popular apps. Our extensive analysis reveals: (1) the user enthusiasm for GPT apps consistently rises, whereas creator interest plateaus within three months of GPTs' launch; (2) nearly 90\% system prompts can be easily accessed due to widespread failure to secure GPT app configurations, leading to considerable plagiarism and duplication among apps. Our findings highlight the necessity of enhancing the LLM app ecosystem by the app stores, creators, and users.

cs.CR↗

arXiv2024

Dual-view X-ray Detection: Can AI Detect Prohibited Items from Dual-view X-ray Images like Humans?

To detect prohibited items in challenging categories, human inspectors typically rely on images from two distinct views (vertical and side). Can AI detect prohibited items from dual-view X-ray images in the same way humans do? Existing X-ray datasets often suffer from limitations, such as single-view imaging or insufficient sample diversity. To address these gaps, we introduce the Large-scale Dual-view X-ray (LDXray), which consists of 353,646 instances across 12 categories, providing a diverse and comprehensive resource for training and evaluating models. To emulate human intelligence in dual-view detection, we propose the Auxiliary-view Enhanced Network (AENet), a novel detection framework that leverages both the main and auxiliary views of the same object. The main-view pipeline focuses on detecting common categories, while the auxiliary-view pipeline handles more challenging categories using ``expert models" learned from the main view. Extensive experiments on the LDXray dataset demonstrate that the dual-view mechanism significantly enhances detection performance, e.g., achieving improvements of up to 24.7% for the challenging category of umbrellas. Furthermore, our results show that AENet exhibits strong generalization across seven different detection models for X-ray Inspection

cs.CV↗

arXiv2024

Promoting and imaging intervalley coherent order in rhombohedral tetralayer graphene on MoS2

Multilayer rhombohedral graphene (RG) has recently emerged as a new, structurally simple flat-band system, which facilitates the exploration of interaction-driven correlation states with highly ordered electron arrangements. Despite a variety of many-body order behaviors observed in RG by transport measurements, the direct microscopic visualization of such correlated phases in real space is still lacking. Here, we show the discovery of a robust intervalley coherent order, a long-predicted ground state in RG, at 77 K in tetralayer RG placed on MoS2 via imaging atomic-scale spatial reconstruction of wave functions for correlated states. By using scanning tunnelling microscopy, we observe spectroscopic signatures of electronic correlations at partially filled flat bands, where distinct splitting appears. At ~60% and ~70% fillings of the flat bands, we visualize atomic-scale reconstruction patterns with a 3 x 3 supercell on graphene lattice at liquid nitrogen temperature, which indicates a robust intervalley coherent phase of the interacting electrons. The 3 x 3 pattern is observed in MoS2-supported RG, while it is absent in hBN-based ones under the same experimental conditions, suggesting the significant influence of spin-orbit proximity effect. Our results provide microscopic insights into the correlated phases in tetralayer RG and highlight the significant potential for realizing highly accessible collective phenomena through Van der Waals proximity.

cond-mat.mes-hall↗

arXiv2024

Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation

With contributions from the open-source community, a vast amount of instruction tuning (IT) data has emerged. Given the significant resource allocation required for training and evaluating models, it is advantageous to have an efficient method for selecting high-quality IT data. However, existing methods for instruction data selection have limitations such as relying on fragile external APIs, being affected by biases in GPT models, or reducing the diversity of the selected instruction dataset. In this paper, we propose an industrial-friendly, expert-aligned and diversity-preserved instruction data selection method: Clustering and Ranking (CaR). CaR employs a two-step process: first, it ranks instruction pairs using a high-accuracy (84.25%) scoring model aligned with expert preferences; second, it preserves dataset diversity through clustering. In our experiment, CaR efficiently selected a mere 1.96% of Alpaca's IT data, yet the resulting AlpaCaR model surpassed Alpaca's performance by an average of 32.1% in GPT-4 evaluations. Moreover, we find that data selecting is a consistent paradigm whether the pre-trained model is more capable or the model parameters scaling up. Our approach employs compact models with 550M parameters and incurs just 11.2% of the financial outlay of current methods, enhancing its industrial deployability.

cs.CL↗

arXiv2024

Layer-dependent evolution of electronic structures and correlations in rhombohedral multilayer graphene

The recent discovery of superconductivity and magnetism in trilayer rhombohedral graphene (RG) establishes an ideal, untwisted platform to study strong correlation electronic phenomena. However, the correlated effects in multilayer RG have received limited attention, and, particularly, the evolution of the correlations with increasing layer number remains an unresolved question. Here, we show the observation of layer-dependent electronic structures and correlations, under surprising liquid nitrogen temperature, in RG multilayers from 3 to 9 layers by using scanning tunneling microscopy and spectroscopy. We explicitly determine layer-enhanced low-energy flat bands and interlayer coupling strengths. The former directly demonstrates the further flattening of low-energy bands in thicker RG, and the latter indicates the presence of varying interlayer interactions in RG multilayers. Moreover, we find significant splittings of the flat bands, ranging from ~50-80 meV, at 77 K when they are partially filled, indicating the emergence of interaction-induced strongly correlated states. Particularly, the strength of the correlated states is notably enhanced in thicker RG and reaches its maximum in the six-layer, validating directly theoretical predictions and establishing abundant new candidates for strongly correlated systems. Our results provide valuable insights into the layer dependence of the electronic properties in RG and demonstrate it as a suitable system for investigating robust and highly accessible correlated phases.

cond-mat.mes-hall↗