Search arXiv⌕ Search

arXiv subjects

Jie Li

Publications and source records attributed to Jie Li.

At least 199 records · Page 11Linked to original sources
arXiv2025

Kling-Omni Technical Report

We present Kling-Omni, a generalist generative framework designed to synthesize high-fidelity videos directly from multimodal visual language inputs. Adopting an end-to-end perspective, Kling-Omni bridges the functional separation among diverse video generation, editing, and intelligent reasoning tasks, integrating them into a holistic system. Unlike disjointed pipeline approaches, Kling-Omni supports a diverse range of user inputs, including text instructions, reference images, and video contexts, processing them into a unified multimodal representation to deliver cinematic-quality and highly-intelligent video content creation. To support these capabilities, we constructed a comprehensive data system that serves as the foundation for multimodal video creation. The framework is further empowered by efficient large-scale pre-training strategies and infrastructure optimizations for inference. Comprehensive evaluations reveal that Kling-Omni demonstrates exceptional capabilities in in-context generation, reasoning-based editing, and multimodal instruction following. Moving beyond a content creation tool, we believe Kling-Omni is a pivotal advancement toward multimodal world simulators capable of perceiving, reasoning, generating and interacting with the dynamic and complex worlds.

cs.CV↗
arXiv2025

Cooling mechanical motion with polaritons

The strong coupling between light and matter gives rise to polaritons. Further coupling polaritons to phonons leads to the formation of hybrid polaromechanical systems. Recent experiments have achieved the strong coupling between polaritons and phonons in two configurations, namely, the magnon-photon-phonon and exciton-photon-phonon systems, which enables the control of mechanical motion via manipulating polaritons. Here, we present a polaromechanical cooling theory and explicitly show how two polaritons can be used to simultaneously cool two mechanical modes. The unique advantage of our protocol lies in the fact that the continuous tunability of the polariton frequencies over a wide range allows for the cooling of any two mechanical modes with their frequency difference falling within this range. We further discuss how to extend the theory to cool multiple mechanical modes. The protocol is designed for cooling mechanical motion in various emerging polaromechanical platforms, such as magnon-, exciton-, and plasmon-polaromechanical systems, which is the first step towards quantum states generation in these hybrid systems.

quant-ph↗
arXiv2025

Soot Planets instead of Water Worlds

Some low-density exoplanets are thought to be water-rich worlds that formed beyond the snow line of their protoplanetary disc, possibly accreting coequal portions of rock and water. However, the compositions of bodies within the Solar System and the stability of volatile-rich solids in accretionary disks suggest that a planet rich in water should also acquire as much as 40% refractory organic carbon (``soot''). This would reduce the water mass fraction well below 50%, making the composition of these planets similar to those of Solar System comets. Here we show that soot-rich planets, with or without water, can account for the low average densities of exoplanets that were previously attributed to a binary combination of rock and water. Formed in locations beyond the soot and/or snow lines in disks, these planets are likely common in our galaxy and already observed by JWST. The surfaces and interiors of soot-rich planets will be influenced by the chemical and physical properties of carbonaceous phases, and the atmospheres of such planets may contain plentiful methane and other hydrocarbons, with implications for photochemical haze generation and habitability.

astro-ph.EP↗
arXiv2025

Optimal Perturbation Budget Allocation for Data Poisoning in Offline Reinforcement Learning

Offline Reinforcement Learning (RL) enables policy optimization from static datasets but is inherently vulnerable to data poisoning attacks. Existing attack strategies typically rely on locally uniform perturbations, which treat all samples indiscriminately. This approach is inefficient, as it wastes the perturbation budget on low-impact samples, and lacks stealthiness due to significant statistical deviations. In this paper, we propose a novel Global Budget Allocation attack strategy. Leveraging the theoretical insight that a sample's influence on value function convergence is proportional to its Temporal Difference (TD) error, we formulate the attack as a global resource allocation problem. We derive a closed-form solution where perturbation magnitudes are assigned proportional to the TD-error sensitivity under a global L2 constraint. Empirical results on D4RL benchmarks demonstrate that our method significantly outperforms baseline strategies, achieving up to 80% performance degradation with minimal perturbations that evade detection by state-of-the-art statistical and spectral defenses.

cs.LG↗
arXiv2025

Derived Representation Type and Field Extensions

Let $A$ be a finite-dimensional algebra over a field $k$. We define $A$ to be $\mathbf{C}$-dichotomic if it has the dichotomy property of the representation type on complexes of projective $A$-modules. $\mathbf{C}$-dichotomy implies the dichotomy properties of representation type on the levels of homotopy category and derived category. If $k$ admits a finite separable field extension $K/k$ such that $K$ is algebraically closed, the real number field for example, we prove that $A$ is $\mathbf{C}$-dichotomic. As a consequence, the second derived Brauer-Thrall type theorem holds for $A$, i.e., $A$ is either derived discrete or strongly derived unbounded.

math.RT↗
arXiv2025

Algebra extensions and derived-discrete algebras

Let $ϕ\colon A\rightarrow B$ be an algebra extension. We prove that if $ϕ$ is split, the derived-discreteness of $A$ implies the derived-discreteness of $B$; if $ϕ$ is separable and the right $A$-module $B$ is projective, the converse holds. We prove an analogous statement for piecewise hereditary algebras.

math.RT↗
arXiv2025

The derived-discrete algebras over the real numbers

We classify derived-discrete algebras over the real numbers up to Morita equivalence, using the classification of complex derived-discrete algebras in [{\sc D. Vossieck}, {\em The algebras with discrete derived category}, J. Algebra {\bf 243} (2001), 168--176]. To this end, we investigate the quiver presentation of the complexified algebra of a real algebra given by a modulated quiver and an admissible ideal.

math.RT↗
arXiv2025

Metasurface-based all-optical diffractive convolutional neural networks

The escalating energy demands and parallel-processing bottlenecks of electronic neural networks underscore the need for alternative computing paradigms. Optical neural networks, capitalizing on the inherent parallelism and speed of light propagation, present a compelling solution. Nevertheless, physically realizing convolutional neural network (CNN) components all-optically remains a significant challenge. To this end, we propose a metasurface-based all-optical diffractive convolutional neural network (MAODCNN) for computer vision tasks. This architecture synergistically integrates metasurface-based optical convolutional layers, which perform parallel convolution on the optical field, with cascaded diffractive neural networks acting as all-optical decoders. This co-design facilitates layer-wise feature extraction and optimization directly within the optical domain. Numerical simulations confirm that the fusion of convolutional and diffractive layers markedly enhances classification accuracy, a performance that scales with the number of diffractive layers. The MAODCNN framework establishes a viable foundation for practical all-optical CNNs, paving the way for high-efficiency, low-power optical computing in advanced pattern recognition.

physics.optics↗
arXiv2025

GigaBrain-0: A World Model-Powered Vision-Language-Action Model

Training Vision-Language-Action (VLA) models for generalist robots typically requires large-scale real-world robot data, which is expensive and time-consuming to collect. The inefficiency of physical data collection severely limits the scalability, and generalization capacity of current VLA systems. To address this challenge, we introduce GigaBrain-0, a novel VLA foundation model empowered by world model-generated data (e.g., video generation, real2real transfer, human transfer, view transfer, sim2real transfer data). By leveraging world models to generate diverse data at scale, GigaBrain-0 significantly reduces reliance on real robot data while improving cross-task generalization. Our approach further improves policy robustness through RGBD input modeling and embodied Chain-of-Thought (CoT) supervision, enabling the model to reason about spatial geometry, object states, and long-horizon dependencies during task execution. This leads to substantial gains in real-world performance on dexterous, long-horizon, and mobile manipulation tasks. Extensive experiments demonstrate that GigaBrain-0 achieves superior generalization across variations in appearances (e.g., textures, colors), object placements, and camera viewpoints. Additionally, we present GigaBrain-0-Small, an optimized lightweight variant designed to run efficiently on devices such as the NVIDIA Jetson AGX Orin.

cs.RO↗
arXiv2025

Highly Anisotropic Charge Dynamics and Spectral Weight Redistribution in the Trilayer Nickelate La$_{4}$Ni$_{3}$O$_{10}$

We study the $ab$-plane and $c$-axis charge dynamics of La$_{4}$Ni$_{3}$O$_{10}$ using optical spectroscopy. While a pronounced Drude profile, i.e. metallic response, is observed in the $ab$-plane optical conductivity $σ_{1}^{ab}(ω)$, the $c$-axis optical spectra $σ_{1}^{c}(ω)$ exhibit semiconducting behavior. The zero-frequency extrapolation of the optical conductivity $σ_{1}(ω\rightarrow 0) \equiv 1/ρ_{\text{dc}}$ gives a resistivity anisotropy of $ρ_{c}/ρ_{ab} \simeq 366$ at 300~K for La$_{4}$Ni$_{3}$O$_{10}$, which is much larger than the values in iron-based superconductors but comparable to those in high-$T_{c}$ cuprates. The interband response is also highly anisotropic, showing salient orbital selectivity for light polarized in the $ab$ plane and along the $c$ axis. The interband-transition peaks in both $σ_{1}^{ab}(ω)$ and $σ_{1}^{c}(ω)$ are located at lower energies compared to density-functional-theory predictions, signifying considerable electronic correlations. By investigating the spectral weight transfer, we find that in the pristine phase, Coulomb correlations have a marked impact on the charge dynamics of \LNO, whereas in the density-wave state, a gap opens with the Ni-$d_{z^{2}}$ orbital being involved.

cond-mat.supr-con↗
arXiv2025

Shift-Window Meets Dual Attention: A Multi-Model Architecture for Specular Highlight Removal

Inevitable specular highlights in practical environments severely impair the visual performance, thus degrading the task effectiveness and efficiency. Although there exist considerable methods that focus on local information from convolutional neural network models or global information from transformer models, the single-type model falls into a modeling dilemma between local fine-grained details and global long-range dependencies, thus deteriorating for specular highlights with different scales. Therefore, to accommodate specular highlights of all scales, we propose a multi-model architecture for specular highlight removal (MM-SHR) that effectively captures fine-grained features in highlight regions and models long-range dependencies between highlight and highlight-free areas. Specifically, we employ convolution operations to extract local details in the shallow layers of MM-SHR, and utilize the attention mechanism to capture global features in the deep layers, ensuring both operation efficiency and removal accuracy. To model long-range dependencies without compromising computational complexity, we utilize a coarse-to-fine manner and propose Omni-Directional Attention Integration Block(OAIBlock) and Adaptive Region-Aware Hybrid-Domain Dual Attention Convolutional Network(HDDAConv) , which leverage omni-directiona pixel-shifting and window-dividing operations at the raw features to achieve specular highlight removal. Extensive experimental results on three benchmark tasks and six types of surface materials demonstrate that MM-SHR outperforms state-of-the-art methods in both accuracy and efficiency for specular highlight removal. The implementation will be made publicly available at https://github.com/Htcicv/MM-SHR.

cs.CV↗
arXiv2025

287,872 Supermassive Black Holes Masses: Deep Learning Approaching Reverberation Mapping Accuracy

We present a population-scale catalogue of 287,872 supermassive black hole masses with high accuracy. Using a deep encoder-decoder network trained on optical spectra with reverberation-mapping (RM) based labels of 849 quasars and applied to all SDSS quasars up to $z=4$, our method achieves a root-mean-square error of $0.058$\,dex, a relative uncertainty of $\approx 14\%$, and coefficient of determination $R^{2}\approx0.91$ with respect to RM-based masses, far surpassing traditional single-line virial estimators. Notably, the high accuracy is maintained for both low ($<10^{7.5}\,M_\odot$) and high ($>10^{9}\,M_\odot$) mass quasars, where empirical relations are unreliable.

astro-ph.GA↗
arXiv2025

Contextual Image Attack: How Visual Context Exposes Multimodal Safety Vulnerabilities

While Multimodal Large Language Models (MLLMs) show remarkable capabilities, their safety alignments are susceptible to jailbreak attacks. Existing attack methods typically focus on text-image interplay, treating the visual modality as a secondary prompt. This approach underutilizes the unique potential of images to carry complex, contextual information. To address this gap, we propose a new image-centric attack method, Contextual Image Attack (CIA), which employs a multi-agent system to subtly embeds harmful queries into seemingly benign visual contexts using four distinct visualization strategies. To further enhance the attack's efficacy, the system incorporate contextual element enhancement and automatic toxicity obfuscation techniques. Experimental results on the MMSafetyBench-tiny dataset show that CIA achieves high toxicity scores of 4.73 and 4.83 against the GPT-4o and Qwen2.5-VL-72B models, respectively, with Attack Success Rates (ASR) reaching 86.31\% and 91.07\%. Our method significantly outperforms prior work, demonstrating that the visual modality itself is a potent vector for jailbreaking advanced MLLMs.

cs.CV↗
arXiv2025

TokenPowerBench: Benchmarking the Power Consumption of LLM Inference

Large language model (LLM) services now answer billions of queries per day, and industry reports show that inference, not training, accounts for more than 90% of total power consumption. However, existing benchmarks focus on either training/fine-tuning or performance of inference and provide little support for power consumption measurement and analysis of inference. We introduce TokenPowerBench, the first lightweight and extensible benchmark designed for LLM-inference power consumption studies. The benchmark combines (i) a declarative configuration interface covering model choice, prompt set, and inference engine, (ii) a measurement layer that captures GPU-, node-, and system-level power without specialized power meters, and (iii) a phase-aligned metrics pipeline that attributes energy to the prefill and decode stages of every request. These elements make it straight-forward to explore the power consumed by an LLM inference run; furthermore, by varying batch size, context length, parallelism strategy and quantization, users can quickly assess how each setting affects joules per token and other energy-efficiency metrics. We evaluate TokenPowerBench on four of the most widely used model series (Llama, Falcon, Qwen, and Mistral). Our experiments cover from 1 billion parameters up to the frontier-scale Llama3-405B model. Furthermore, we release TokenPowerBench as open source to help users to measure power consumption, forecast operating expenses, and meet sustainability targets when deploying LLM services.

cs.LG↗
arXiv2025

Dual instability of superconductivity from oxygen defects in La$_3$Ni$_2$O$_{7+δ}$

We uncover a dual mechanism by which oxygen defects suppress superconductivity in the bilayer nickelate La$_3$Ni$_2$O$_{7+δ}$ using density functional theory, dynamical mean-field theory, and functional renormalization group analysis. Apical vacancies and interbilayer interstitials emerge as the dominant low-energy defect species and are further stabilized by orthorhombic domain walls. These two defect classes drive the electronic structure in opposing directions. Vacancy-induced disorder generates local magnetic moments and promotes Anderson localization at moderate concentrations, whereas periodic interstitial ordering yields a coherent but weakly correlated metallic background that fails to support superconductivity. These findings highlight the decisive role of oxygen defects in shaping the superconducting and provide microscopic guidance for improving superconductivity through controlled defect engineering.

cond-mat.supr-con↗
arXiv2025

Shrinking the Teacher: An Adaptive Teaching Paradigm for Asymmetric EEG-Vision Alignment

Decoding visual features from EEG signals is a central challenge in neuroscience, with cross-modal alignment as the dominant approach. We argue that the relationship between visual and brain modalities is fundamentally asymmetric, characterized by two critical gaps: a Fidelity Gap (stemming from EEG's inherent noise and signal degradation, vs. vision's high-fidelity features) and a Semantic Gap (arising from EEG's shallow conceptual representation, vs. vision's rich semantic depth). Previous methods often overlook this asymmetry, forcing alignment between the two modalities as if they were equal partners and thereby leading to poor generalization. To address this, we propose the adaptive teaching paradigm. This paradigm empowers the ``teacher" modality (vision) to dynamically shrink and adjust its knowledge structure under task guidance, tailoring its semantically dense features to match the ``student" modality (EEG)'s capacity. We implement this paradigm with the ShrinkAdapter, a simple yet effective module featuring a residual-free design and a bottleneck structure. Through extensive experiments, we validate the underlying rationale and effectiveness of our paradigm. Our method achieves a top-1 accuracy of 60.2\% on the zero-shot brain-to-image retrieval task, surpassing previous state-of-the-art methods by a margin of 9.8\%. Our work introduces a new perspective for asymmetric alignment: the teacher must shrink and adapt to bridge the vision-brain gap.

cs.CV↗
arXiv2025

DIFFA: Large Language Diffusion Models Can Listen and Understand

Recent advances in large language models (LLMs) have shown remarkable capabilities across textual and multimodal domains. In parallel, diffusion-based language models have emerged as a promising alternative to the autoregressive paradigm, offering improved controllability, bidirectional context modeling, and robust generation. However, their application to the audio modality remains underexplored. In this work, we introduce \textbf{DIFFA}, the first diffusion-based large audio-language model designed to perform spoken language understanding. DIFFA integrates a frozen diffusion language model with a lightweight dual-adapter architecture that bridges speech understanding and natural language reasoning. We employ a two-stage training pipeline: first, aligning semantic representations via an ASR objective; then, learning instruction-following abilities through synthetic audio-caption pairs automatically generated by prompting LLMs. Despite being trained on only 960 hours of ASR and 127 hours of synthetic instruction data, DIFFA demonstrates competitive performance on major benchmarks, including MMSU, MMAU, and VoiceBench, outperforming several autoregressive open-source baselines. Our results reveal the potential of diffusion-based language models for efficient and scalable audio understanding, opening a new direction for speech-driven AI. Our code will be available at https://github.com/NKU-HLT/DIFFA.git.

cs.SD↗