arXiv subjects

Hao Zhang

Publications and source records attributed to Hao Zhang.

At least 559 records · Page 31Linked to original sources

arXiv2025

Nonlinear Shaping in the Picosecond Gap

Lightwave pulse shaping in the picosecond regime has remained unaddressed because it resides beyond the limits of state-of-the-art techniques, either due to its inherently narrow spectral content or fundamental speed limitations in electronic devices. The so-called picosecond shaping gap hampers progress in all areas correlated with time-modulated light-matter interactions, such as photoelectronics, health and medical technologies, and energy and material sciences. We report on a novel nonlinear method to simultaneously frequency-convert and adaptably shape the envelope of light wavepackets in the picosecond regime by balancing spectral engineering and nonlinear conversion in solid-state nonlinear media, without requiring active devices. We capture computationally the versatility of this methodology across a diverse set of nonlinear conversion chains and initial conditions. We also provide experimental evidence of this framework producing picosecond-shaped, ultra-narrowband, near-transform limited light pulses from broadband, femtosecond input pulses, paving the way toward programmable lightwave shaping at GHz-to-THz frequencies.

physics.optics↗

arXiv2025

ArcPro: Architectural Programs for Structured 3D Abstraction of Sparse Points

We introduce ArcPro, a novel learning framework built on architectural programs to recover structured 3D abstractions from highly sparse and low-quality point clouds. Specifically, we design a domain-specific language (DSL) to hierarchically represent building structures as a program, which can be efficiently converted into a mesh. We bridge feedforward and inverse procedural modeling by using a feedforward process for training data synthesis, allowing the network to make reverse predictions. We train an encoder-decoder on the points-program pairs to establish a mapping from unstructured point clouds to architectural programs, where a 3D convolutional encoder extracts point cloud features and a transformer decoder autoregressively predicts the programs in a tokenized form. Inference by our method is highly efficient and produces plausible and faithful 3D abstractions. Comprehensive experiments demonstrate that ArcPro outperforms both traditional architectural proxy reconstruction and learning-based abstraction methods. We further explore its potential to work with multi-view image and natural language inputs.

cs.GR↗

arXiv2025

Coordinated Multi-Robot Navigation with Formation Adaptation

Coordinated multi-robot navigation is an essential ability for a team of robots operating in diverse environments. Robot teams often need to maintain specific formations, such as wedge formations, to enhance visibility, positioning, and efficiency during fast movement. However, complex environments such as narrow corridors challenge rigid team formations, which makes effective formation control difficult in real-world environments. To address this challenge, we introduce a novel Adaptive Formation with Oscillation Reduction (AFOR) approach to improve coordinated multi-robot navigation. We develop AFOR under the theoretical framework of hierarchical learning and integrate a spring-damper model with hierarchical learning to enable both team coordination and individual robot control. At the upper level, a graph neural network facilitates formation adaptation and information sharing among the robots. At the lower level, reinforcement learning enables each robot to navigate and avoid obstacles while maintaining the formations. We conducted extensive experiments using Gazebo in the Robot Operating System (ROS), a high-fidelity Unity3D simulator with ROS, and real robot teams. Results demonstrate that AFOR enables smooth navigation with formation adaptation in complex scenarios and outperforms previous methods. More details of this work are provided on the project website: https://hcrlab.gitlab.io/project/afor.

cs.RO↗

arXiv2025

RAPID: Efficient Retrieval-Augmented Long Text Generation with Writing Planning and Information Discovery

Generating knowledge-intensive and comprehensive long texts, such as encyclopedia articles, remains significant challenges for Large Language Models. It requires not only the precise integration of facts but also the maintenance of thematic coherence throughout the article. Existing methods, such as direct generation and multi-agent discussion, often struggle with issues like hallucinations, topic incoherence, and significant latency. To address these challenges, we propose RAPID, an efficient retrieval-augmented long text generation framework. RAPID consists of three main modules: (1) Retrieval-augmented preliminary outline generation to reduce hallucinations, (2) Attribute-constrained search for efficient information discovery, (3) Plan-guided article generation for enhanced coherence. Extensive experiments on our newly compiled benchmark dataset, FreshWiki-2024, demonstrate that RAPID significantly outperforms state-of-the-art methods across a wide range of evaluation metrics (e.g. long-text generation, outline quality, latency, etc). Our work provides a robust and efficient solution to the challenges of automated long-text generation.

cs.CL↗

arXiv2025

Monitoring AGNs with H$β$ Asymmetry. V. Long-term Variation and Evolution of the Broad H$β$ Emission-Line Profiles

The physical origins of the diverse emission-line asymmetries observed in the spectra of active galactic nuclei (AGNs) remain incompletely understood. Monitoring the temporal variations of line profiles offers a promising approach to investigating the underlying physics. In this study, we present an analysis of the broad H$β$ emission line profiles of eight AGNs observed from the end of 2016 to May 2023 as part of the reverberation mapping campaign titled "Monitoring AGNs with H$β$ Asymmetry" (MAHA), utilizing data obtained from the Wyoming Infrared Observatory (WIRO) 2.3-meter telescope. We measure the temporal variations of line asymmetry, width, and central velocity shift for the eight objects. Our findings reveal that the variation in asymmetry is positively correlated with H$β$ flux in five of the eight objects, while the remaining objects exhibit negative or complex correlations. Furthermore, we observe anti-correlations between line width and H$β$ flux for most objects, indicating the presence of the "breathing" phenomenon in their H$β$ emission lines. In contrast, two objects demonstrate an "anti-breathing" phenomenon or complex behavior. We discuss the physical origins of the temporal variations in line profiles and propose the possibility of decomposing the variations in H$β$ asymmetry and width into components: one that corresponds to short-term variations in H$β$ flux and another that reflects long-term variations in continuum light curves, perhaps driven by radiation pressure.

astro-ph.GA↗

arXiv2025

Muon is Scalable for LLM Training

Recently, the Muon optimizer based on matrix orthogonalization has demonstrated strong results in training small-scale language models, but the scalability to larger models has not been proven. We identify two crucial techniques for scaling up Muon: (1) adding weight decay and (2) carefully adjusting the per-parameter update scale. These techniques allow Muon to work out-of-the-box on large-scale training without the need of hyper-parameter tuning. Scaling law experiments indicate that Muon achieves $\sim\!2\times$ computational efficiency compared to AdamW with compute optimal training. Based on these improvements, we introduce Moonlight, a 3B/16B-parameter Mixture-of-Expert (MoE) model trained with 5.7T tokens using Muon. Our model improves the current Pareto frontier, achieving better performance with much fewer training FLOPs compared to prior models. We open-source our distributed Muon implementation that is memory optimal and communication efficient. We also release the pretrained, instruction-tuned, and intermediate checkpoints to support future research.

cs.LG↗

arXiv2025

Reverse Modeling in Large Language Models

Humans are accustomed to reading and writing in a forward manner, and this natural bias extends to text understanding in auto-regressive large language models (LLMs). This paper investigates whether LLMs, like humans, struggle with reverse modeling, specifically with reversed text inputs. We found that publicly available pre-trained LLMs cannot understand such inputs. However, LLMs trained from scratch with both forward and reverse texts can understand them equally well during inference across multiple languages. Our case study shows that different-content texts result in different losses if input (to LLMs) in different directions -- some get lower losses for forward while some for reverse. This leads us to a simple and nice solution for data selection based on the loss differences between forward and reverse directions. Using our selected data in continued pretraining can boost LLMs' performance by a large margin across different language understanding benchmarks.

cs.CL↗

arXiv2025

Efficient-vDiT: Efficient Video Diffusion Transformers With Attention Tile

Despite the promise of synthesizing high-fidelity videos, Diffusion Transformers (DiTs) with 3D full attention suffer from expensive inference due to the complexity of attention computation and numerous sampling steps. For example, the popular Open-Sora-Plan model consumes more than 9 minutes for generating a single video of 29 frames. This paper addresses the inefficiency issue from two aspects: 1) Prune the 3D full attention based on the redundancy within video data; We identify a prevalent tile-style repetitive pattern in the 3D attention maps for video data, and advocate a new family of sparse 3D attention that holds a linear complexity w.r.t. the number of video frames. 2) Shorten the sampling process by adopting existing multi-step consistency distillation; We split the entire sampling trajectory into several segments and perform consistency distillation within each one to activate few-step generation capacities. We further devise a three-stage training pipeline to conjoin the low-complexity attention and few-step generation capacities. Notably, with 0.1% pretraining data, we turn the Open-Sora-Plan-1.2 model into an efficient one that is 7.4x -7.8x faster for 29 and 93 frames 720p video generation with a marginal performance trade-off in VBench. In addition, we demonstrate that our approach is amenable to distributed inference, achieving an additional 3.91x speedup when running on 4 GPUs with sequence parallelism.

cs.CV↗

arXiv2025

Bandwidth-Adaptive Spatiotemporal Correspondence Identification for Collaborative Perception

Correspondence identification (CoID) is an essential capability in multi-robot collaborative perception, which enables a group of robots to consistently refer to the same objects within their respective fields of view. In real-world applications, such as connected autonomous driving, vehicles face challenges in directly sharing raw observations due to limited communication bandwidth. In order to address this challenge, we propose a novel approach for bandwidth-adaptive spatiotemporal CoID in collaborative perception. This approach allows robots to progressively select partial spatiotemporal observations and share with others, while adapting to communication constraints that dynamically change over time. We evaluate our approach across various scenarios in connected autonomous driving simulations. Experimental results validate that our approach enables CoID and adapts to dynamic communication bandwidth changes. In addition, our approach achieves 8%-56% overall improvements in terms of covisible object retrieval for CoID and data sharing efficiency, which outperforms previous techniques and achieves the state-of-the-art performance. More information is available at: https://gaopeng5.github.io/acoid.

cs.RO↗

arXiv2025

Revisiting the Design of In-Memory Dynamic Graph Storage

The effectiveness of in-memory dynamic graph storage (DGS) for supporting concurrent graph read and write queries is crucial for real-time graph analytics and updates. Various methods have been proposed, for example, LLAMA, Aspen, LiveGraph, Teseo, and Sortledton. These approaches differ significantly in their support for read and write operations, space overhead, and concurrency control. However, there has been no systematic study to explore the trade-offs among these dimensions. In this paper, we evaluate the effectiveness of individual techniques and identify the performance factors affecting these storage methods by proposing a common abstraction for DGS design and implementing a generic test framework based on this abstraction. Our findings highlight several key insights: 1) Existing DGS methods exhibit substantial space overhead. For example, Aspen consumes 3.3-10.8x more memory than CSR, while the optimal fine-grained methods consume 4.1-8.9x more memory than CSR, indicating a significant memory overhead. 2) Existing methods often overlook memory access impact of modern architectures, leading to performance degradation compared to continuous storage methods. 3) Fine-grained concurrency control methods, in particular, suffer from severe efficiency and space issues due to maintaining versions and performing checks for each neighbor. These methods also experience significant contention on high-degree vertices. Our systematic study reveals these performance bottlenecks and outlines future directions to improve DGS for real-time graph analytics.

cs.DB↗

arXiv2025

GameArena: Evaluating LLM Reasoning through Live Computer Games

Evaluating the reasoning abilities of large language models (LLMs) is challenging. Existing benchmarks often depend on static datasets, which are vulnerable to data contamination and may get saturated over time, or on binary live human feedback that conflates reasoning with other abilities. As the most prominent dynamic benchmark, Chatbot Arena evaluates open-ended questions in real-world settings, but lacks the granularity in assessing specific reasoning capabilities. We introduce GameArena, a dynamic benchmark designed to evaluate LLM reasoning capabilities through interactive gameplay with humans. GameArena consists of three games designed to test specific reasoning capabilities (e.g., deductive and inductive reasoning), while keeping participants entertained and engaged. We analyze the gaming data retrospectively to uncover the underlying reasoning processes of LLMs and measure their fine-grained reasoning capabilities. We collect over 2000 game sessions and provide detailed assessments of various reasoning capabilities for five state-of-the-art LLMs. Our user study with 100 participants suggests that GameArena improves user engagement compared to Chatbot Arena. For the first time, GameArena enables the collection of step-by-step LLM reasoning data in the wild.

cs.AI↗

arXiv2025

High-Order Exceptional Point-Based Rotation Sensing in Anti-Parity time Symmetric Microresonators

Exceptional points (EPs), which arise from non-Hermitian systems, have been extensively investigated for the development of high-performance gyroscopes. However, operating a non-Hermitian gyroscope at high-order EP (HOEP) to achieve extreme performance requires strict and precise control of parameters. Here, we propose the design of an anti-parity-time (anti-PT) symmetric optical gyroscope operating at a fourth-order EP, achieving both ultra-sensitivity and high robustness. Our configuration exhibits eigenfrequency splitting two orders of magnitude higher than that of anti-PT gyroscopes operating at second-order EP. Furthermore, we demonstrate a significant reduction in angular random walk (ARW) under noise limits, compared to anti-parity symmetric gyroscopes based on second-order EP. Our results provide a novel approach for developing high-sensitivity rotation detection based on HOEPs.

physics.optics↗

arXiv2025

LMS-Net: A Learned Mumford-Shah Network For Few-Shot Medical Image Segmentation

Few-shot semantic segmentation (FSS) methods have shown great promise in handling data-scarce scenarios, particularly in medical image segmentation tasks. However, most existing FSS architectures lack sufficient interpretability and fail to fully incorporate the underlying physical structures of semantic regions. To address these issues, in this paper, we propose a novel deep unfolding network, called the Learned Mumford-Shah Network (LMS-Net), for the FSS task. Specifically, motivated by the effectiveness of pixel-to-prototype comparison in prototypical FSS methods and the capability of deep priors to model complex spatial structures, we leverage our learned Mumford-Shah model (LMS model) as a mathematical foundation to integrate these insights into a unified framework. By reformulating the LMS model into prototype update and mask update tasks, we propose an alternating optimization algorithm to solve it efficiently. Further, the iterative steps of this algorithm are unfolded into corresponding network modules, resulting in LMS-Net with clear interpretability. Comprehensive experiments on three publicly available medical segmentation datasets verify the effectiveness of our method, demonstrating superior accuracy and robustness in handling complex structures and adapting to challenging segmentation scenarios. These results highlight the potential of LMS-Net to advance FSS in medical imaging applications. Our code will be available at: https://github.com/SDZhang01/LMSNet

cs.CV↗

arXiv2025

3DMolFormer: A Dual-channel Framework for Structure-based Drug Discovery

Structure-based drug discovery, encompassing the tasks of protein-ligand docking and pocket-aware 3D drug design, represents a core challenge in drug discovery. However, no existing work can deal with both tasks to effectively leverage the duality between them, and current methods for each task are hindered by challenges in modeling 3D information and the limitations of available data. To address these issues, we propose 3DMolFormer, a unified dual-channel transformer-based framework applicable to both docking and 3D drug design tasks, which exploits their duality by utilizing docking functionalities within the drug design process. Specifically, we represent 3D pocket-ligand complexes using parallel sequences of discrete tokens and continuous numbers, and we design a corresponding dual-channel transformer model to handle this format, thereby overcoming the challenges of 3D information modeling. Additionally, we alleviate data limitations through large-scale pre-training on a mixed dataset, followed by supervised and reinforcement learning fine-tuning techniques respectively tailored for the two tasks. Experimental results demonstrate that 3DMolFormer outperforms previous approaches in both protein-ligand docking and pocket-aware 3D drug design, highlighting its promising application in structure-based drug discovery. The code is available at: https://github.com/HXYfighter/3DMolFormer .

cs.CE↗

arXiv2025

Learning to Compress Contexts for Efficient Knowledge-based Visual Question Answering

Multimodal large language models (MLLMs) have demonstrated great performance on visual question answering (VQA). When it comes to knowledge-based Visual Question Answering (KB-VQA), MLLMs may lack the specialized domain knowledge needed to answer questions, necessitating the retrieval of necessary information from external knowledge sources. Previous works like Retrival-Augmented VQA-v2 (RAVQA-v2) focus on utilizing as much input information, such as image-based textual descriptions and retrieved knowledge, as possible to improve performance, but they all overlook the issue that with the number of input tokens increasing, inference efficiency significantly decreases, which contradicts the demands of practical applications. To address this issue, we propose \textbf{R}etrieval-\textbf{A}ugmented MLLMs with Compressed Contexts (RACC). RACC learns to compress and aggregate retrieved knowledge for a given image-question pair, generating a compact modulation in the form of Key-Value (KV) cache to adapt the downstream frozen MLLM, thereby achieving effective and efficient inference. RACC achieves a state-of-the-art (SOTA) performance of 63.92\% on OK-VQA. Moreover, it significantly reduces inference latency by 22.0\%-59.7\% compared to the prominent RAVQA-v2. Abundant experiments show RACC's broad applicability. It is compatible with various off-the-shelf MLLMs and can also handle different knowledge sources including textual and multimodal documents.

cs.CV↗

arXiv2025

Long$^2$RAG: Evaluating Long-Context & Long-Form Retrieval-Augmented Generation with Key Point Recall

Retrieval-augmented generation (RAG) is a promising approach to address the limitations of fixed knowledge in large language models (LLMs). However, current benchmarks for evaluating RAG systems suffer from two key deficiencies: (1) they fail to adequately measure LLMs' capability in handling long-context retrieval due to a lack of datasets that reflect the characteristics of retrieved documents, and (2) they lack a comprehensive evaluation method for assessing LLMs' ability to generate long-form responses that effectively exploits retrieved information. To address these shortcomings, we introduce the Long$^2$RAG benchmark and the Key Point Recall (KPR) metric. Long$^2$RAG comprises 280 questions spanning 10 domains and across 8 question categories, each associated with 5 retrieved documents with an average length of 2,444 words. KPR evaluates the extent to which LLMs incorporate key points extracted from the retrieved documents into their generated responses, providing a more nuanced assessment of their ability to exploit retrieved information.

cs.CL↗

arXiv2025

Sunny.jl: A Julia Package for Spin Dynamics

Sunny is a Julia package designed to serve the needs of the quantum magnetism community. It supports the specification of a very broad class of spin models and a diverse suite of numerical solvers. These include powerful methods for simulating spin dynamics both in and out of equilibrium. Uniquely, it features a broad generalization of classical and semiclassical approaches to SU(N) coherent states, which is useful for studying systems exhibiting strong spin-orbit coupling or local entanglement effects. Sunny also offers a well-developed framework for calculating the dynamical spin structure factor, enabling direct comparison with scattering experiments. Ease of use is a priority, with tools for symmetry-guided modeling and interactive visualization.

quant-ph↗

arXiv2025

Iterative Proximal-Minimization for Computing Saddle Points with Fixed Index

Computing saddle points with a prescribed Morse index on potential energy surfaces is crucial for characterizing transition states for nosie-induced rare transition events in physics and chemistry. Many numerical algorithms for this type of saddle points are based on the eigenvector-following idea and can be cast as an iterative minimization formulation (SINUM. Vol. 53, p.1786, 2015), but they may struggle with convergence issues and require good initial guesses. To address this challenge, we discuss the differential game interpretation of this iterative minimization formulation and investigate the relationship between this game's Nash equilibrium and saddle points on the potential energy surface. Our main contribution is that adding a proximal term, which grows faster than quadratic, to the game's cost function can enhance the stability and robustness. This approach produces a robust Iterative Proximal Minimization (IPM) algorithm for saddle point computing. We show that the IPM algorithm surpasses the preceding methods in robustness without compromising the convergence rate or increasing computational expense. The algorithm's efficacy and robustness are showcased through a two-dimensional test problem, and the Allen-Cahn, Cahn-Hilliard equation, underscoring its numerical robustness.

math.OC↗