arXiv subjects

Peng Sun

Publications and source records attributed to Peng Sun.

At least 37 records · Page 2Linked to original sources

arXiv2026

OctoPipe: Reducing Pipeline Bubbles for Heterogeneous Models via Co-Optimizing Partitioning, Placement, and Scheduling

Pipeline parallelism is widely used to train large language models (LLMs). However, increasing heterogeneity in model architectures exacerbates pipeline bubbles, thereby reducing training efficiency. Prior approaches typically optimize a single phase of the pipeline schedule (i.e., partitioning, placement, or scheduling), leaving substantial pipeline bubbles. While promising, co-optimization poses three key challenges: (1) complex performance modeling, (2) a combinatorial search space, and (3) irregular execution orders. To address these challenges, we propose OctoPipe, a pipeline parallelism system to jointly optimize partitioning, placement, and scheduling. First, we build a graph-based pipeline simulator to model heterogeneous pipeline execution for co-optimization. Second, on top of the simulator, we develop an iterative bubble-aware tuner to efficiently explore the combinatorial search space. Third, we implement a unified pipeline executor that dynamically orchestrates computation and communication to support irregular execution orders without deadlocks while maximizing communication-computation overlap. Experiments show that OctoPipe achieves 1.09--1.49$\times$ throughput improvement over the state-of-the-art pipeline parallelism approaches across various heterogeneous model configurations and GPU cluster scales.

cs.DC↗

arXiv2026

Open-Source Autonomous Driving System Analysis and Multi-Disciplinary Hardware-in-the-Loop Research Paradigm with Reinforcement-Learning Testing and Large Language Models

Open-source autonomous driving systems provide an inspectable software foundation for intelligent vehicle research. Under real-vehicle deployment conditions, the recording and review of experimental conditions are important for interpreting system behavior and reusing experimental results. However, in a shared real-vehicle environment involving multiple vehicles, task processes, code modifications, and hardware testing feedback are often distributed across different teams and experimental stages, making it challenging to maintain continuous and reviewable experimental records. To address this limitation, this paper examines an Apollo-on-Hongqi EV environment and proposes a real-vehicle experimental framework. The framework connects multi-vehicle experiments, repository-based code reuse and software-hardware testing feedback within a unified review process. Large language models and RL-based testing serve as auxiliary components for record organization, anomaly summarization, and simulation-based candidate scenario generation. Based on this setting, this paper analyzes preliminary evidence from multi-vehicle collaborative experimentation, code and experimental-skill sharing, and software-hardware collaborative testing. The analysis shows that experimental records can be examined together with their operating conditions, providing a reviewable basis for Apollo-on-Hongqi EV research.

cs.SE↗

arXiv2026

Belayer: Efficient Fault Tolerance for LLM Agentic RL Training

Large language model (LLM) agents are increasingly trained with reinforcement learning in long-horizon, sandboxed environments. Unlike conventional RL, agentic RL couples GPU-intensive rollout engines with stateful environment containers whose actions may produce visible side effects, such as file edits, command execution, and dependency installation. A single trajectory can span many rounds of gen- eration and environment interaction, so a component failure can discard completed work or expose the model to an environment state that is inconsistent with its context. However, existing systems lack efficient and correct recovery mechanisms for this distributed execution model. This paper presents Belayer, an efficient fault-tolerant system for LLM agentic RL training. Belayer handles failures in both rollout engines and environment execution while targeting low failure-free overhead. For scoped worker-local rollout failures, Belayer equips each pre-initialized shadow worker with a selective GPU-state reuse protocol that retains independently owned weights and raw KV-arena allocations after owner and GPU health checks, reinitializes worker-local state, and rebuilds request-specific KV contents from logged token prefixes. For environment failures, Belayer introduces full checkpoint and full restore to jointly capture and restore container file-system and runtime state, and coordinates the recovered environment with the LLM context to preserve prefix consistency. An adaptive policy opportunistically overlaps full-state checkpointing with natural LLM inference bubbles when the predicted interval is long enough. Empirical results show low measured overhead during failure-free training, a worker-recovery-time reduction of up to 42 times faster compared with a full engine cold start, and 1.5 to 3.5 times faster recovery from environment failures.

cs.DC↗

arXiv2026

Bridging Severe Cross-Modal Misalignment: End-to-End Visible-Infrared Object Detection via Explicit Feature-Domain Affine Registration

Visible-infrared object detection relies on complementary RGB and thermal cues, but its performance is often degraded by cross-modal spatial misalignment. Most existing methods rely on implicit feature adaptation to handle weakly misaligned scenarios, while large-offset geometric discrepancies remain insufficiently addressed. In this paper, we propose a Joint Feature-domain Registration and Detection network (JFRDet), an end-to-end visible-infrared oriented object detector tailored for severely cross-modal geometric discrepancies. JFRDet introduces a Cross-Modal Affine Alignment (CMAA) module to estimate an image-level affine transformation for explicit multi-level feature alignment. Note that illumination changes directly affect the reliability of RGB cues, an Illumination-Guided Complementary Fusion (IGCF) module adaptively exploits modality reliability under varying illumination conditions for cross-modal fusion. Then, an Alignment Quality-Consistency Gating (AQCG) strategy stabilizes joint optimization by modulating detection supervision according to alignment reliability and gradient consistency. We further construct DroneVehicle Misaligned (DVMA), a benchmark for evaluating visible-infrared oriented object detection under severe cross-modal geometric misalignment. The proposed JFRDet achieves 69.7\% $\mathrm{mAP}_{50}$ on DVMA, which represents state-of-the-art (SOTA) performance. The code and dataset will be available on GitHub.

cs.CV↗

arXiv2026

Data-DPO: Direct Preference Optimization for Target Model Data Selection in LLM Post-Training

Data selection in supervised fine-tuning aims to select a small set of effective samples from large-scale candidate data, reducing training cost while preserving model performance. However, existing methods usually treat data value as a relatively static property, and pay limited attention to the compatibility between data and the capability distribution of the target model. To address this issue, we propose Data-DPO, a target model-oriented SFT data selection method. Data-DPO observes the local training feedback of the target model on different samples through one-step probing, transforms activation differences among samples into pairwise data preferences, and trains a lightweight reward model to learn target-model-aware data preferences. In the final selection stage, Data-DPO further combines target model preference, external quality scores, and marginal diversity to construct a more stable and effective training subset. Experimental results on Vision-Flan and LLaVA-CoT show that Data-DPO consistently outperforms existing data selection baselines under multiple data budgets and stably surpasses full data training performance.

cs.LG↗

arXiv2026

Hierarchical Data Selection via Manifold Coverage and Sparse Feature Coverage in LLM Post-training

As supervised fine-tuning data continues to scale, selecting high-value subsets from large candidate pools is crucial for reducing training cost and improving model performance. Existing methods often measure diversity directly in the original embedding space, where geometric metrics entangle dominant semantic directions, fine-grained supervision differences, and local noise. We address this limitation by formulating data selection as a coarse-to-fine hierarchical coverage problem and propose MASS. MASS learns low-dimensional principal manifold coordinates with a dense autoencoder for coarse semantic grouping, and then performs quality-aware sparse feature coverage within each group using a TopK sparse autoencoder. Experiments on Vision Flan and LLaVA-CoT show that MASS consistently outperforms strong data selection baselines across multiple budgets, and in several settings matches or surpasses full data training with only a small subset of data.

cs.LG↗

arXiv2026

CMuon: Accelerating and Stabilizing Diffusion Transformer Training via Chunked Momentum Orthogonalization

Diffusion Transformers (DiTs) have achieved state-of-the-art (SOTA) performance in visual generative modeling, yet their training remains computationally prohibitive. While the recently proposed Momentum Orthogonalization (Muon) optimizer offers a promising alternative to AdamW, its direct application to DiTs yields suboptimal late-stage convergence. In this paper, we identify the root cause of this bottleneck: standard DiT architectures fuse functionally distinct weights (e.g., within AdaLN and QKV layers) into unified tensors for computational efficiency. Applying Muon to these fused tensors inadvertently induces implicit subspace coupling, which distorts update directions and degrades global optimization. To address this, we introduce Chunked Muon (CMuon), a simple yet highly effective strategy that partitions these matrices into independent sub-components prior to orthogonalization. Extensive experiments demonstrate that a 675M-parameter DiT trained with CMuon achieves a FID of 1.18 on ImageNet 256 in just 200 epochs. This represents more than a 2x training speedup over AdamW, while effectively overcoming the late-stage convergence plateaus of vanilla Muon.

cs.AI↗

arXiv2026

Unpolarized gluon PDF of the nucleon from lattice QCD at physical point in the continuum limit

We report a state-of-the-art lattice QCD calculation of the nucleon unpolarized gluon parton distribution function employing large-momentum effective theory. The calculation is carried out on the 2+1 flavor CLQCD ensembles with five lattice spacings a={0.105,0.0897,0.0775, 0.0688, 0.0519} fm and various pion masses ranging from 136 MeV to 317 MeV, covering nulceon momenta up to 3 GeV. Distillation technique is applied to improve the signal of two-point correlators. We then apply the state-of-the-art hybrid renormalization and one-loop perturbative matching, and extrapolate the result to the continuum limit, infinite momentum limit and physical pion mass.

hep-lat↗

arXiv2026

Pion Transition Form Factor in Lattice QCD

We investigate the neutral pion transition form factor $F_{π^0γ^\astγ^\ast}(q_1^2,q_2^2)$ in lattice QCD and confirm that the connected and disconnected contributions have the same sign. We employ the recently proposed blending method, which supplies an unbiased and cheap estimators for the required all-to-all propagators. The external pion states are treated within the distillation framework, while the electromagnetic currents are evaluated in the full blending space. Numerical tests are performed on an $N_f=2+1$ lattice ensemble. Our result shows that the contribution of the disconnected part is approximately $1\%$ of that of the connected part and enables constructive interference of probability amplitudes.

hep-lat↗

arXiv2026

The limit and boundary characteristic classes in Borel-Moore motivic homology

We show that the zero-dimensional part of the pro-Chern-Schwarz-MacPherson class defined by Aluffi can be lifted to the zeroth Suslin homology. The proof uses the pro-characteristic class in the limit Borel-Moore motivic homology, which has a quadratic refinement in the limit Borel-Moore Milnor-Witt homology. In characteristic zero, this construction factors through the group of constructible functions, in a way compatible with the covariant functoriality; in positive characteristic this property fails, and we show that the failure can be measured by the boundary characteristic class in the boundary Borel-Moore motivic homology. We prove a push-forward formula for the boundary characteristic class, and conjecture it to agree with the Swan class defined by Kato-Saito.

math.AG↗

arXiv2026

Two-nucleon systems at $m_π\approx292$ MeV from lattice QCD

Nucleon-nucleon systems in the $^3S_1$ and the $^1S_0$ channels are studied in lattice quantum chromodynamics at a pion mass of approximately $m_π\approx292$ MeV, employing three $N_f = 2+1$ ensembles with the same pion mass and lattice spacing $a=0.10530(18)$ fm but different spatial volumes. Finite-volume energies of the nucleon-nucleon systems are determined in both the rest frame and a moving frame. The distillation quark smearing method is applied to improve the precision and to ensure the symmetric correlators by using the same interpolating operators at sink and source. The scattering amplitudes are extracted from the finite-volume spectra using the Lüscher's finite-volume method. At the studied pion mass, both the $^3S_1$ (deuteron) and $^1S_0$(di-neutron) channels exhibit a virtual state pole, with binding energies of $6^{+5}_{-3}$ MeV and $11^{+6}_{-5}$ MeV, respectively. To investigate the effects of the left-hand cut, an alternative method -- the Non-Perturbative Hamiltonian framework (NPHF) -- is used for the scattering analysis and yields consistent results with those from the Lüscher method.

hep-lat↗

arXiv2026

Three-Body Scattering for Generative Modeling

Modern generative models typically rely on an adversarial critic, a prescribed noise-to-data path, or an autoregressive factorization. Instead, we show that a proper distributional energy can induce sample-level motion and provide direct regression supervision for a one-step generator. Three-Body Scattering Modeling (TBSM) for generation turns the energy distance into a constant-size per-projectile interaction: each projectile is attracted toward one real source and repelled from one independently generated source. Conditioned on the projectile and its condition, its expectation equals the $2$-Wasserstein gradient-flow velocity of $\frac12D_E^2(P_θ,Q)$. A batch of $B$ frozen-target events yields $O(B)$ sample-level losses, each using one reference for its condition instead of the minibatch-wide all-pairs field used by methods such as Drifting Models. Tracking this conditional expectation online can reduce field noise. Using scattering in frozen image features, TBSM trains one-step generators on ImageNet-256, achieving FID${}=2.23$ with pixel-space PixelDiT-XL and FID${}=1.63$ with latent-space DiT-XL at NFE${}=1$. We provide a design map relating diffusion-related supervision, Drift-like dynamics, and GAN-like objectives. These results establish tracked scattering as a route to high-dimensional one-step generation. Code: https://github.com/sp12138/TBSM.

cs.LG↗

arXiv2026

Low-energy $DD$ scattering in lattice QCD

We present the first lattice QCD calculation of single-channel $DD$ scattering with quantum numbers $I(J^P)=1(0^+)$ and $0(1^-)$. The calculation is performed on the $2+1$ flavor Wilson-Clover ensembles with a lattice spacing $a\simeq 0.077$ fm and two different pion masses, $m_π\simeq207$ and $305$ MeV. The scattering parameters are determined using the Lüscher's finite volume method. Our results indicate a weak repulsive interaction in the $1(0^+)$ channel and a slightly attractive interaction in the $0(1^-)$ channel. The $S$-wave isovector $DD$ scattering length and effective range, extrapolated to the physical pion mass, are $(-0.25\pm0.08\pm 0.12)$ fm and $(-5.7\pm4.5\pm 1.7)$ fm, respectively.

hep-lat↗

arXiv2026

BoRP: Bootstrapped Regression Probing for Scalable and Human-Aligned LLM Evaluation

Accurate evaluation of user satisfaction is critical for iterative development of conversational AI. However, for open-ended assistants, traditional A/B testing lacks reliable metrics: explicit feedback is sparse, while implicit metrics are ambiguous. To bridge this gap, we introduce BoRP (Bootstrapped Regression Probing), a scalable framework for high-fidelity satisfaction evaluation. Unlike generative approaches, BoRP leverages the geometric properties of LLM latent space. It employs a polarization-index-based bootstrapping mechanism to automate rubric generation and utilizes Partial Least Squares (PLS) to map hidden states to continuous scores. Experiments on industrial datasets show that BoRP (Qwen3-8B/14B) significantly outperforms generative baselines (even Qwen3-Max) in alignment with human judgments. Furthermore, BoRP reduces inference costs by orders of magnitude, enabling full-scale monitoring and highly sensitive A/B testing via CUPED.

cs.CL↗

arXiv2026

Condensing Large-Scale Datasets Directly with Minimal Information Loss

Recent advancements in scaling dataset distillation rely heavily on decoupled information extraction pipelines, comprising SQUEEZE, RECOVER, and RELABEL stages. Despite their scalability to large-scale datasets, these methods suffer from prohibitive computational overhead and poor cross-architecture generalization. In this paper, we reveal the root cause of these bottlenecks: the implicit dual-compression process, from data to model and back to images, inherently induces severe information loss. Crucially, we empirically and theoretically demonstrate that this loss creates a distribution shift that fundamentally compromises the widely adopted RELABEL strategy, transforming the pre-trained model into an unreliable labeler that yields sub-optimal labels. To overcome these critical flaws, we propose CIM, a novel, metric-driven framework that abandons the flawed dual-compression paradigm. Instead, CIM explicitly quantifies and minimizes the information gap between the original and synthetic datasets. By directly aligning the data distributions, our approach ensures high-fidelity information condensation and inherently satisfies the prerequisites for effective relabeling. Extensive experiments demonstrate that CIM establishes a new state-of-the-art. Notably, it distills ImageNet-1K at an IPC=10 in merely 80 minutes on a single RTX-4090 GPU, achieving an unprecedented 48.7% Top-1 accuracy on ResNet-18 and significantly outperforming previous SOTA approaches, such as NRR-DD and DELT, by 2.6% and 2.9%, respectively. Our code is available at https://github.com/LINs-lab/CIM.

cs.CV↗

arXiv2026

Impact of Dynamical Charm Quark and Mixed Action Effect on Light Hadron Masses and Decay Constants

We investigate the impact of including a dynamical charm quark on the properties of light hadrons. Our study compares the calculations performed on 2+1+1 flavor (HISQ fermion) ensembles at four lattice spacings to those on 2+1 flavor (clover fermion) ensembles at six lattice spacings, with both sets of ensembles employing the identical Symanzik gauge action. For the light, strange and charm flavor observables, we employ the same tadpole-improved clover fermion action. From numerical results for light and strange quark masses, pion and kaon decay constants, and $Ω$ and $Ω_{ccc}$ baryon masses, we find that the values obtained after continuum, chiral, and infinite-volume extrapolations are consistent within uncertainties. Even though the mixed action setup can introduce additional discretization effects, our calculation shows evidences that those effects can cancel with the discretization error in the unitary setup, resulting in better convergence in the continuum extrapolation.

hep-lat↗

arXiv2026

Does the Question Really Matter? Training-Free Data Selection for Vision-Language SFT

Visual instruction tuning is crucial for improving vision-language large models (VLLMs). However, many samples can be solved via linguistic patterns or common-sense shortcuts, without genuine cross-modal reasoning, limiting the effectiveness of multimodal learning. Prior data selection methods often rely on costly proxy model training and focus on difficulty or diversity, failing to capture a sample's true contribution to vision-language joint reasoning. In this paper, we propose CVS, a training-free data selection method based on the insight that, for high-quality multimodal samples, introducing the question should substantially alter the model's assessment of answer validity given an image. CVS leverages a frozen VLLM as an evaluator and measures the discrepancy in answer validity with and without conditioning on the question, enabling the identification of samples that require vision-language joint reasoning while filtering semantic-conflict noise. Experiments on Vision-Flan and The Cauldron show that CVS achieves solid performance across datasets. On Vision-Flan, CVS outperforms full-data training by 3.5% and 4.8% using only 10% and 15% of the data, respectively, and remains robust on the highly heterogeneous Cauldron dataset. Moreover, CVS reduces computational cost by 17.3% and 44.4% compared to COINCIDE and XMAS.

cs.AI↗

arXiv2026

PlexRL: Cluster-Level Orchestration of Serviceized LLM Execution for RLVR

Reinforcement learning with verifiable rewards (RLVR) has recently unlocked strong reasoning capabilities in large language models (LLMs), triggering rapid exploration of new algorithms and data. However, RLVR training is notoriously inefficient: long-tailed rollouts, tool-induced stalls, and asymmetric resource requirements between rollout and training introduce substantial idle time that cannot be eliminated by job-local optimizations such as synchronous pipelining, asynchronous rollout, or colocated execution. We argue that this inefficiency is structural. While idle gaps are unavoidable within individual RLVR jobs, they are largely anti-correlated across jobs and therefore exploitable at the cluster level. Leveraging this observation, we present PlexRL, a cluster-level runtime for multiplexing unified LLM services across RLVR jobs. By centrally managing model placement, state transitions, and function-level scheduling under strict affinity constraints, PlexRL time-slices LLM execution across jobs to fill otherwise idle periods without expensive model migration. Our implementation and evaluations demonstrate that PlexRL significantly improves effective cluster capacity and reduces user GPU hour cost by maximum 37.58% while preserving algorithmic flexibility and introducing minimal per-job overhead.

cs.DC↗