Search arXiv⌕ Search

arXiv subjects

Haolin Liu

Publications and source records attributed to Haolin Liu.

At least 19 recordsLinked to original sources
arXiv2026

Breaking $\mathsf{TraintoDefend}$ via Adaptive Neuron Clustering in Cryptanalytic Model Extraction

Cryptanalytic model extraction poses a fundamental threat to deep learning model confidentiality by enabling precise parameter recovery with theoretical guarantees of functional equivalence. To counter this growing threat, $\mathsf{TraintoDefend}$ (NeurIPS 2025) was proposed as the first dedicated defense against cryptanalytic extraction and remains the only defense believed to be effective to date. It encourages neurons to learn similar weights so that they switch activation states together, making their individual contributions harder to distinguish. However, we notice that neurons with similar weights need not switch activation states together under small perturbations. Instead, weight similarity can cause the original clustering rule to merge signatures from distinct neurons when only a subset of their normalized weight coordinates agree. Motivated by this observation, we introduce $\mathsf{SimBreak}$, an adaptive clustering rule that breaks the weight-similarity defense of $\mathsf{TraintoDefend}$. $\mathsf{SimBreak}$ estimates a relative scale from all jointly observed weight coordinates and uses the same scale to test both weight direction and bias consistency. Across six protected fully connected networks trained on MNIST, $\mathsf{SimBreak}$ raises target layer signature recovery from $0\%$ to $100\%$, with a maximum relative recovery error of $2^{-22.27}$. Our work challenges the use of neuron weight similarity as a sufficient basis for protection and establishes adaptive evaluation as essential to assessing cryptanalytic extraction defenses.

cs.CR↗
arXiv2026

Reverse-KL Policy Distillation: Statistical Guarantees and Limits

Reverse Kullback-Leibler (KL) distillation has shown empirical promise for training language models, but its statistical guarantees and limits remain poorly understood. In particular, the effects of two design choices remain unclear: Does access to the teacher's full action distribution provide an advantage over observing only the probability it assigns to the sampled action? Must training trajectories come from the student (online distillation), or can teacher-generated trajectories (offline distillation) suffice? Our work takes a first step toward understanding the theory underlying these questions. We study the sample complexity of online reverse-KL distillation under two feedback models: the teacher reveals either its full action distribution or only the probability of the student-generated action. We establish minimax bounds for the former and show that substantially more data may be required for the latter. In contrast, offline reverse-KL distillation can attain the same minimax rate whether the teacher reveals its full distribution or only its probability of the teacher-generated action. The key distinction is that teacher-generated actions themselves reveal the teacher's preferences, while queries at student-selected actions can leave those preferences unobserved. This highlights an additional statistical challenge for online distillation compared to offline distillation. These statistical guarantees do not follow simply from optimizing standard distillation objectives: our lower bounds show that common online objectives can fail when candidates are evaluated only on past student-generated trajectories. We develop trajectory-loss and importance-ratio truncation techniques together with policy-space learning to achieve optimal online distillation guarantees.

cs.LG↗
arXiv2026

Evolving Language Models without Labels: Majority Drives Selection, Novelty Promotes Variation

Large language models (LLMs) can improve through reinforcement learning with verifiable rewards (RLVR), but many settings lack reference answers or external verifiers. Label-free self-improvement instead constructs training signals from the model's own outputs. Because the same policy generates both the candidate responses and the signals derived from them, each update also changes the candidates available for future training, creating a self-reinforcing feedback loop. We identify a failure mode of this loop, which we call selection-induced diversity collapse: repeated self-selection can progressively narrow the candidate pool and make alternative correct solutions harder to sample. To address this problem, we propose EVOL-RL, which balances selection and variation. Majority voting provides a stable selection signal, while embedding-based novelty refines credit among responses with the same selection status; entropy regularization and asymmetric clipping further support this process by maintaining diverse rollouts and preserving rare, high-advantage updates. Across multiple training distributions, model scales, backbones, and matched label-free baselines, EVOL-RL consistently improves pass@n while generally maintaining or improving pass@1. For example, when trained without labels on AIME24, Qwen3-4B-Base improves over TTRL on held-out AIME25 from 4.6% to 17.1% in pass@1 and from 18.5% to 42.0% in pass@16. Training analyses further show that when the current majority is wrong, EVOL-RL better preserves alternative correct solutions. The code is available at: https://github.com/YujunZhou/EVOL-RL.

cs.LG↗
arXiv2026

A Computationally Efficient and Rate-Optimal Algorithm for Adversarial Linear MDPs with (Aggregate) Bandit Feedback

We study episodic linear Markov decision processes with unknown transitions, adversarial losses, and bandit feedback. The state-of-the-art result for this setting (Liu et al., 2024) either achieves $\sqrt{K}$ regret with a computationally inefficient algorithm or achieves suboptimal $K^{3/4}$ regret, where $K$ is the number of episodes. Our work gives the first $\sqrt{K}$ regret with a polynomial-time algorithm. Our method reduces adversarial linear MDPs to adversarial linear bandits whose action set consists of policies' expected feature vectors. The central challenge is that this action set is unknown and can be estimated only through exploration. We address this challenge by constructing an approximate separation oracle for the action set that either generates executable policies from queried features, or refines an outer approximation of the action set. This approach draws inspiration from the reduction of van Erven et al. (2025) for linear contextual bandits and the oracle-efficient linear bandit algorithm of Ito et al. (2019), but additionally deals with the imprecision of the oracle. The resulting algorithm requires only aggregate loss feedback, and to our knowledge. This is the first polynomial-time algorithm for adversarial MDPs with aggregate feedback beyond the tabular case.

cs.LG↗
arXiv2026

Hierarchical Graph Neural Network for Probabilistic Power Flow with Grid Topology Contingencies

Data-driven surrogates reduce the nonlinear solves required for probabilistic power flow (PPF), but models tied to a fixed bus--branch graph often generalize poorly when outages change electrical coupling. This paper proposes a topology-decomposition-driven hierarchical bidirectional graph neural network (TDD--HBGNN) for PPF under unseen topologies. For each realized topology, the network is decomposed into fundamental topological units (FTUs), dynamically homogeneous voltage clusters (DHVCs), and a core backbone sub-grid (CBSG), with equivalent-admittance representations preserving electrical ports between units. A sensitivity-based linear prior captures the first-order voltage-magnitude and phase-angle response, while the HBGNN learns topology-dependent nonlinear residuals. Impedance- and uncertainty-aware bottom-up aggregation condenses local perturbations, while mode-modulated top-down broadcasting restores system context at buses. We evaluate our approach on the IEEE 118-bus and Polish 2746-bus systems under unseen N--1 and N--2 contingencies against six baseline methods. Compared with these methods, TDD--HBGNN achieves 2.47--4.20-fold lower all-bus phase-angle errors and lower phase-angle errors in every held-out event. Its voltage-magnitude advantage is concentrated near the outaged branches, where paired error ratios reach 1.75--2.58 on the IEEE system and 1.87--1.97 in Polish N--2 events. All system--contingency--baseline comparisons favor TDD--HBGNN in joint voltage--angle error. Moreover, shared hierarchical operators keep the model size at 0.658 million parameters across both grids and yield 0.161--1.793~ms forward latency per probabilistic case. These results demonstrate system-wide angular accuracy, outage-local voltage accuracy, and parameter-efficient scaling across unseen topologies.

eess.SY↗
arXiv2026

Full-Vector Diffractive Deep Neural Networks

Conventional diffractive deep neural networks (D2NNs) treat polarization as independent channels and rely on scalar or semi-vectorial propagation models, which neglect cross polarization coupling and vector diffraction. Here we propose a vector D2NN (V-D2NN) that embeds the full vector angular spectrum method into the end-to-end training pipeline. This physically rigorous framework describes vectorial light-matter interactions across cascaded diffractive layers, enabling direct optimization of polarization conversion, spin-orbit coupling, and vectorial interference without external polarization optics. We demonstrate the V-D2NN on polarization-multiplexed tasks--ector beam generation, polarization dependent imaging and classification, and multiple channel optical encryption--where it consistently outperforms scalar and semi-vectorial counterparts. In longitudinal-field engineering, the V-D2NN actively shapes a prescribed longitudinal field with a normalized correlation of 0.839, a capability inaccessible to scalar-propagation models. An open-source training framework is also provided to support further development of vectorial diffractive optics for computing, sensing, and communications.

physics.optics↗
arXiv2026

Observable sets for the Schrödinger equation with potential $\lvert x\rvert$

In this paper, we study observable sets for the one-dimensional Schrödinger equation associated with the operator $H=-d^2/dx^2+|x|$ on $L^2(\mathbb R)$. We prove that a measurable set is observable at every positive time if and only if it is thick. For each fixed observation time, the observability constant is uniform over all measurable sets with prescribed thickness parameters. The theorem answers the question concerning the relationship between thick and observable set raised by Gengsheng Wang [12] for this anharmonic oscillator.

math.AP↗
arXiv2026

Dream-RSI: Recursive Self-Improvement through Evolving Worlds

Recursive self-improvement is becoming essential for autonomous AI agents, whose progress depends on discovering high-value solutions across complex domains. Effective exploration drives this process, yet managing and improving exploration strategies remains a major bottleneck. Current systems face a fundamental dilemma: fixed strategies fail to adapt as search spaces scale, while online policy optimization must navigate vast meta-search spaces under delayed, expensive feedback from long-horizon rollouts. We introduce \textsc{Dream-RSI}, a framework for scalable, recursively self-improving exploration. A lightweight orchestration layer makes exploration explicit and programmable while leaving the underlying base agent unchanged. Our key insight is that accumulated discovery history can act as a replay simulator over the realized search space. By dreaming within this simulator built from historical discovery trees, \textsc{Dream-RSI} obtains immediate, low-cost off-policy feedback to evaluate and refine exploration policies without repeated, expensive online evaluation. The improved policy is then redeployed online to drive further discovery, continuously expanding the simulator pool in a self-improving loop. Across 9 tasks in 4 domains, \textsc{Dream-RSI} achieves competitive quality and improves discovery efficiency in several settings.

cs.CL↗
arXiv2026

A reduced-order analysis for multi-reflection-driven laser absorptance in vaporization-induced cavities

Multiple internal reflections enhance laser absorption in vaporization-induced cavities, but connecting ray histories, cavity geometry, and absorptance typically requires full ray tracing. We develop a reduced-order framework based on Beer-Lambert-type attenuation governed by an effective per-reflection optical depth (EPROD). A power-weighted formulation defines an absorptance-equivalent interaction count under constant effective single-interaction absorptance, while a cumulant expansion separates the EPROD-independent mean interaction count from ray-path heterogeneity corrections. Across nine simulation groups spanning three alloys and three spot diameters, the mean interaction count follows a unit-intercept linear depth relation with R-squared values of 0.7824-0.9843. Its slope varies by only 3.4%-6.3% across alloys at fixed spot diameter and decreases with increasing spot diameter. Fixing this slope from ray statistics yields an absorptance-depth model with one fitted parameter and a median prediction error (MAPE) of 2.61%. An independent synchronized Ti-64 experiment yields model parameters within approximately 2.4% of corresponding simulation values. This framework separates geometric interaction accumulation from optical attenuation, enabling efficient energy-coupling prediction and absorptance-based keyhole characterization.

physics.app-ph↗
arXiv2026

Boundary Spectral Inequalities from Measurable Sets on $W^{2,\infty}$ manifolds and their applications

Let $(\mathcal{M},g)$ be a compact connected $W^{2,\infty}$ Riemannian manifold with nonempty boundary and a uniformly elliptic Lipschitz metric. We establish quantitative boundary spectral inequalities for both Dirichlet and Neumann eigenfunctions from arbitrary measurable subsets of $\partial\mathcal{M}\times(-s_0,s_0)$ of positive surface measure. Moreover, we apply the Dirichlet spectral inequality to establish the boundary observability estimate \[ \|u(T)\|_{L^2(\mathcal{M})}^2 \le C \int_J \|\partial_{ν_g}u(x,t)\|^2 \,\mathrm{d}S_g\,\mathrm{d}t . \] where $J\subset\partial\mathcal{M}\times(0,T)$ is measurable and has positive surface--time measure. A localized theorem requiring $W^{2,\infty}$ regularity only near the observed boundary patch is retained as a separate consequence. The proof of the observability inequality combines quantitative continuation from positive-measure boundary sets with low-frequency spectral concentration. This result shows that boundary observability of solutions to the heat equation can also be achieved in $W^{2,\infty}$ domains. Finally, as an application, we prove that measurements of the boundary normal derivative on $J$ uniquely determine the initial state of heat equation. Under an a priori bound in $D(-Δ_{g, D}^{s/2})$, the recovery satisfies a logarithmic stability estimate of optimal order $s/2$.

math.AP↗
arXiv2026

Beyond Global Latents: Chunk-Based Sparse Grid VAE for Scalable 3D Modeling

Sparse voxel grids preserve the spatial structure needed for detailed 3D reconstruction, but their memory still grows rapidly with resolution as active surface cells increase. We introduce ChunkVAE, a sparse grid variational autoencoder organized around local chunks rather than a global latent volume. Local learned operators permit independently chosen encoder and decoder partitions and allow inference chunk sizes to differ from training. Two complementary data operators make this flexibility practical: Balanced Binary Object Partitioning distributes active cells while limiting replicated overlap, while S-Curve weighted stitching attenuates unreliable boundary features when assembling a global latent or reconstruction. Across three object benchmarks, ChunkVAE is competitive with or better than strong baselines from $512^3$ to $1536^3$; smaller chunks lower peak allocated memory and shorten per-chunk compute, enabling faster parallel inference. Stable stitched latents and improved image to 3D metrics indicate that local compression can scale geometry while retaining the global interface required downstream.

cs.CV↗
arXiv2026

Anisotropic minimal surface equation with Dirichlet boundary condition

This paper investigates the Dirichlet problem for the anisotropic minimal surface equation in a bounded domain. Under the natural assumption of non-negative boundary anisotropic mean curvature, we establish the unique solvability of the Dirichlet problem for continuous boundary data. To achieve this, an essential a priori gradient estimate is established, which also allows us to prove a weak version of Bernstein's theorem for entire solutions under a sharp, one-sided linear growth assumption. Moreover, using the direct method in the calculus of variations, we prove the existence and local Lipschitz regularity of generalized minimizers in $BV(Ω)$ with $L^1(\partialΩ)$ boundary data. We also find that this variational formulation naturally yields a Neumann-type boundary condition, geometrically explaining why the Neumann problem requires no boundary curvature constraints.

math.AP↗
arXiv2026

Moving Out: Physically-grounded Human-AI Collaboration

The ability to adapt to physical actions and constraints in an environment is crucial for embodied agents (e.g., robots) to effectively collaborate with humans. Such physically grounded human-AI collaboration must account for the increased complexity of the continuous state-action space and constrained dynamics caused by physical constraints. However, most existing collaboration benchmarks are discrete or do not consider physical attributes and constraints. To address this, we introduce Moving Out, a human-AI collaboration benchmark that resembles a wide range of collaboration modes affected by physical attributes and constraints, such as moving heavy items together and coordinating actions to move an item around a corner. Moving Out consists of two challenges and human-human interaction data to comprehensively evaluate models' abilities to adapt to diverse human behaviors and unseen physical attributes. To give embodied agents the capability to collaborate with humans under physical attributes and constraints, we propose a novel method, BASS (Behavior Augmentation, Simulation, and Selection), to enhance the diversity of agents and their understanding of the outcome of actions. We systematically compare BASS and state-of-the-art models in AI-AI and human-AI experiments, showing that BASS can effectively collaborate with both unseen AI and humans. The project page is available at https://live-robotics-uva.github.io/movingout_ai/.

cs.LG↗
arXiv2026

One Token to Fool LLM-as-a-Judge

Large language models (LLMs) are increasingly trusted as automated judges, assisting evaluation and providing reward signals for training other models, particularly in reference-based settings like Reinforcement Learning with Verifiable Rewards (RLVR). However, we uncover a critical vulnerability even in this reference-based paradigm: generative reward models are systematically susceptible to reward hacking. We find that superficial inputs, which we term ''master keys'' such as non-word symbols (e.g., '':'' or ''.'') or generic reasoning openers (e.g., ''Thought process:'' or ''Let's solve this problem step by step.''), can consistently elicit false positive rewards without any substantive reasoning. Our systematic evaluation demonstrates this is a widespread failure affecting a diverse range of models, including leading proprietary systems such as GPT-o1 and Claude-4. These results challenge the assumed robustness of LLM judges and pose a significant threat to their reliability. To address this, we propose a simple yet effective data augmentation strategy using truncated model outputs as adversarial negative examples. The resulting Master Reward Models (Master-RMs) demonstrate state-of-the-art robustness against these ''master key'' attacks while maintaining high performance in standard evaluation settings. We supplement these findings with a comprehensive analysis of the vulnerability across model scales, prompt variations, and common inference-time strategies, offering insights to guide future research on robust LLM evaluation. We release our robust, general-domain reward models and the synthetic training data at https://huggingface.co/sarosavo/Master-RM and https://huggingface.co/datasets/sarosavo/Master-RM.

cs.LG↗
arXiv2026

On the Complexity of Offline Reinforcement Learning with $Q^\star$-Approximation and Partial Coverage

We study offline reinforcement learning under $Q^\star$-approximation and partial coverage, a setting that motivates practical algorithms such as Conservative $Q$-Learning (CQL; Kumar et al., 2020) but has received limited theoretical attention. Our work is inspired by the following open question: "Are $Q^\star$-realizability and Bellman completeness sufficient for sample-efficient offline RL under partial coverage?" We answer in the negative via an information-theoretic lower bound. To identify additional structure that enables sample-efficient offline RL under partial coverage, we introduce a general decision-estimation framework, inspired by model-free decision-estimation coefficients (DEC) for online RL (Foster et al., 2023b; Liu et al., 2025b). Our framework decomposes offline RL complexity into decision complexity and value estimation error. This allows modular study of both sub-problems. Our result not only unifies existing results (Chen and Jiang, 2022; Uehara et al., 2023), but further improves and generalizes them. On the decision complexity side, our improvement includes: the first $ε^{-2}$ sample complexity bound for soft $Q$-learning under partial coverage that improves Uehara et al.'s (2023) $ε^{-4}$ bound, the removal of the need for additional online interaction in the value-gap setting of Chen and Jiang (2022), and new learnable settings beyond the above two cases. On the value estimation side, we provide a new characterization of the role of Bellman completeness under partial coverage, and the first characterization of offline learnability for general low-Bellman-rank MDPs (Jiang et al., 2017; Du et al., 2021; Jin et al., 2021). The latter is a canonical online RL setting that has remained unexplored in offline RL except for special cases. As a side contribution, our techniques give the first analysis of CQL in the function approximation setting.

cs.LG↗
arXiv2026

Supervising the search process produces reliable and generalizable information-seeking agents

Large language models (LLMs) are transforming web search by shifting from document ranking to synthesizing answers, and are increasingly deployed as autonomous agentic search systems that iteratively interact with external knowledge sources. Despite this progress, building effective search agents remains challenging because high-quality intermediate search steps are difficult to generate. Previous approaches have primarily relied on outcome supervision, rewarding agents only for producing correct final answers. This often leads to reward hacking and excessive dependence on parametric memory, limiting generalization to out-of-domain tasks. To address these limitations, we introduce RAG-Gym, a framework that shifts supervision from final answers to the search process itself. With RAG-Gym, we systematically investigate architecture design, parameter optimization, and action evaluation, identifying reasoning reflection as a critical capability for search agents. Building on this insight, we propose Re$^2$Search++, a process-supervised agent that achieves substantial improvements on multi-hop information-seeking benchmarks, especially in out-of-domain settings. Performance gains are driven primarily by higher-quality search queries rather than answer optimization alone, and the learned search critics transfer across models, including proprietary LLMs. These findings show that supervising the search process produces more reliable and generalizable information-seeking agents.

cs.CL↗
arXiv2026

Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation

We present Hunyuan3D 2.0, an advanced large-scale 3D synthesis system for generating high-resolution textured 3D assets. This system includes two foundation components: a large-scale shape generation model -- Hunyuan3D-DiT, and a large-scale texture synthesis model -- Hunyuan3D-Paint. The shape generative model, built on a scalable flow-based diffusion transformer, aims to create geometry that properly aligns with a given condition image, laying a solid foundation for downstream applications. The texture synthesis model, benefiting from strong geometric and diffusion priors, produces high-resolution and vibrant texture maps for either generated or hand-crafted meshes. Furthermore, we build Hunyuan3D-Studio -- a versatile, user-friendly production platform that simplifies the re-creation process of 3D assets. It allows both professional and amateur users to manipulate or even animate their meshes efficiently. We systematically evaluate our models, showing that Hunyuan3D 2.0 outperforms previous state-of-the-art models, including the open-source models and closed-source models in geometry details, condition alignment, texture quality, and etc. Hunyuan3D 2.0 is publicly released in order to fill the gaps in the open-source 3D community for large-scale foundation generative models. The code and pre-trained weights of our models are available at: https://github.com/Tencent/Hunyuan3D-2

cs.CV↗