arXiv subjects

Ruichen Xu

Publications and source records attributed to Ruichen Xu.

At least 19 recordsLinked to original sources

arXiv2026

Pseudonullity for fine Selmer groups over multiple $\mathbb{Z}_p^{d}$-extensions

In this article, we develop vertical control theorems for fine Selmer groups associated with Galois representations and their deformations in $p$-adic families. We apply them to provide new evidence for Conjectures A and B of Coates--Sujatha and to obtain pseudonullity of the fine Selmer groups for cyclotomic characters, elliptic curves, classical cuspidal newforms, and Hida families of $p$-ordinary cuspidal newforms.

math.NT↗

arXiv2026

HP-JEPA: Hierarchical Partitioning for Multi-Resolution Graph Joint-Embedding Predictive Learning

Graph self-supervised learning aims to learn transferable representations from large-scale unlabeled graph data. Joint-embedding predictive architectures (JEPAs) avoid explicit negative-pair construction and raw-input reconstruction by predicting masked targets directly in latent space. However, existing graph JEPAs typically rely on a single predefined graph partition, biasing the learned representations toward one structural granularity and limiting their ability to capture complementary patterns at different graph scales. To address this limitation, we propose HP-JEPA, a hierarchical partitioning framework for multi-resolution graph joint-embedding prediction. HP-JEPA organizes each graph into an ordered bank of coarse-to-fine partition resolutions and performs context-target latent prediction separately at each resolution using an online encoder, an exponential-moving-average target encoder, and a latent predictor. The resulting resolution-specific graph representations are subsequently integrated through concatenation or task-specific resolution weighting, allowing downstream models to combine complementary local, regional, and global structural information. Experiments on seven graph classification benchmarks and one graph regression benchmark show that HP-JEPA outperforms the fixed-resolution Graph-JEPA baseline on 6 of 8 tasks, improving upon Graph-JEPA on most evaluated benchmarks. Size-stratified analyses further show that HP-JEPA achieves higher accuracy than Graph-JEPA in most evaluated graph-size quartiles on three representative datasets. These results highlight the effectiveness of hierarchical multi-resolution partitioning for transferable graph representation learning.

cs.LG↗

arXiv2026

Towards Principled Test-Time Adaptation for Time Series Forecasting

Test-time adaptation (TTA) has recently emerged as a promising approach for improving time series forecasting (TSF) under distribution shift. Existing TSF-TTA methods differ in how they utilize revealed targets, yet the resulting adaptation protocols remain heterogeneous and lack a clearly unified formulation. To address this issue, we revisit TSF-TTA from the perspective of protocol cleanliness and propose an adaptation protocol based solely on matured ground truth, yielding a more principled setting for adaptation. Under this protocol, we further diagnose existing adapters in the frequency domain and find that their prediction corrections often exhibit limited and weakly structured spectral modifications. Motivated by this diagnosis, we propose Frequency-Aware Calibration (FAC), a lightweight calibration method that directly parameterizes prediction corrections in the frequency domain. Across diverse datasets, forecasting horizons, and source forecasters, FAC achieves competitive and consistent performance while requiring substantially fewer trainable parameters than the compared TSF-TTA adapters.

cs.LG↗

arXiv2026

Differential Privacy in Two-Layer Networks: How DP-SGD Harms Fairness and Robustness

Differentially private learning is essential for training models on sensitive data, but empirical studies consistently show that it can degrade performance, introduce fairness issues like disparate impact, and reduce adversarial robustness. The theoretical underpinnings of these phenomena in modern, non-convex neural networks remain largely unexplored. This paper introduces a unified feature-centric framework to analyze the feature learning dynamics of differentially private stochastic gradient descent (DP-SGD) in two-layer ReLU convolutional neural networks. Our analysis establishes test loss bounds governed by a crucial metric: the feature-to-noise ratio (FNR). We demonstrate that the noise required for privacy leads to suboptimal feature learning, and specifically show that: 1) imbalanced FNRs across classes and subpopulations cause disparate impact; 2) even in the same class, noise has a greater negative impact on semantically long-tailed data; and 3) noise injection exacerbates vulnerability to adversarial attacks. Furthermore, our analysis reveals that the popular paradigm of public pre-training and private fine-tuning does not guarantee improvement, particularly under significant feature distribution shifts between datasets. Experiments on synthetic and real-world data corroborate our theoretical findings.

cs.LG↗

arXiv2026

Feature Resemblance: Towards a Theoretical Understanding of Analogical Reasoning in Transformers

Understanding reasoning in large language models is complicated by evaluations that conflate multiple reasoning types. We isolate analogical reasoning, where a model transfers an attribute between entities that share known properties, and study when such transfer can emerge from training. To make the problem analytically tractable, we study a minimal transformer-style abstraction that isolates how learned representations support analogical reasoning. Within this setting, we prove three key results. First, joint training on similarity and attribution premises enables analogical reasoning through aligned representations. Second, sequential training succeeds only when similarity structure is learned before specific attributes, revealing a curriculum asymmetry. Third, in our stylized setting, two-hop reasoning $(a \to b, b \to c \Rightarrow a \to c)$ can be viewed as analogical reasoning with identity bridges $(b=b)$, which appear explicitly in training data. Together, these results reveal a unified mechanism: entities with shared properties become aligned in representation space, enabling property transfer through feature resemblance. Experiments with architectures up to 8B parameters show qualitative agreement with the theory and suggest that representational geometry plays an important role in analogical reasoning beyond the stylized model.

cs.CL↗

arXiv2026

Tackling Privacy Heterogeneity in Differentially Private Federated Learning

Differentially private federated learning (DP-FL) enables clients to collaboratively train machine learning models while preserving the privacy of their local data. However, most existing DP-FL approaches assume that all clients share a uniform privacy budget, an assumption that does not hold in real-world scenarios where privacy requirements vary widely. This privacy heterogeneity poses a significant challenge: conventional client selection strategies, which typically rely on data quantity, cannot distinguish between clients providing high-quality updates and those introducing substantial noise due to strict privacy constraints. To address this gap, we present the first systematic study of privacy-aware client selection in DP-FL. We establish a theoretical foundation by deriving a convergence analysis that quantifies the impact of privacy heterogeneity on training error. Building on this analysis, we propose a privacy-aware client selection strategy, formulated as a convex optimization problem, that adaptively adjusts selection probabilities to minimize training error. Extensive experiments on benchmark datasets demonstrate that our approach achieves up to a 10% improvement in test accuracy on CIFAR-10 compared to existing baselines under heterogeneous privacy budgets. These results highlight the importance of incorporating privacy heterogeneity into client selection for practical and effective federated learning.

cs.LG↗

arXiv2026

JSAM: Privacy Straggler-Resilient Joint Client Selection and Incentive Mechanism Design in Differentially Private Federated Learning

Differentially private federated learning faces a fundamental tension: privacy protection mechanisms that safeguard client data simultaneously create quantifiable privacy costs that discourage participation, undermining the collaborative training process. Existing incentive mechanisms rely on unbiased client selection, forcing servers to compensate even the most privacy-sensitive clients ("privacy stragglers"), leading to systemic inefficiency and suboptimal resource allocation. We introduce JSAM (Joint client Selection and privacy compensAtion Mechanism), a Bayesian-optimal framework that simultaneously optimizes client selection probabilities and privacy compensation to maximize training effectiveness under budget constraints. Our approach transforms a complex 2N-dimensional optimization problem into an efficient three-dimensional formulation through novel theoretical characterization of optimal selection strategies. We prove that servers should preferentially select privacy-tolerant clients while excluding high-sensitivity participants, and uncover the counter-intuitive insight that clients with minimal privacy sensitivity may incur the highest cumulative costs due to frequent participation. Extensive evaluations on MNIST and CIFAR-10 demonstrate that JSAM achieves up to 15% improvement in test accuracy compared to existing unbiased selection mechanisms while maintaining cost efficiency across varying data heterogeneity levels.

cs.LG↗

arXiv2025

Boundary-Informed Method of Lines for Physics Informed Neural Networks

We propose a hybrid solver that fuses the dimensionality-reduction strengths of the Method of Lines (MOL) with the flexibility of Physics-Informed Neural Networks (PINNs). Instead of approximating spatial derivatives with fixed finite-difference stencils - whose truncation errors force extremely fine meshes - our method trains a neural network to represent the initial spatial profile and then employs automatic differentiation to obtain spectrally accurate gradients at arbitrary nodes. These high-fidelity derivatives define the right-hand side of the MOL-generated ordinary-differential system, and time integration is replaced with a secondary temporal PINN while spatial accuracy is retained without mesh refinement. The resulting "boundary-informed MOL-PINN" matches or surpasses conventional MOL in accuracy using an order of magnitude fewer collocation points, thereby shrinking memory footprints, lessening dependence on large data sets, and increasing complexity robustness. Because it relies only on automatic differentiation and standard optimizers, the framework extends naturally to linear and nonlinear PDEs in any spatial dimension.

physics.comp-ph↗

arXiv2025

Can Data-Driven Dynamics Reveal Hidden Physics? There Is A Need for Interpretable Neural Operators

Recently, neural operators have emerged as powerful tools for learning mappings between function spaces, enabling data-driven simulations of complex dynamics. Despite their successes, a deeper understanding of their learning mechanisms remains underexplored. In this work, we classify neural operators into two types: (1) Spatial domain models that learn on grids and (2) Functional domain models that learn with function bases. We present several viewpoints based on this classification and focus on learning data-driven dynamics adhering to physical principles. Specifically, we provide a way to explain the prediction-making process of neural operators and show that neural operator can learn hidden physical patterns from data. However, this explanation method is limited to specific situations, highlighting the urgent need for generalizable explanation methods. Next, we show that a simple dual-space multi-scale model can achieve SOTA performance and we believe that dual-space multi-spatio-scale models hold significant potential to learn complex physics and require further investigation. Lastly, we discuss the critical need for principled frameworks to incorporate known physics into neural operators, enabling better generalization and uncovering more hidden physical phenomena.

cs.LG↗

arXiv2025

Kolmogorov-Arnold Representation for Symplectic Learning: Advancing Hamiltonian Neural Networks

We propose a Kolmogorov-Arnold Representation-based Hamiltonian Neural Network (KAR-HNN) that replaces the Multilayer Perceptrons (MLPs) with univariate transformations. While Hamiltonian Neural Networks (HNNs) ensure energy conservation by learning Hamiltonian functions directly from data, existing implementations, often relying on MLPs, cause hypersensitivity to the hyperparameters while exploring complex energy landscapes. Our approach exploits the localized function approximations to better capture high-frequency and multi-scale dynamics, reducing energy drift and improving long-term predictive stability. The networks preserve the symplectic form of Hamiltonian systems, and thus maintain interpretability and physical consistency. After assessing KAR-HNN on four benchmark problems including spring-mass, simple pendulum, two- and three-body problem, we foresee its effectiveness for accurate and stable modeling of realistic physical processes often at high dimensions and with few known parameters.

cs.LG↗

arXiv2025

Provable Knowledge Acquisition and Extraction in One-Layer Transformers

Large language models may encounter factual knowledge during pre-training yet fail to reliably use that knowledge after fine-tuning. Despite growing empirical evidence that MLP layers store factual associations and fine-tuning affects factual recall, the training-dynamics mechanisms linking next-token pre-training, knowledge storage, and post-fine-tuning extraction remain poorly understood. We study this problem in a stylized one-layer transformer with self-attention and MLP modules, trained by next-token prediction and subsequently fine-tuned on question-answering data. Under suitable regularity conditions, we first prove that the model reaches near-optimal pre-training loss while learning structured attention patterns and relation-specific feature directions, giving a mechanism for factual knowledge acquisition. We then show that fine-tuning can turn the Q&A prompt format into a trigger for pre-trained relation features, enabling the model to extract facts that are not revisited during fine-tuning. Our analysis yields a relation-covering characterization of knowledge extraction: fine-tuning need not revisit every stored subject-answer pair, but it must cover enough latent relation-template directions through which facts were encoded during pre-training. Consequently, extraction improves with pre-training multiplicity and fine-tuning coverage, but becomes harder as the relation-template universe grows. Conversely, insufficient coverage leads to a failure regime in which facts may be stored but remain inaccessible, providing a stylized mechanism for hallucination. The theory applies to both full and low-rank fine-tuning, offering insight into why low-rank adaptation can recover pre-trained factual knowledge when relation coverage is sufficient. Experiments on synthetic data and PopQA-based GPT-2/Llama models support the predicted trends.

cs.LG↗

arXiv2025

GAGA: Gaussianity-Aware Gaussian Approximation for Efficient 3D Molecular Generation

Gaussian Probability Path based Generative Models (GPPGMs) generate data by reversing a stochastic process that progressively corrupts samples with Gaussian noise. Despite state-of-the-art results in 3D molecular generation, their deployment is hindered by the high cost of long generative trajectories, often requiring hundreds to thousands of steps during training and sampling. In this work, we propose a principled method, named GAGA, to improve generation efficiency without sacrificing training granularity or inference fidelity of GPPGMs. Our key insight is that different data modalities obtain sufficient Gaussianity at markedly different steps during the forward process. Based on this observation, we analytically identify a characteristic step at which molecular data attains sufficient Gaussianity, after which the trajectory can be replaced by a closed-form Gaussian approximation. Unlike existing accelerators that coarsen or reformulate trajectories, our approach preserves full-resolution learning dynamics while avoiding redundant transport through truncated distributional states. Experiments on 3D molecular generation benchmarks demonstrate that our GAGA achieves substantial improvement on both generation quality and computational efficiency.

cs.LG↗

arXiv2025

An Iterative Framework for Generative Backmapping of Coarse Grained Proteins

The techniques of data-driven backmapping from coarse-grained (CG) to fine-grained (FG) representation often struggle with accuracy, unstable training, and physical realism, especially when applied to complex systems such as proteins. In this work, we introduce a novel iterative framework by using conditional Variational Autoencoders and graph-based neural networks, specifically designed to tackle the challenges associated with such large-scale biomolecules. Our method enables stepwise refinement from CG beads to full atomistic details. We outline the theory of iterative generative backmapping and demonstrate via numerical experiments the advantages of multistep schemes by applying them to proteins of vastly different structures with very coarse representations. This multistep approach not only improves the accuracy of reconstructions but also makes the training process more computationally efficient for proteins with ultra-CG representations.

cs.LG↗

arXiv2025

Velocity-Inferred Hamiltonian Neural Networks: Learning Energy-Conserving Dynamics from Position-Only Data

Data-driven modeling of physical systems often relies on learning both positions and momenta to accurately capture Hamiltonian dynamics. However, in many practical scenarios, only position measurements are readily available. In this work, we introduce a method to train a standard Hamiltonian Neural Network (HNN) using only position data, enabled by a theoretical result that permits transforming the Hamiltonian $H(q,p)$ into a form $H(q, v)$. Under certain assumptions, namely, an invertible relationship between momentum and velocity, we formally prove the validity of this substitution and demonstrate how it allows us to infer momentum from position alone. We apply our approach to canonical examples including the spring-mass system, pendulum, two-body, and three-body problems. Our results show that using only position data is sufficient for stable and energy-consistent long-term predictions, suggesting a promising pathway for data-driven discovery of Hamiltonian systems when momentum measurements are unavailable.

physics.comp-ph↗

arXiv2025

The Impact of Move Schemes on Simulated Annealing Performance

Designing an effective move-generation function for Simulated Annealing (SA) in complex models remains a significant challenge. In this work, we present a combination of theoretical analysis and numerical experiments to examine the impact of various move-generation parameters -- such as how many particles are moved and by what distance at each iteration -- under different temperature schedules and system sizes. Our numerical studies, carried out on both the Lennard-Jones problem and an additional benchmark, reveal that moving exactly one randomly chosen particle per iteration offers the most efficient performance. We analyze acceptance rates, exploration properties, and convergence behavior, providing evidence that partial-coordinate updates can outperform full-coordinate moves in certain high-dimensional settings. These findings offer practical guidelines for optimizing SA methods in a broad range of complex optimization tasks.

math.OC↗

arXiv2025

On the Mordell-Weil rank of certain CM abelian varieties over anticyclotomic towers

Let $K/\mathbb{Q}$ be an imaginary quadratic extension, and let $p$ be an odd prime. In this paper, we investigate the growth of Mordell-Weil ranks of CM abelian varieties associated with Hecke characters over $K$ of infinite type $(1, 0)$ along the $\mathbb{Z}_p$-anticyclotomic tower of $K$. Our results cover all decomposition types of $p$ in $K$. The analytic aspect of our proof is based on our computations of the local and global root numbers of Hecke characters, together with a recent generalization by H. Jia of D. Rohrlich's result concerning the relation between the vanishing orders of Hecke $L$-functions and their root numbers. The arithmetic conclusions then follow from the Gross-Zagier formula and the Kolyvagin machinery.

math.NT↗

arXiv2025

Rethinking Benign Overfitting in Two-Layer Neural Networks

Recent theoretical studies (Kou et al., 2023; Cao et al., 2022) have revealed a sharp phase transition from benign to harmful overfitting when the noise-to-feature ratio exceeds a threshold-a situation common in long-tailed data distributions where atypical data is prevalent. However, harmful overfitting rarely happens in overparameterized neural networks. Further experimental results suggested that memorization is necessary for achieving near-optimal generalization error in long-tailed data distributions (Feldman & Zhang, 2020). We argue that this discrepancy between theoretical predictions and empirical observations arises because previous feature-noise data models overlook the heterogeneous nature of noise across different data classes. In this paper, we refine the feature-noise data model by incorporating class-dependent heterogeneous noise and re-examine the overfitting phenomenon in neural networks. Through a comprehensive analysis of the training dynamics, we establish test loss bounds for the refined model. Our findings reveal that neural networks can leverage "data noise" to learn implicit features that improve the classification accuracy for long-tailed data. Our analysis also provides a training-free metric for evaluating data influence on test performance. Experimental validation on both synthetic and real-world datasets supports our theoretical results.

cs.LG↗

arXiv2024

The Gan-Gross-Prasad period of Klingen Eisenstein families over unitary groups

In this article, we compute the Gan-Gross-Prasad period integral of Klingen Eisenstein series over the unitary group $\mathrm{U}(m+1, n+1)$ with a cuspidal automorphic form over $\mathrm{U}(m+1, n)$, and show that it is related to certain special Rankin-Selberg $L$-values. We $p$-adically interpolate these Gan-Gross-Prasad period integrals as the Klingen Eisenstein series and the cuspidal automorphic form vary in Hida families. As a byproduct, we obtain a $p$-adic $L$-function of Rankin-Selberg type over $\mathrm{U}(m,n) \times \mathrm{U}(m+1, n)$. The ultimate motivation is to show the $p$-primitive property of Klingen Eisenstein series over unitary groups, by computing such Gan-Gross-Prasad period integrals, and this article is a starting point of this project. The $p$-primitivity of Eisenstein series is an essential property in the automorphic method in Iwasawa theory.

math.NT↗