arXiv subjects

Haotian Jiang

Publications and source records attributed to Haotian Jiang.

3 recordsLinked to original sources

arXiv2026

Performance-Efficiency Tradeoffs in Transformers: An Approximation Theory Perspective

Transformers have achieved remarkable successes across a wide range of applications, yet the theoretical foundation of their model efficiency remains underexplored. In this work, we investigate how the model parameters -- mainly attention heads and head dimensions -- should be allocated across layers to balance expressivity and efficiency. We first provide mathematical analysis on the role of early layers in information extraction from an approximation perspective, with a theoretical characterization on the trade-off between the number of heads and head dimension under a fixed parameter budget. In addition, we uncover and prove the \emph{saturation} behavior of softmax activations: Continuously increasing head dimensions can lead to diminishing returns in learning errors, particularly for long sequences. Supported by both theory and experiments, this saturation pattern suggests that later layers can operate more efficiently with reduced parameters. Combining these insights, we propose principled strategies for allocating attention heads and dimensions across Transformers' layers, shedding light on theoretically-grounded model efficiency of Transformer-based architectures.

cs.LG↗

arXiv2026

A Simpler Analysis of the Bansal-Jiang Quasi Monte-Carlo Algorithm via Haar Wavelets

Numerical integration---approximating the integral of a function $f$ using $n$ point evaluations---is a central task in science and engineering. The two main paradigms for this problem, the Monte Carlo and quasi-Monte Carlo methods, have distinct strengths and limitations, and a fundamental question is to design a method that combines the benefits of both. \smallskip Building on recent algorithmic advances in discrepancy theory, Bansal and Jiang \cite{BJ25a} gave a randomized QMC method that naturally bridges the MC and QMC error guarantees. Their method also achieves a surprising improvement over the classical Koksma--Hlawka inequality for QMC methods: it attains an error bound of $\widetilde{O}(σ_{\mathsf{SO}}(f)/n)$, where $σ_{\mathsf{SO}}(f)$ is a new notion of \emph{smoothed-out variation} that they introduced and showed to be substantially smaller than the Hardy--Krause variation governing the classical bound. \smallskip However, the analysis in \cite{BJ25a} is quite involved: it must carefully exploit the structure of the dyadic decomposition and the randomness of the algorithm inside a sufficiently fine discretization of the Hlawka--Zaremba formula to obtain cancellations among the high-frequency components in the Fourier decomposition of $f$. The contribution of this article is twofold: (1) We give an equivalent characterization of $σ_{\mathsf{SO}}(f)$ in terms of the Haar--Besov seminorm of $f$, relating this new notion of smoothed-out variation to classical quantities. (2) Through this characterization, we provide a conceptually simpler and more direct analysis of the Bansal--Jiang QMC method via Haar decomposition, bypassing the use of the Hlawka--Zaremba formula, Fourier decomposition, and the delicate cancellation arguments of \cite{BJ25a} that heavily exploit the structure of dyadic decomposition.

cs.DS↗

arXiv2026

An Exposition of the $\widetilde{O}(\log^{1/4} n)$ Bound for the Komlós Problem

A conjecture of Komlós states that the combinatorial discrepancy of any matrix $A\in\mathbb R^{m\times n}$ whose columns have Euclidean norm at most one is bounded by a universal constant. We prove that the combinatorial discrepancy of every such matrix is at most $O((\log n)^{1/4}(\log\log n)^{7/4})$. This is the first asymptotic improvement over the $O(\sqrt{\log n})$ bound established by Banaszczyk [Banaszczyk, Random Struct.\ Algorithms, 1998], and it refutes a conjecture of Hajela [Hajela, European J.\ Combin., 1988] that a lower bound of order $Ω(\sqrt{\log n})$ should hold.

math.CO↗