Qian Cheng | Search arXiv

arXiv2026

Sample-Guided Exact Top-K Selection for Long-Context Sparse Attention

Sparse attention bounds downstream attention work by retaining a fixed-size subset of indexed tokens, but its standalone exact Top-$K$ stage must still process materialized score rows whose length grows with context. Production radix selectors discover their first actionable boundary only after a complete-row pass, forcing another row-scale traversal before exact refinement. We observe that locating a compact upper tail requires substantially less resolution than identifying the exact rank boundary, and that fixed-stride partial views of the current row remain calibrated to the corresponding complete-row rank across ragged lengths. We present HPC-Ops Top-K, a sample-guided exact selector for ragged sparse-attention score rows. A fixed-stride view proposes a row-local coarse boundary; the mandatory complete-row pass certifies its sufficiency, forms the admitted candidate set, and initializes exact FP32 refinement over the unresolved frontier. A nested secondary boundary and exact recovery handle underfilled proposals before any output is committed, so sampling controls common-path work but never correctness. The GPU implementation fuses complete-row certification and candidate formation, and combines persistent, KV-split, and direct-exact execution behind graph-capturable ragged-row dispatch. We evaluate HPC-Ops Top-K on indexer scores from Hy4-Preview. It outperforms the fastest verified external exact baseline by $1.29$--$1.75\times$ across 20 operator configurations, with a $1.55\times$ geometric-mean speedup. It further achieves $1.36\times$ and $1.48\times$ speedups on two framework-derived sparse-attention traces. The implementation is available in HPC-Ops, Tencent's open-source high-performance operator library for LLM inference, at https://github.com/Tencent/hpc-ops.

cs.DC↗

arXiv2026

A Framework Integrating the Dynamic Stiffness Matrix with Physics-Informed Neural Networks for Solving Eigenvalue Problems and Analysing Dynamic Response

This paper introduces a framework that integrates the dynamic stiffness matrix (DSM) with physics-informed neural networks (PINN). The DSM-PINN embeds physical constraints within the model and demonstrates robustness, particularly when addressing limited datasets across diverse investigations. In this approach, deep neural network outputs approximate the displacement fields of element nodes. Unlike the finite element method (FEM), the element shape functions are homogeneous solutions to the governing partial differential equation, forming the basis of the exact dynamic stiffness matrix, thereby avoiding high-order derivative terms. This matrix also serves as a frequency-domain spectral element, resulting in a strong-form PINN. The loss function is produced by connecting neural networks with dynamic stiffness matrices. We focus on utilising PINNs to resolve eigenvalue problems by employing the Wittrick-Williams algorithm, which overcomes the challenge of neural networks failing to converge to higher-order eigenvalues. Additionally, the frequency domain-PINN method is used to analyse structural dynamic responses under moving and impulsive loads, addressing the limitation of neural networks in handling complex numbers. Theoretical convergence stability of the suggested approach is also analysed even DSM is an indefinite matrix after implementing the boundary condition. The numerical results validate the practicality and efficacy of the recommended approach.

math.NA↗