Search arXiv⌕ Search

SEARCH · Search arXiv

Results for “cs.IT”

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

5,200 records · Page 3Linked to original sources
arXiv2026

Spectral Theory of Semisimple Bivariate Bicycle Codes

Extending the classical theory of two-dimensional cyclic codes, we develop an algebraic approach to bivariate bicycle codes. Using Frobenius-orbit idempotents, formulas for logical dimensions are derived and lower bounds on minimum distances are established. A systematic theory of code symmetries is formulated to construct a structured block-monomial subgroup of coordinate permutations. Several explicit examples show how to generate these codes from first principles without relying on numerical searches. An appendix extends the analysis to BCH-based product constructions.

quant-ph↗
arXiv2026

Linear Coding of LTI Sources Over Vector Gaussian Channels: A Majorization Approach

We study the design of linear time-invariant (LTI) encoder-decoder pairs for transmitting the state of a discrete-time LTI vector source over power-constrained parallel Gaussian channels with feedback. Two types of power constraints are considered. Under individual subchannel power constraints, a necessary and sufficient condition for designing an encoder-decoder pair that achieves bounded estimation error covariance (EEC) is established via two coupled majorization inequalities involving the subchannel signal-to-noise ratios and the antistable poles of the source. Under total channel power constraint, we derive the minimum total power required for a feasible encoder-decoder design by exploiting partial-order progamming under majorization order. An analytical optimal power allocation is obtained for the case of equal noise variances, which admits a water-filling interpretation; for general noise case, a sequential water-filling algorithm is developed. Our results reveal that the difficulty of transmitting a discrete-time LTI source via LTI coding is governed not only by its topological entropy, but also by the evenness of the log-magnitudes of its antistable poles. The design methods for feasible encoder-decoder pairs are also provided.

cs.IT↗
arXiv2023

High-dimensional Expansion of Product Codes is Stronger than Robust and Agreement Testability

We study the coboundary expansion property of product codes called product expansion, which played a key role in all recent constructions of good qLDPC codes. It was shown before that this property is equivalent to robust testability and agreement testability for products of two codes with linear distance. First, we show that robust testability for product of many codes with linear distance is equivalent to agreement testability. Second, we provide an example of product of three codes with linear distance which is robustly testable but not product expanding.

cs.IT↗
arXiv2026

New classes of trace-form permutation polynomials and their compositional inverses

We construct permutation polynomials of the form x + γTr^{q^2}_q(h(x)) over the finite field F_{q^2}. More precisely, we present two families ofpermutation polynomials whose coefficients range over all elements of the field rather than being restricted to a proper subfield. We also compute the compositional inverses of both families. Our techniques involve the evaluation of certain Kloosterman sums together with the analysis of some equations over finite fields, and we believe these can be of independent interest.

math.NT↗
arXiv2026

Ada-TokenCom: Rate-Adaptive Token Communications via Large-Model-Driven Token Compression and Generation

Token Communications (TokenCom) has recently emerged as a new paradigm in which tokens serve as unified units for communication and computation, enabling efficient multimodal semantic and goal-oriented transmission. In this paper, we develop Ada-TokenCom, a rate-adaptive TokenCom framework based on large autoregressive models, which integrates next-token prediction with arithmetic coding to achieve ultra-low bitrate semantic communication at the token level. We propose a mixed reconstruction/generation scheme, where the transmitter encodes and transmits the highly informative tokens at the beginning of the token sequence leveraging a pre-trained autoregressive large model, while the receiver uses an identical model to predict the rest. Moreover, we design a Lyapunov-based algorithm to dynamically optimize both the source compression rate and the modulation and coding scheme, adapting to time-varying network conditions. Simulation results demonstrate that our proposed Ada-TokenCom framework outperforms both digital and deep joint source-channel coding-based semantic communication baselines.

cs.IT↗
arXiv2026

Joint Accuracy and Confidentiality in Semantic-Aware Secure Remote Reconstruction

In this paper, we consider remote reconstruction over wireless networks when simultaneous accuracy at the legitimate receiver and confidentiality against eavesdropping are required. These two objectives are often treated separately, even though they arise from the same update process and are marginals of a joint reconstruction event. This paper introduces confidential reconstruction accuracy (CRA), a metric to capture the joint event in which the legitimate receiver reconstructs correctly while the eavesdropper fails. Under randomized stationary policies, we develop a three-dimensional stationary analysis and derive closed-form expressions for the long-term average CRA and the optimal transmission probability. The results show that conventional marginal analysis can misidentify the optimal policy and misestimate the achievable simultaneous accuracy-confidentiality performance. They also reveal nontrivial behaviors: more frequent transmissions or better legitimate channels do not necessarily improve joint accurate and confidential reconstruction, and when the eavesdropping channel is strong, improving the legitimate channel alone may be insufficient. Finally, the framework induces the spatial safety boundary in a geofencing setting for secure remote reconstruction.

cs.IT↗
arXiv2026

The Entropy of Floating-Point Numbers

Here we present an analytic approximation for the entropy of floating-point numbers, along with bounds on the error of this approximation. It is well-known that the differential entropy is tightly linked to the discrete entropy of a uniformly quantized random variable. Our approximation uncovers a different quantity that provides this link for floating-point quantization. Additionally, we prove that the entropy of a floating-point quantized random variable is approximately unchanged under scaling. Closed-form expressions for the floating-point entropy of common distributions are provided and compared to exact results.

cs.IT↗
arXiv2026

Identification for ISI Gaussian Channels

We establish lower and upper bounds for the identification capacity of discrete-time Gaussian channels subject to inter-symbol interference (ISI), a canonical model in wireless communication. Our analysis accounts for deterministic encoders under peak power constraint. A principal finding is that, even when the number of ISI taps scales sub-linearly with the codeword length $n$ as $\sim n^κ$ with $κ\in [0,1/2),$ the number of messages that can be reliably identified grows super-exponentially in $n$ as $\sim 2^{(n \log n)R},$ where $R$ denotes the coding rate.

cs.IT↗
arXiv2026

Give it Space! Explicit Disentangling of Positional and Semantic Representations in Encoders

Positional encoding (PE) underpins how permutation-invariant Transformers represent sequence order, yet how positional information is processed and stored remains poorly understood. Modern PE methods such as RoPE still struggle on tasks such as long-context understanding or retrieval \cite{chen-etal-2025-hope}. Hence, a better understanding of the internal positional mechanism could help design better PE. Building on evidence that positional and semantic signals occupy nearly orthogonal subspaces in trained Transformers, we modify an encoder Transformer to process three explicitly disentangled streams: semantic, absolute positional (AP) and relative positional (RP), and confine the masked-language-modeling (MLM) objective to the semantic stream. This decoupling enables a clean mechanistic study and yields three take-aways. (1) The isolated AP subspace spontaneously collapses into a low-frequency two-dimensional manifold that captures the structure of the document; (2) Attention heads specialize into structure and semantic-oriented groups, with RP exclusively supporting the latter; (3) Standard positional encodings do not robustly retain macroscopic structure: RoPE and RP only weakly encode it, and entangled AP loses it in the final layers under MLM pressure. The disentangled approach preserves positional encoding, which improves linguistic representation on 49 of the 65 linguistic phenomena of the Flash-Holmes probing benchmark.

cs.CL↗
arXiv2026

Sharp Restricted Isometry Thresholds for Global Minima of Rank-Restricted Matrix LASSO

We determine the sharp restricted isometry threshold for recovery at global minima of the rank-restricted matrix LASSO. For target rank $r_{\star}$, if the rank-$k$ RIP constant satisfies $δ<δ_{\mathrm{sharp}}(k/r_{\star})$, where $δ_{\mathrm{sharp}}(t)=t/(4-t)$ for $0<t<4/3$ and $δ_{\mathrm{sharp}}(t)=\sqrt{(t-1)/t}$ for $t\ge4/3$, then every global minimizer has Frobenius error $\lesssim\sqrt{r_{\star}}λ$ for all $λ\gtrsim\|\mathcal{A}^{*}(ξ)\|_{\mathrm{op}}$ and at every search rank $r\ge r_{\star}$. The constants depend only on the RIP constant and $t=k/r_{\star}$, and in particular are independent of the search rank. When the rank restriction is inactive, the result specializes to the ordinary convex matrix LASSO. We also obtain the analogous results for sparsity-restricted vector LASSO. Conversely, we show that the threshold $δ<δ_{\mathrm{sharp}}(k/r_{\star})$ cannot be improved, due to the existence of counterexamples whose global minimizers fail to recover the ground truth.

stat.ML↗
arXiv2026

Recovery Models for Linear Batch Codes

Various types of recovery algorithms for batch codes have been considered previously, including asynchronous recovery and recovery algorithms arising from Almost Affinely Disjoint (AAD) families. We initiate a systematic investigation of linear batch codes equipped with recovery algorithms. We introduce online and strongly online batch codes, investigate their relations with asynchronous batch codes, and introduce (m,L)-strongly online batch codes, which provide particularly simple recovery algorithms. We then study simple batch codes associated with arbitrary bipartite graphs, obtaining graph-theoretic conditions for such codes and generalizing several known results.

cs.IT↗
arXiv2026

Shannon's problem on the monotonicity of entropy and a Conjecture of Tao

Let $X_1,X_2,\ldots$ be i.i.d. finitely supported random variables in a torsion-free abelian group, and write $S_k=X_1+\cdots+X_k$, and $H(S_k)$ is the Shannon entropy $S_k$, for all $k \ge 1$. We prove that, for every fixed $n\geq1$, \[ H(S_{n+1})-H(S_n) \geq \frac12\log\frac{n+1}{n} -o_{H(X_1)\to\infty}(1), \] uniformly over the ambient group and the input law. This proves a conjecture of Tao [29] in 2010.

math.PR↗
arXiv2026

Towards a mathematical theory of superposition

We develop a mathematical theory of superposition in neural networks using tools from frame theory and compressed sensing. In our model, a sparse binary vector \(x\) of active features is encoded through an overcomplete dictionary \(W\), and feature recovery is performed by applying \(\operatorname{ReLU}(W^\top W x+b)\) with an appropriate bias vector \(b\). We prove several recovery theorems for this model. In the random-support setting, we establish high-probability support recovery for nearly tight, low-coherence dictionaries, with guarantees when the expected sparsity is up to order \(d/\log n\). In the worst-case support setting, we give a sharp and computable criterion for which sparsity levels permit support recovery. We apply this criterion to Gaussian random matrices and equiangular tight frames. For real equiangular tight frames with \(n>d+1\), we determine the exact recovery threshold in terms of the coherence. The proof of this result for real equiangular tight frames relies on a novel characterization---which should be of independent interest to frame theorists---of the distribution of signs in the Gram matrix.

stat.ML↗
arXiv2026

MultiGait: A Multi-Sensor Multi-Perspective Multi-Session Biometric Inference Benchmark and its Dataset

A lack of suitable datasets has limited the research into the privacy risks of novel smart city sensors, such as thermal cameras, depth cameras, and lidar. Given the number of unsubstantiated privacy claims and their potential widespread deployment into many people's everyday life, understanding the privacy risks of these sensors -- in isolation and in like-for-like comparisons -- is crucial. With MultiGait, we collected the first multi-sensor, multi-perspective, multi-session gait-focused dataset, for the corresponding, and additional more far-reaching investigations. The dataset, validated with multiple state-of-the-art recognition systems, comprises various walking modes and annotated personal attributes for 199 individuals, to ensure the benefit for advanced studies including cross-sensor recognition and anonymization at the edge. MultiGait represents a foundation for rigorous privacy investigations, demonstrated through an extensive identity inference benchmark across eight sensors, four perspectives, and three recording sessions. Our benchmark incidentally reveals that sensors often assumed to be privacy-friendly do still entail considerable identity inference risks, while the poor cross-session generalization of existing methods underscores an important research gap.

cs.CR↗
arXiv2026

Chase-like Decoding: Test-Pattern Design and Performance Analysis

Chase-like decoding algorithms are a popular choice for soft-input decoding of algebraic codes. We evaluate different test-pattern sets for Chase-like decoding. Structured sets, such as Chase-II patterns or patterns chosen by logistic weight, are analyzed using order statistics, while arbitrary sets are evaluated by calculating covered-space probabilities and by performing Monte Carlo simulation. We further propose an algorithm that designs test-pattern sets to cover likely error patterns, achieving comparable performance with half the number of test patterns compared with conventional sets for high-rate BCH codes.

cs.IT↗
arXiv2026

On systematicity of linear function-correcting codes

The standard formulation of function-correcting codes uses systematic encodings. We study the redundancy cost of this constraint when both the prescribed function and the encoding are linear. We introduce the function-separation distance and show that several classical unequal error protection parameters are special cases. For linear functions and encodings, this distance is the first relative generalized Hamming weight of the code relative to the encoded kernel. We formulate free and systematic linear separation problems. We prove that the optimal free redundancy depends only on the rank of the function, and determine the optimal systematic redundancy for prescribed separations $d\leq3$.

cs.IT↗
arXiv2026

What It Costs to Compose, Rebuild, and Correct Precomputed Memory

Language models can answer from precomputed memory, a model's saved reading of a body of material, reused across requests instead of read again at each. This paper maps where that practice preserves correctness and the conditions under which it fails. Across experiments on Llama-3.1-8B-Instruct using both saved key-value caches and trained compressions of them, precomputed memory degrades when assembled from separately prepared parts, stays current only through rebuilds costing a large fraction of full preparation in our measurements, and ignores corrections served beside it conditional on phrasing. If precomputed memories can be served alongside one another, be cost-efficiently rebuilt, and be superseded by new information arriving in real-time, they can serve as a way to avoid re-feeding context to a model over repeated queries. The implication of our results for a deployed system that deals with a variety of queries is that precomputed memories are best rebuilt on the cadence at which new information changes what the memory was originally computed from. Both warm-rebuilding trained compressions of key-value caches and serving specifically-phrased updates beside a memory, as pasted text or injected cache state, show particular promise for keeping precomputed memories current, the latter as an interim measure between rebuilds, and we measure the cost and name the remaining questions associated with each.

cs.CL↗
arXiv2026

Exact Limits of Random Projections for Preserving Geometry: Distance Recovery, Nearest-Neighbor Rankings, and Covariance Shape in Gaussian Models

The Johnson-Lindenstrauss (JL) lemma guarantees that a random projection of $n$ points to $m=O(\varepsilon^{-2}\log n)$ dimensions preserves pairwise squared distances within relative error $\varepsilon$ with high probability, and this dimension order is asymptotically optimal. In high dimensions, however, distances concentrate around a baseline while key geometric information lies in much smaller fluctuations. We show that the JL bound can therefore be uninformative about retained geometry: an independent Gaussian replacement map can satisfy it even though the replacement cloud is independent of the original data. We then ask how well any decoder can recover a feature $f(D)$ of a squared distance $D$ from a linear sketch. Under squared-error loss, the optimal decoder is conditional expectation, so recovery defines a linear operator whose singular values quantify feature recovery. For isotropic Gaussian data ($Σ=σ^2 I_d$), we diagonalize this operator in closed form. For fixed $k$ with $m,d-m\to\infty$, its $k$th singular value satisfies $\ell_k\approx(m/ d)^{k/2}$. This yields three sharp consequences. A rank-$m$ sketch retains at most an $m/d$ fraction of the variance of any feature of one squared distance. If $m\to\infty$ and $m/d\to0$, the expected Kendall correlation is $\frac{2}π\sqrt{m/d}(1+o(1))$; for fixed $q$, nearest- neighbor agreement tends to $1/q$. Yet one projection can satisfy the JL bound while mean Kendall correlation vanishes when $\log n\ll m\ll d$. After removing scale, Haar-averaged retained covariance-shape information is $(m/d)^2$. Thus JL distance preservation does not quantify the geometry available for comparison or inference.

cs.LG↗