Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2404.04862

Efficient and Provably Convergent Computation of Information Bottleneck: A Semi-Relaxed Approach

Lingyi Chen·Shitong Wu·Jiachuan Ye·Huihui Wu·Wenyi Zhang·Hao Wu
View original source ↗Read source document ↗Export metadata ↓

Abstract

Information Bottleneck (IB) is a technique to extract information about one target random variable through another relevant random variable. This technique has garnered significant interest due to its broad applications in information theory and deep learning. Hence, there is a strong motivation to develop efficient numerical methods with high precision and theoretical convergence guarantees. In this paper, we propose a semi-relaxed IB model, where the Markov chain and transition probability condition are relaxed from the relevance-compression function. Based on the proposed model, we develop an algorithm, which recovers the relaxed constraints and involves only closed-form iterations. Specifically, the algorithm is obtained by analyzing the Lagrangian of the relaxed model with alternating minimization in each direction. The convergence property of the proposed algorithm is theoretically guaranteed through descent estimation and Pinsker's inequality. Numerical experiments across classical and discrete distributions corroborate the analysis. Moreover, our proposed algorithm demonstrates notable advantages in terms of computational efficiency, evidenced by significantly reduced run times compared to existing methods with comparable accuracy.

Explore related subjects

cs.ITmath.IT

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Lingyi Chen, Shitong Wu, Jiachuan Ye, Huihui Wu, Wenyi Zhang, Hao Wu. 2024-04-07. Efficient and Provably Convergent Computation of Information Bottleneck: A Semi-Relaxed Approach. https://arxiv.org/abs/2404.04862

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Beyond the "G" Frontier: A Time Traveler's Century-Long Vision for Wireless Intelligence

This article travels one century into the future--from 2025 to 2125--through the analytical lens of the Information--Curvature Efficiency Law (ICEL), an organizing ansatz that reframes wireless capacity around the curvature of the information manifold. It contends that wireless evolution will not proceed through incremental generations such as 6G or 7G, but through a curvature-managed integration of electromagnetics, biology, and thermodynamics. The technical instantiation of ICEL for phase-coded continuous apertures--where curvature is realized as the affine-quotient second derivative of the aperture phase, with a compact synthesis operator and a Fredholm-determinant capacity--is developed rigorously in a companion theory paper and stress-tested against SVD, Fourier, Zernike-like, matched-focus, and RIS baselines in a companion benchmark paper. The present essay supplies the physical intuition, the century-scale narrative, and a set of cross-domain extensions (biology, thermodynamics, ecology) that are explicitly labeled as illustrative extrapolations, not independent derivations.

cs.IT↗
arXiv2026

New lower bounds for kissing numbers in dimensions $25$--$31$

The kissing number in dimension $d$ is the largest number of non-overlapping congruent spheres that can simultaneously touch a central sphere of the same size. We study dimensions $25$-$31$, where the best previous constructions are based on Leech lifting from the optimal kissing configuration in dimension $24$. Our method exploits the absence of contacts between the unlifted bulk and the block consisting of lifted and auxiliary vectors. Rotating this block while keeping the bulk fixed creates room for two antipodal points in dimensions $26$, $27$, and $28$, and one point in dimension $29$. Three further modifications yield improvements in dimensions $25$, $30$ and $31$: (a) a nonorthogonal diagonal linear deformation of the lifted block admits two antipodal points in dimension $25$; (b) rotating the additional coordinates of the lifted vectors and then applying a small orthogonal transformation to the resulting lifted block as a whole admits two antipodal points in dimension $30$; (c) rotating only the additional coordinates of the lifted vectors admits four nonantipodal points in dimension $31$. Together, these constructions yield the new lower bounds $τ_{25}\geq 197058$, $τ_{26}\geq 198552$, $τ_{27}\geq 200046$, $τ_{28}\geq 204522$, $τ_{29}\geq 209497$, $τ_{30}\ge 220442$, and $τ_{31}\geq 238354$.

cs.IT↗
arXiv2026

Minimum distances of primitive narrow-sense BCH codes via good zero-sets

Determining the exact minimum distances of BCH codes remains a open problem. We establish the minimum distances of several families of primitive narrow-sense BCH codes, showing that they attain their designed distances. Our approach centers on $\mathbb{F}_q$-good zero-sets, which we introduce through a derivative condition on their vanishing polynomials. We show that a $q$-ary primitive narrow-sense BCH code of length $q^m-1$ and designed distance $2\leqδ\leq q^m-1$ has minimum distance $δ$ if and only if there exists an $\mathbb{F}_q$-good zero-set of cardinality $δ+1$ in the finite field $\mathbb{F}_{q^m}$ with $q^m$ elements. To construct $\mathbb{F}_q$-good zero-sets, we develop several methods based on polynomial substitutions, power maps, and shifted inverses, as well as direct constructions using polynomials of special forms. Together with suitable initial $\mathbb{F}_q$-good zero-sets, including those arising from known minimum-distance results, these methods yield new good zero-sets of various cardinalities and hence families of primitive narrow-sense BCH codes whose minimum distances equal their designed distances. These families cover a broad range of designed distances, with several known minimum-distance results recovered as special cases.

cs.IT↗