Search arXiv⌕ Search

arXiv · 0708.2084

Empirical entropy in context

Abstract

We trace the history of empirical entropy, touching briefly on its relation to Markov processes, normal numbers, Shannon entropy, the Chomsky hierarchy, Kolmogorov complexity, Ziv-Lempel compression, de Bruijn sequences and stochastic complexity.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Travis Gagie. 2007-08-15. Empirical entropy in context. https://arxiv.org/abs/0708.2084

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

Bistatic Target Detection by Exploiting Both Deterministic Pilots and Unknown Random Data Payloads

Integrated sensing and communication (ISAC) plays a crucial role in 6G, enabling innovative applications such as drone surveillance, urban air mobility, and low-altitude logistics. However, the hybrid ISAC signal, which comprises deterministic pilot and random data payload components, poses challenges for target detection due to two reasons: 1) these two components cause coupled shifts in both the mean and variance of the received signal, and 2) the random data payloads are typically unknown to the sensing receiver in the bistatic setting. Unfortunately, these challenges could not be tackled by existing target detection algorithms. In this paper, a generalized likelihood ratio test (GLRT)-based detector is derived by leveraging the known deterministic pilots and the statistical characteristics of the unknown random data payloads. Due to the analytical intractability of exact performance characterization, we perform an asymptotic analysis for the false alarm probability and detection probability of the proposed detector. The results reveal a deterministic-random trade-off: the deterministic pilots enhance detection through a target-induced mean shift, while the random data payloads provide an additional covariance-domain target signature but simultaneously introduce statistical uncertainty due to their unknown realizations. Simulations validate the theoretical findings and demonstrate the effectiveness of the proposed detector, which highlights the necessity of designing a dedicated detector to fully exploit the signaling resources assigned to random data payloads.

cs.IT↗

Fisher Information and the Minimum Mean-Square Error for Locating Closely Spaced Point Sources

We determine the optimal average error of locating closely spaced sources from noisy superpositions of a known smooth response with unknown complex coefficients. Relative phases control how changes in one location can be offset by the other unknowns, so local Fisher information alone does not determine the attainable error. For every fixed number $K\ge2$ of sources with independent uniform phases, we prove matching Bayes risk bounds jointly in separation and noise, for all sufficiently small separations and all noise levels. Odd source counts have two risk regimes; even counts have an intermediate regime in which mean squared error is linear in the noise standard deviation. A projected least-squares estimator attains these orders without knowing the phases or the prior density. Bounded nonuniform phase laws preserve the orders under stated support conditions, whereas priors concentrated near poorly identified configurations can change the exponents. A common finite comparison yields both a global inverse bound and paths controlled by the original prior. These give sufficient conditions under which the risk has the order of the pointwise capped Cramér--Rao bound, and a lower-tail calculation explains the power and logarithmic regimes. We also give conditional guarantees when the cluster center, separation, and source count are unknown, with an additional low-noise requirement for count selection. A calibrated covariance factor model illustrates the same criterion beyond source localization.

cs.IT↗

Encoding-Sensitive Nullstellensatz Certificates for Syndrome Exclusion in LDPC Codes: Sharp Local Bounds and a Linear Separation

Certifying that a parity-check matrix has no nonempty set of at most t variable nodes leaving at most b checks unsatisfied arises in minimum-distance and trapping-set analysis. Such absence is the infeasibility of a polynomial system, provable by an independently checkable Nullstellensatz certificate. The degree of these certificates depends strongly on how the two cardinality constraints are encoded, although the encodings have identical solutions. For a local encoding with inclusion-exclusion and cardinality obstructions, we construct explicit certificates of degree at most t + b + K + 1, with K logarithmic in the maximum check degree; this bound is independent of the block length and cannot be improved in general. For a standard counting encoding, a generalized planted-knapsack theorem, which pulls back a knapsack functional with an arbitrary target, gives lower bounds linear in the block length for cycle codes, regular codes of girth at least six near the feasibility boundary, array-based quasi-cyclic codes and minimum-distance certificates for arrays of circulant permutation matrices, where the local degree stays bounded. On array codes and regular codes of girth at least eight, planted column pairs extend these bounds to odd column weight with no unsatisfied checks. The planting argument is optimal among such pullbacks. All computed degrees and certificates are verified in exact rational arithmetic; bounds are distinguished from exact minimum degrees. Local certificates for quasi-cyclic matrices, including a lifted fifth-generation New Radio base graph, pass two independent verifiers.

cs.IT↗