Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 1912.02012

Variability as a better characterization of Shannon entropy

Gabriele Carcassi·Christine A. Aidala·Julian Barbour
View original source ↗Read source document ↗Export metadata ↓

Abstract

The Shannon entropy, one of the cornerstones of information theory, is widely used in physics, particularly in statistical mechanics. Yet its characterization and connection to physics remain vague, leaving ample room for misconceptions and misunderstanding. We will show that the Shannon entropy can be fully understood as measuring the variability of the elements within a given distribution: it characterizes how much variation can be found within a collection of objects. We will see that it is the only indicator that is continuous and linear, that it quantifies the number of yes/no questions (i.e. bits) that are needed to identify an element within the distribution, and we will see how applying this concept to statistical mechanics in different ways leads to the Boltzmann, Gibbs and von Neumann entropies.

Explore related subjects

cond-mat.stat-mechquant-ph

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Gabriele Carcassi, Christine A. Aidala, Julian Barbour. 2021-02-04. Variability as a better characterization of Shannon entropy. https://doi.org/10.1088/1361-6404%2Fabe361

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

The meaning of entropy (demonstration of a much needed theorem)

The association of information with entropy has been argued on plausibility arguments involving the operation of imaginary engines and beings, and it is not a universal theorem. In this paper, a theorem by Charles Bennett on reversible computation that associates entropy with erasure of information is recognized as this much needed theorem. It is proposed a real, non thermal engine, operated by humans. It is proved: (1) The engine obeys two laws, identical {\it mutatis mutandis} to the two laws of thermodynamics; therefore, the entropy that arises in the operation of the engine has the same meaning of the entropy that arises in the operation of thermal engines. (2) The engine operates in stages similar to the stages in Bennett's three tapes reversible computer; therefore the entropy in the engine has the same meaning of the entropy in computation. The conclusion is that also the thermal entropy is a measure of erased or missing information. As a side result, information is measured in physical units, which complies with Landauer's principle. A prototype at work is shown in video.

cond-mat.stat-mech↗
arXiv2026

Information geometry of perturbed gradient flow systems on hypergraphs: A perspective towards nonequilibrium physics

This article serves to concisely review the link between gradient flow systems on hypergraphs and information geometry which has been established within the last five years. Gradient flow systems describe a wealth of physical phenomena and provide powerful analytical technquies which are based on the variational energy-dissipation principle. Modern nonequilbrium physics has complemented this classical principle with thermodynamic uncertaintly relations, speed limits, entropy production rate decompositions, and many more. In this article, we formulate these modern principles within the framework of perturbed gradient flow systems on hypergraphs. In particular, we discuss the geometry induced by the Bregman divergence, the physical implications of dual foliations, as well as the corresponding infinitesimal Riemannian geometry for gradient flow systems. Through the geometrical perspective, we are naturally led to new concepts such as moduli spaces for perturbed gradient flow systems and thermodynamical area which is crucial for understanding speed limits. We hope to encourage the readers working in either of the two fields to further expand on and foster the interaction between the two fields.

cond-mat.stat-mech↗
arXiv2026

Anomalous diffusion and singular transport from hydrodynamic recoupling

In charge neutral fluids, such as the Dirac fluid in graphene at the Dirac point, charge transport remains diffusive despite the presence of ballistically propagating sound waves: sound waves ``hydrodynamically decouple'' from the slower charge fluctuations. For quasi-one-dimensional charge neutral fluids, we show that this convective charge diffusion is not smoothly connected to the normal diffusion that arises when momentum conservation is broken by noise (or static impurities). Instead, the charge diffusion constant is a discontinuous function of noise, which (in the weak-noise limit) depends only on the ratio of momentum and energy relaxation rates. In the special limit of momentum-conserving noise (e.g., spatially uniform fluctuations of the Hamiltonian), the diffusion constant diverges in the presence of noise. We describe the resulting superdiffusion in terms of coupled Burgers equations. We present a general mechanism---hydrodynamic recoupling---by which weak noise can induce singular changes in transport coefficients. Our results highlight the limits of zero-noise extrapolation for predicting dynamical quantities like diffusion constants.

cond-mat.stat-mech↗