A software for learning Information Theory basics with emphasis on Entropy of Spanish

Fabio G. Guerrero; Lucio A. Perez

Home/Archive/arXiv

arXiv · 0709.3262

A software for learning Information Theory basics with emphasis on Entropy of Spanish

Fabio G. Guerrero·Lucio A. Perez

View original source ↗Read source document ↗Export metadata ↓

Abstract

In this paper, a tutorial software to learn Information Theory basics in a practical way is reported. The software, called IT-tutor-UV, makes use of a modern existing Spanish corpus for the modeling of the source. Both the source and the channel coding are also included in this educational tool as part of the learning experience. Entropy values of the Spanish language obtained with the IT-tutor-UV are discussed and compared to others that were previously calculated under limited conditions.

Explore related subjects

cs.IT math.IT

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Fabio G. Guerrero, Lucio A. Perez. 2007-09-20. A software for learning Information Theory basics with emphasis on Entropy of Spanish. https://arxiv.org/abs/0709.3262

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Towards a Measure Theory of Semantic Information

A classic account of the quantification of semantic information is that of Bar-Hiller and Carnap. Their account proposes an inverse relation between the informativeness of a statement and its probability. However, their approach assigns the maximum informativeness to a contradiction: which Floridi refers to as the Bar-Hillel-Carnap paradox. He developed a novel theory founded on a distance metric and parabolic relation, designed to remove this paradox. Unfortunately is approach does not succeed in that aim. In this paper I critique Floridi's theory of strongly semantic information on its own terms and show where it succeeds and fails. I then present a new approach based on the unit circle (a relation that has been the basis of theories from basic trigonometry to quantum theory). This is used, by analogy with von Neumann's quantum probability to construct a measure space for informativeness that meets all the requirements stipulated by Floridi and removes the paradox. In addition, while contradictions and tautologies have zero informativeness, it is found that messages which are contradictory to each other are equally informative. The utility of this is explained by means of an example.

cs.IT↗

arXiv2026

Linear codes over a mixed-alphabet ring and their Gray images with applications to projective and locally repairable codes

Let $m \geq 2$ be an integer, and let $\mathbb{F}_q$ be the finite field of prime power order $q.$ Let $\mathcal{R}=\frac{\mathbb{F}_q[u]}{\langle u^2 \rangle}\times \mathbb{F}_q$ be the mixed-alphabet ring, where $\frac{\mathbb{F}_q[u]}{\langle u^2 \rangle}$ is the quasi-Galois ring with maximal ideal $\langle u\rangle$ of nilpotency index $2$ and residue field $\mathbb{F}_q.$ In this paper, we construct four infinite families of linear codes over the ring $\frac{\mathbb{F}_q[u]}{\langle u^2 \rangle}$ whose defining sets are certain non-empty subsets of $\mathcal{R}^m$ associated with three simplicial complexes of $\mathbb{F}_q^m,$ each possessing a single maximal element. We explicitly determine the parameters and Lee weight distributions of these codes. We also study their Gray images and identify several infinite families of few-weight codes over $\mathbb{F}_q,$ as well as an infinite family of minimal, near-Griesmer and distance-optimal codes over $\mathbb{F}_q.$ We also observe that their Gray images are self-orthogonal codes for $q=2$ or $3.$ We determine spanning matrices of these codes. Leveraging this result, we provide two constructions of infinite families of projective few-weight codes over $\mathbb{F}_q$ with new parameters. As an application of our newly constructed minimal codes over $\mathbb{F}_q,$ we examine the minimal access structures of Masseys secret sharing schemes based on their duals and determine the number of dictatorial participants in these schemes. Finally, we investigate the locality properties of our newly constructed projective codes and show that these codes have locality either $2$ or $3.$ As a consequence, we obtain four infinite families of $q$-ary locally repairable codes (LRCs) with locality $2,$ and two infinite families of binary LRCs with locality $3.$

cs.IT↗

arXiv2026

Information-Theoretic Limits of Fluid Antenna-Aided ISAC with Representation Information Constraints

This paper characterizes the fundamental limits of integrated sensing and communication (ISAC) when the transmitter is subject to a representation-constrained encoder modeled via a mutual information (MI) bottleneck and the receiver employs a fluid antenna system (FAS). Specifically, the message is first encoded into an ideal Gaussian waveform and then mapped by a stochastic encoder into a finite capacity representation that serves as the channel input in the adopted analytical abstraction, while the FAS receiver selects the port with the most favorable channel conditions. From an information-theoretic perspective, we show that the representation bottleneck is equivalent to an additive representation noise, which reduces both the communication and sensing signal-to-noise ratios (SNRs) at the selected port. We then derive the resulting ISAC capacity-distortion region and establish tight converse and achievability bounds under general fading models, including Jakes correlated channels. Leveraging the spatial degrees of freedom (DoF) characterization of the Jakes' model, we prove that the port selection gain is fundamentally constrained by the physical length of the FAS region: the effective diversity order equals the numerical rank of the Jakes' correlation matrix and increases only with the FAS length. Consequently, enlarging the FAS length allows the selected communication SNR to approach the representation-limited ceiling, driving the achievable rate toward its fundamental bound. The sensing mean square error (MSE) similarly approaches its bound when the selected sensing gain also increases with the FAS aperture. Numerical results corroborate the analysis and scaling laws.

cs.IT↗