Generalized Measures of Information Transfer

Paul L. Williams; Randall D. Beer

Home/Archive/arXiv

arXiv · 1102.1507

Generalized Measures of Information Transfer

Paul L. Williams·Randall D. Beer

View original source ↗Read source document ↗Export metadata ↓

Abstract

Transfer entropy provides a general tool for analyzing the magnitudes and directions---but not the \emph{kinds}---of information transfer in a system. We extend transfer entropy in two complementary ways. First, we distinguish state-dependent from state-independent transfer, based on whether a source's influence depends on the state of the target. Second, for multiple sources, we distinguish between unique, redundant, and synergistic transfer. The new measures are demonstrated on several systems that extend examples from previous literature.

Explore related subjects

physics.data-an cs.IT math.DS math.IT

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Paul L. Williams, Randall D. Beer. 2011-02-08. Generalized Measures of Information Transfer. https://arxiv.org/abs/1102.1507

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

When Should Team KPIs Be Absolute or Relative for Match-Outcome Prediction?

In rugby union and association football, team key performance indicators (KPIs) can be represented in absolute terms or relative to the opponent. Relativisation sometimes improves match-outcome prediction and sometimes harms it. There has been no general account of when each occurs. The answer depends on how much the two teams differ in variability and how strongly their KPI values rise and fall together. We combine these properties into the Paired Efficiency Factor (PEF), which generalises Fisher's paired-efficiency result to the unequal-variance conditions typical of competitive sport. The PEF also connects a KPI's statistical efficiency to how much information its relative form carries about the outcome. Combined KPIs can interact in complex ways, so we analyse each indicator on its own. Across 86 team KPIs from professional rugby union and association football, the PEF places every metric in one of four regimes. Anti-correlation is common, especially for high-volume competitive counts, and absolute measures are then usually preferable. Relativisation can still improve prediction when a noisier difference carries more outcome information. An idealised simulation and one representative KPI from each regime confirm both signs under team-blocked cross-validation. The same pairing geometry appears in healthcare, genomics, finance, and manufacturing. The PEF turns an ad hoc feature-engineering choice into a transparent, data-informed diagnostic for when to relativise performance metrics and when not to.

physics.data-an↗

arXiv2026

Comparison of Image Processing Models in Quark Gluon Jet Classification

Quark-gluon discrimination provides a useful test case for studying how different machine-learning architectures learn the spatial structure of QCD radiation. In this work, we compare convolutional neural network (CNN), Vision Transformers (ViT), and hierarchical Swin Transformers using the same three-channel jet-image representation, consisting of charged-particle momentum, neutral-particle momentum, and charged-particle multiplicity from PYTHIA 8 jets. We study their performance for different training-set sizes and fine-tuning configurations, with particular attention to the role of local and global information in the jet images. CNN and Swin models consistently perform better than ViT in the cases studied. Since both CNN and Swin retain a strong local component in their architectures, this suggests that local jet substructure plays an important role in quark-gluon discrimination. The performance of the hierarchical Swin model also suggests that combining local features over larger spatial scales is useful. Block-wise fine-tuning improves the performance of the Transformer models, although the improvement becomes smaller and the training less stable as more blocks are unfrozen. We also find that self-supervised Momentum Contrast (MoCo) pretraining improves the model initialization, particularly when the amount of labeled training data is limited. Based on these observations, we developed a smaller Swin model adopted to the jet-image representation used in this study. It achieves comparable performance with substantially fewer parameters. The results show that it is important to adapt the model architecture and training procedure to the specific input characteristics of High Energy Physics (HEP) data when applying vision models in HEP.

physics.data-an↗

arXiv2026

Online local learning for generative thermodynamic computing

Generative thermodynamic computers turn thermal noise into structured data through Langevin dynamics. We train these systems with a local update at each integration step. The reverse-path Onsager-Machlup objective yields a coupling gradient that is a symmetric sum of local residual-state correlations. We apply this gradient immediately rather than accumulating it over a full trajectory. In digital simulations using MNIST prototypes, online and trajectory-batch training reach similar validation losses on fixed noising paths. Models trained online release less heat on average in all five independently seeded pairs, with both models' parameters held fixed during sampling. Auxiliary classifier and nearest-prototype measures change modestly, while pairwise diversity decreases. The response to noise depends strongly on where the errors enter: independent zero-mean errors in the formed updates produce little heat change over a finite range of noise amplitudes, whereas residual offset and temporal correlation have much larger effects. Storing trained couplings requires substantially less precision than resolving deterministic updates during training. Together, these results establish a local online training method and show how update timing, noise structure, and precision affect generative thermodynamic computing.

physics.data-an↗