arXiv subjects

Haizi Yu

Publications and source records attributed to Haizi Yu.

9 recordsLinked to original sources

arXiv2026

Information Lattice Learning as Probabilistic Graphical Model Structure Learning

Information lattice learning (ILL) learns interpretable rules of a signal by alternately projecting the signal onto a partition lattice that encodes a hierarchy of abstractions and lifting selected rules back to the signal domain. When the signal is a probability mass function, we show the probabilistic rules learned by ILL admit a natural probabilistic graphical model (PGM) interpretation and develop this interpretation in detail. A partition in ILL induces a deterministic quotient variable, and a rule is the marginal law of that quotient variable. A rule set is therefore a collection of marginal constraints over interpretable abstractions. General lifting is the feasible family of all joint distributions satisfying those constraints, while special lifting chooses a maximum-ignorance reconstruction, implemented in ILL by an L2 uniformity principle closely related to maximum entropy. Under a Shannon-entropy lifting, the same constraints yield a log-linear factor graph whose factors are indexed by learned abstractions. The information lattice itself, however, is not a Bayesian network: its edges encode refinement and coarsening of abstractions, not conditional dependence. Thus ILL is best viewed as structure learning for interpretable constraint-based factor graphs over quotient variables. This view clarifies how ILL relates to graphical models and maximum entropy models, while suggesting new directions for inference, identifiability, and hybrid symbolic-probabilistic learning.

cs.LG↗

arXiv2024

Semantic Compression with Information Lattice Learning

Data-driven artificial intelligence (AI) techniques are becoming prominent for learning in support of data compression, but are focused on standard problems such as text compression. To instead address the emerging problem of semantic compression, we argue that the lattice theory of information is particularly expressive and mathematically precise in capturing notions of abstraction as a form of lossy semantic compression. As such, we demonstrate that a novel AI technique called information lattice learning, originally developed for knowledge discovery and creativity, is powerful for learning to compress in a semantically-meaningful way. The lattice structure further implies the optimality of group codes and the successive refinement property for progressive transmission.

cs.IT↗

arXiv2022

Accelerated Design and Deployment of Low-Carbon Concrete for Data Centers

Concrete is the most widely used engineered material in the world with more than 10 billion tons produced annually. Unfortunately, with that scale comes a significant burden in terms of energy, water, and release of greenhouse gases and other pollutants; indeed 8% of worldwide carbon emissions are attributed to the production of cement, a key ingredient in concrete. As such, there is interest in creating concrete formulas that minimize this environmental burden, while satisfying engineering performance requirements including compressive strength. Specifically for computing, concrete is a major ingredient in the construction of data centers. In this work, we use conditional variational autoencoders (CVAEs), a type of semi-supervised generative artificial intelligence (AI) model, to discover concrete formulas with desired properties. Our model is trained just using a small open dataset from the UCI Machine Learning Repository joined with environmental impact data from standard lifecycle analysis. Computational predictions demonstrate CVAEs can design concrete formulas with much lower carbon requirements than existing formulations while meeting design requirements. Next we report laboratory-based compressive strength experiments for five AI-generated formulations, which demonstrate that the formulations exceed design requirements. The resulting formulations were then used by Ozinga Ready Mix -- a concrete supplier -- to generate field-ready concrete formulations, based on local conditions and their expertise in concrete design. Finally, we report on how these formulations were used in the construction of buildings and structures in a Meta data center in DeKalb, IL, USA. Results from field experiments as part of this real-world deployment corroborate the efficacy of AI-generated low-carbon concrete mixes.

cs.AI↗

arXiv2022

Learning from One and Only One Shot

Humans can generalize from only a few examples and from little pretraining on similar tasks. Yet, machine learning (ML) typically requires large data to learn or pre-learn to transfer. Motivated by nativism and artificial general intelligence, we directly model human-innate priors in abstract visual tasks such as character and doodle recognition. This yields a white-box model that learns general-appearance similarity by mimicking how humans naturally ``distort'' an object at first sight. Using just nearest-neighbor classification on this cognitively-inspired similarity space, we achieve human-level recognition with only $1$--$10$ examples per class and no pretraining. This differs from few-shot learning that uses massive pretraining. In the tiny-data regime of MNIST, EMNIST, Omniglot, and QuickDraw benchmarks, we outperform both modern neural networks and classical ML. For unsupervised learning, by learning the non-Euclidean, general-appearance similarity space in a $k$-means style, we achieve multifarious visual realizations of abstract concepts by generating human-intuitive archetypes as cluster centroids.

cs.CV↗

arXiv2020

Human Evaluation of Interpretability: The Case of AI-Generated Music Knowledge

Interpretability of machine learning models has gained more and more attention among researchers in the artificial intelligence (AI) and human-computer interaction (HCI) communities. Most existing work focuses on decision making, whereas we consider knowledge discovery. In particular, we focus on evaluating AI-discovered knowledge/rules in the arts and humanities. From a specific scenario, we present an experimental procedure to collect and assess human-generated verbal interpretations of AI-generated music theory/rules rendered as sophisticated symbolic/numeric objects. Our goal is to reveal both the possibilities and the challenges in such a process of decoding expressive messages from AI sources. We treat this as a first step towards 1) better design of AI representations that are human interpretable and 2) a general methodology to evaluate interpretability of AI-discovered knowledge representations.

cs.HC↗

arXiv2019

Orbit Computation for Atomically Generated Subgroups of Isometries of $\mathbb{Z}^n$

Isometries are ubiquitous in nature; isometries of discrete (quantized) objects---abstracted as the group of isometries of $\mathbb{Z}^n$ denoted by $\mathsf{ISO}(\mathbb{Z}^n)$---are important concepts in the computational world. In this paper, we compute various isometric invariances which mathematically are orbit-computation problems under various isometry-subgroup actions $H \curvearrowright \mathbb{Z}^n, H \leq \mathsf{ISO}(\mathbb{Z}^n)$. One computational challenge here is about the \emph{infinite}: in general, we can have an infinite subgroup acting on $\mathbb{Z}^n$, resulting in possibly an infinite number of orbits of possibly infinite size. In practice, we restrict the set of orbits (a partition of $\mathbb{Z}^n$) to a finite subset $Z \subseteq \mathbb{Z}^n$ (a partition of $Z$), where $Z$ is specified a priori by an application domain or a data set. Our main contribution is an efficient algorithm to solve this \emph{restricted} orbit-computation problem in the special case of \emph{atomically generated subgroups}---a new notion partially motivated from interpretable AI. The atomic property is key to preserving the \emph{semidirect-product structure}---the core structure we leverage to make our algorithm outperform generic approaches. Besides algorithmic merit, our approach enables \emph{parallel-computing} implementations in many subroutines, which can further benefit from hardware boosts. Moreover, our algorithm works efficiently for \emph{any} finite subset ($Z$) regardless of the shape (continuous/discrete, (non)convex) or location; so it is application-independent.

math.GR↗

arXiv2018

A Group-Theoretic Approach to Computational Abstraction: Symmetry-Driven Hierarchical Clustering

Abstraction plays a key role in concept learning and knowledge discovery; this paper is concerned with computational abstraction. In particular, we study the nature of abstraction through a group-theoretic approach, formalizing it as symmetry-driven---as opposed to data-driven---hierarchical clustering. Thus, the resulting clustering framework is data-free, feature-free, similarity-free, and globally hierarchical---the four key features that distinguish it from common data clustering models such as $k$-means. Beyond a theoretical foundation for abstraction, we also present a top-down and a bottom-up approach to establish an algorithmic foundation for practical abstraction-generating methods. Lastly, via both a theoretical explanation and a real-world application, we illustrate that further coupling of our abstraction framework with statistics realizes Shannon's information lattice and even further, brings learning into the picture. This not only presents one use case of our proposed computational abstraction, but also gives a first step towards a principled and cognitive way of automatic concept learning and knowledge discovery.

cs.LG↗

arXiv2017

Probabilistic Rule Realization and Selection

Abstraction and realization are bilateral processes that are key in deriving intelligence and creativity. In many domains, the two processes are approached through rules: high-level principles that reveal invariances within similar yet diverse examples. Under a probabilistic setting for discrete input spaces, we focus on the rule realization problem which generates input sample distributions that follow the given rules. More ambitiously, we go beyond a mechanical realization that takes whatever is given, but instead ask for proactively selecting reasonable rules to realize. This goal is demanding in practice, since the initial rule set may not always be consistent and thus intelligent compromises are needed. We formulate both rule realization and selection as two strongly connected components within a single and symmetric bi-convex problem, and derive an efficient algorithm that works at large scale. Taking music compositional rules as the main example throughout the paper, we demonstrate our model's efficiency in not only music realization (composition) but also music interpretation and understanding (analysis).

cs.LG↗

arXiv2016

Learning Interpretable Musical Compositional Rules and Traces

Throughout music history, theorists have identified and documented interpretable rules that capture the decisions of composers. This paper asks, "Can a machine behave like a music theorist?" It presents MUS-ROVER, a self-learning system for automatically discovering rules from symbolic music. MUS-ROVER performs feature learning via $n$-gram models to extract compositional rules --- statistical patterns over the resulting features. We evaluate MUS-ROVER on Bach's (SATB) chorales, demonstrating that it can recover known rules, as well as identify new, characteristic patterns for further study. We discuss how the extracted rules can be used in both machine and human composition.

stat.ML↗