arXiv subjects

Timothy Nguyen

Publications and source records attributed to Timothy Nguyen.

18 recordsLinked to original sources

arXiv2026

The Role of Rigor in Artificial Intelligence

Artificial intelligence (AI) has achieved extraordinary capabilities despite lacking many of the conceptual and scientific foundations associated with mature disciplines. Unlike traditional sciences, where reliable technology typically emerges from theoretical understanding, modern AI has progressed largely through performance-driven iteration and "alchemical" experimentation. This tension motivates a systematic analysis of AI through the lens of rigor. We introduce a three-part framework consisting of conceptual rigor (clarifying foundational concepts), epistemic rigor (establishing scientific understanding), and operational rigor (ensuring reliable performance and deployment). Using this framework, we analyze competing conceptions of intelligence and understanding, the strengths and limitations of the empirical approach to deep learning, the power and pitfalls of benchmarks, and the obstacles to theory development posed by modern AI systems. We argue that the distinctive trajectory of AI arises from how forms of rigor interact across paradigms, resulting in the primacy of operational rigor in modern deep learning. This perspective helps explain both AI's rapid advances and its persistent uncertainties, while clarifying the challenges involved in transforming AI into a mature science and reliable technology.

cs.AI↗

arXiv2024

Understanding Transformers via N-gram Statistics

Transformer based large-language models (LLMs) display extreme proficiency with language yet a precise understanding of how they work remains elusive. One way of demystifying transformer predictions would be to describe how they depend on their context in terms of simple template functions. This paper takes a first step in this direction by considering families of functions (i.e. rules) formed out of simple N-gram based statistics of the training data. By studying how well these rulesets approximate transformer predictions, we obtain a variety of novel discoveries: a simple method to detect overfitting during training without using a holdout set, a quantitative measure of how transformers progress from learning simple to more complex statistical rules over the course of training, a model-variance criterion governing when transformer predictions tend to be described by N-gram rules, and insights into how well transformers can be approximated by N-gram rulesets in the limit where these rulesets become increasingly complex. In this latter direction, we find that for 79% and 68% of LLM next-token distributions on TinyStories and Wikipedia, respectively, their top-1 predictions agree with those provided by our N-gram rulesets.

cs.CL↗

arXiv2023

Is forgetting less a good inductive bias for forward transfer?

One of the main motivations of studying continual learning is that the problem setting allows a model to accrue knowledge from past tasks to learn new tasks more efficiently. However, recent studies suggest that the key metric that continual learning algorithms optimize, reduction in catastrophic forgetting, does not correlate well with the forward transfer of knowledge. We believe that the conclusion previous works reached is due to the way they measure forward transfer. We argue that the measure of forward transfer to a task should not be affected by the restrictions placed on the continual learner in order to preserve knowledge of previous tasks. Instead, forward transfer should be measured by how easy it is to learn a new task given a set of representations produced by continual learning on previous tasks. Under this notion of forward transfer, we evaluate different continual learning algorithms on a variety of image classification benchmarks. Our results indicate that less forgetful representations lead to a better forward transfer suggesting a strong correlation between retaining past information and learning efficiency on new tasks. Further, we found less forgetful representations to be more diverse and discriminative compared to their forgetful counterparts.

cs.LG↗

arXiv2022

Architecture Matters in Continual Learning

A large body of research in continual learning is devoted to overcoming the catastrophic forgetting of neural networks by designing new algorithms that are robust to the distribution shifts. However, the majority of these works are strictly focused on the "algorithmic" part of continual learning for a "fixed neural network architecture", and the implications of using different architectures are mostly neglected. Even the few existing continual learning methods that modify the model assume a fixed architecture and aim to develop an algorithm that efficiently uses the model throughout the learning experience. However, in this work, we show that the choice of architecture can significantly impact the continual learning performance, and different architectures lead to different trade-offs between the ability to remember previous tasks and learning new ones. Moreover, we study the impact of various architectural decisions, and our findings entail best practices and recommendations that can improve the continual learning performance.

cs.LG↗

arXiv2021

A Response to Economics as Gauge Theory

We provide an analysis of the recent work by Malaney-Weinstein on "Economics as Gauge Theory" presented on November 10, 2021 at the Money and Banking Workshop hosted by University of Chicago. In particular, we distill the technical mathematics used in their work into a form more suitable to a wider audience. Furthermore, we resolve the conjectures posed by Malaney-Weinstein, revealing that they provide no discernible value for the calculation of index numbers or rates of inflation. Our conclusion is that the main contribution of the Malaney-Weinstein work is that it provides a striking example of how to obscure simple concepts through an uneconomical use of gauge theory.

econ.TH↗

arXiv2021

Dataset Distillation with Infinitely Wide Convolutional Networks

The effectiveness of machine learning algorithms arises from being able to extract useful features from large amounts of data. As model and dataset sizes increase, dataset distillation methods that compress large datasets into significantly smaller yet highly performant ones will become valuable in terms of training efficiency and useful feature extraction. To that end, we apply a novel distributed kernel based meta-learning framework to achieve state-of-the-art results for dataset distillation using infinitely wide convolutional neural networks. For instance, using only 10 datapoints (0.02% of original dataset), we obtain over 65% test accuracy on CIFAR-10 image classification task, a dramatic improvement over the previous best test accuracy of 40%. Our state-of-the-art results extend across many other settings for MNIST, Fashion-MNIST, CIFAR-10, CIFAR-100, and SVHN. Furthermore, we perform some preliminary analyses of our distilled datasets to shed light on how they differ from naturally occurring data.

cs.LG↗

arXiv2020

Dataset Meta-Learning from Kernel Ridge-Regression

One of the most fundamental aspects of any machine learning algorithm is the training data used by the algorithm. We introduce the novel concept of $\epsilon$-approximation of datasets, obtaining datasets which are much smaller than or are significant corruptions of the original training data while maintaining similar model performance. We introduce a meta-learning algorithm called Kernel Inducing Points (KIP) for obtaining such remarkable datasets, inspired by the recent developments in the correspondence between infinitely-wide neural networks and kernel ridge-regression (KRR). For KRR tasks, we demonstrate that KIP can compress datasets by one or two orders of magnitude, significantly improving previous dataset distillation and subset selection methods while obtaining state of the art results for MNIST and CIFAR-10 classification. Furthermore, our KIP-learned datasets are transferable to the training of finite-width neural networks even beyond the lazy-training regime, which leads to state of the art results for neural network dataset distillation with potential applications to privacy-preservation.

cs.LG↗

arXiv2019

Measuring Calibration in Deep Learning

Overconfidence and underconfidence in machine learning classifiers is measured by calibration: the degree to which the probabilities predicted for each class match the accuracy of the classifier on that prediction. How one measures calibration remains a challenge: expected calibration error, the most popular metric, has numerous flaws which we outline, and there is no clear empirical understanding of how its choices affect conclusions in practice, and what recommendations there are to counteract its flaws. In this paper, we perform a comprehensive empirical study of choices in calibration measures including measuring all probabilities rather than just the maximum prediction, thresholding probability values, class conditionality, number of bins, bins that are adaptive to the datapoint density, and the norm used to compare accuracies to confidences. To analyze the sensitivity of calibration measures, we study the impact of optimizing directly for each variant with recalibration techniques. Across MNIST, Fashion MNIST, CIFAR-10/100, and ImageNet, we find that conclusions on the rank ordering of recalibration methods is drastically impacted by the choice of calibration measure. We find that conditioning on the class leads to more effective calibration evaluations, and that using the L2 norm rather than the L1 norm improves both optimization for calibration metrics and the rank correlation measuring metric consistency. Adaptive binning schemes lead to more stablity of metric rank ordering when the number of bins vary, and is also recommended. We open source a library for the use of our calibration measures.

cs.LG↗

arXiv2016

Stochastic Feynman Rules for Yang-Mills Theory on the Plane

We analyze quantum Yang-Mills theory on $\mathbb{R}^2$ using a novel discretization method based on an algebraic analogue of stochastic calculus. Such an analogue involves working with "Gaussian" free fields whose covariance matrix is indefinite rather than positive definite. Specifically, we work with Lie-algebra valued fields on a lattice and exploit an approximate gauge-invariance that is restored when taking the continuum limit. This analysis is applied to show the equivalence between Wilson loop expectations computed using partial axial-gauge, complete axial-gauge, and the Migdal-Witten lattice formulation. As a consequence, we obtain intriguing Lie-theoretic identities involving heat kernels and iterated integrals.

math-ph↗

arXiv2016

Wilson Loop Area Law for 2D Yang-Mills in Generalized Axial Gauge

We prove that Wilson loop expectation values for arbitrary simple closed contours obey an area law up to second order in perturbative two-dimensional Yang-Mills theory. Our analysis occurs within a general family of axial-like gauges, which include and interpolate between holomorphic gauge and the Wu-Mandelstam-Liebrandt light cone gauge. Our methods make use of the homotopy invariance properties of iterated integrals of closed one-forms, which allows us to evaluate the nontrivial integrals occurring at second order. We close with a discussion on complex gauge-fixing and deformation of integration cycles for holomorphic path integrals to shed light on some of the quantum field-theoretic underpinnings of our results.

math-ph↗

arXiv2015

Quantum Yang-Mills Theory in Two Dimensions: Exact versus Perturbative

The standard Feynman diagrammatic approach to quantum field theories assumes that perturbation theory approximates the full quantum theory at small coupling even when a mathematically rigorous construction of the latter is absent. On the other hand, two-dimensional Yang-Mills theory is a rare (if not the only) example of a nonabelian (pure) gauge theory whose full quantum theory has a rigorous construction. Indeed, the theory can be formulated via a lattice approximation, from which Wilson loop expecation values in the continuum limit can be described in terms of heat kernels on the gauge group. It is therefore fundamental to investigate how the exact answer for 2D Yang-Mills compares with that of the continuum perturbative approach, which a priori are unrelated. In this paper, we provide a mathematically rigorous formulation of the perturbative quantization of 2D Yang-Mills, and we consider perturbative Wilson loop expectation values on $\mathbb{R}^2$ and $S^2$ in Coulomb gauge, holomorphic gauge, and axial gauge (on $\mathbb{R}^2$). We show the following equivalences and nonequivalences between these gauges: (i) Coulomb and holomorphic gauge are equivalent and are independent of the choice of gauge-fixing metric; (ii) both are inequivalent with axial-gauge. Additionally, we show that the asymptotics of exact lattice Wilson loop expectations on $S^2$ agree with perturbatively computed expectations in holomorphic gauge for simple closed curves to all orders. However, as a consequence of (ii), this result is necessarily false on $\mathbb{R}^2$. Our work therefore presents fundamental progress in the analysis of how continuum perturbation theory succeeds or fails in capturing the asymptotics of the continuum limit of the lattice theory.

math-ph↗

arXiv2015

The Perturbative Approach to Path Integrals: A Succinct Mathematical Treatment

We study finite-dimensional integrals in a way that elucidates the mathematical meaning behind the formal manipulations of path integrals occurring in quantum field theory. This involves a proper understanding of how Wick's theorem allows one to evaluate integrals perturbatively, i.e., as a series expansion in a formal parameter irrespective of convergence properties. We establish invariance properties of such a Wick expansion under coordinate changes and the action of a Lie group of symmetries, and we use this to study essential features of path integral manipulations, including coordinate changes, Ward identities, Schwinger-Dyson equations, Faddeev-Popov gauge-fixing, and eliminating fields by their equation of motion. We also discuss the asymptotic nature of the Wick expansion and the implications this has for defining path integrals perturbatively and nonperturbatively.

math-ph↗

arXiv2014

Quantization of the Nonlinear Sigma Model Revisited

We revisit the subject of perturbatively quantizing the nonlinear sigma model in two dimensions from a rigorous, mathematical point of view. Our main contribution is to make precise the cohomological problem of eliminating potential anomalies that may arise when trying to preserve symmetries under quantization. The symmetries we consider are twofold: (i) diffeomorphism covariance for a general target manifold; (ii) a transitive group of isometries when the target manifold is a homogeneous space. We show that there are no anomalies in case (i) and that (ii) is also anomaly-free under additional assumptions on the target homogeneous space, in agreement with the work of Friedan. We carry out some explicit computations for the $O(N)$-model. Finally, we show how a suitable notion of the renormalization group establishes the Ricci flow as the one loop renormalization group flow of the nonlinear sigma model.

math-ph↗

arXiv2011

Lagrangian correspondences and Donaldson's TQFT construction of the Seiberg-Witten invariants of 3-manifolds

Using Morse-Bott techniques adapted to the gauge-theoretic setting, we show that the limiting boundary values of the space of finite energy monopoles on a connected 3-manifold with at least two cylindrical ends provides an immersed Lagrangian submanifold of the vortex moduli space at infinity. By studying the signed intersections of such Lagrangians, we supply the analytic details of Donaldson's TQFT construction of the Seiberg-Witten invariants of a closed 3-manifold.

math.DG↗

arXiv2010

Anisotropic Function Spaces and Elliptic Boundary Value Problems

In this paper, we study anisotropic Bessel potential and Besov spaces, where the anisotropy measures the extra amount of regularity in certain directions. Some basic properties of these spaces are established along with applications to elliptic boundary value problems.

math.CA↗

arXiv2010

The Seiberg-Witten Equations on Manifolds with Boundary I: The Space of Monopoles and Their Boundary Values

In this paper, we study the Seiberg-Witten equations on a compact 3-manifold with boundary. Solutions to these equations are called monopoles. Under some simple topological assumptions, we show that the solution space of all monopoles is a Banach manifold in suitable function space topologies. We then prove that the restriction of the space of monopoles to the boundary is a submersion onto a Lagrangian submanifold of the space of connections and spinors on the boundary. Both these spaces are infinite dimensional, even modulo gauge, since no boundary conditions are specified for the Seiberg-Witten equations on the 3-manifold. We study the analytic properties of these monopole spaces with an eye towards developing a monopole Floer theory for 3-manifolds with boundary, which we pursue in Part II.

math.DG↗

arXiv2010

The Seiberg-Witten Equations on Manifolds with Boundary II: Lagrangian Boundary Conditions for a Floer Theory

In this paper, we study the Seiberg-Witten equations on the product R x Y, where Y is a compact 3-manifold with boundary. Following the approach of Salamon and Wehrheim in the instanton case, we impose Lagrangian boundary conditions for the Seiberg- Witten equations. The resulting equations we obtain constitute a nonlinear, nonlocal boundary value problem. We establish regularity, compactness, and Fredholm properties for the Seiberg- Witten equations supplied with Lagrangian boundary conditions arising from the monopole spaces studied in [20]. This work therefore serves as an analytic foundation for the construction of a monopole Floer theory for 3-manifolds with boundary.

math.DG↗

arXiv2008

On discrete features of the wave equation in singular pp-wave backgrounds

We analyze the wave equation in families of pp-wave geometries developing strong localized scale-invariant singularities in certain limits. For both cases of well-localized pp-waves and the so-called null-cosmologies, we observe an intriguing discrete dependence of the existence of a singular limit on the normalization of the pp-wave profile. We also find restrictive matching conditions relating the geometries before and after the singularity (if a singular limit for the solutions of the wave equation with initial conditions specified away from the near-singular region is assumed to exist).

hep-th↗