arXiv subjects

Gilbert Strang

Publications and source records attributed to Gilbert Strang.

9 recordsLinked to original sources

arXiv2026

Generalized Inverses of Matrix Products: From Fundamental Subspaces to Randomized Decompositions

We investigate the Moore-Penrose pseudoinverse and generalized inverse of a matrix product $A=CR$ to establish a unifying framework for generalized and randomized matrix inverses. This analysis is rooted in first principles, focusing on the geometry of the four fundamental subspaces. We examine: (1) the reverse order law, $A^+ = R^+C^+$, which holds when $C$ has independent columns and $R$ has independent rows, (2) the universally correct formula, $A^+ = (C^+CR)^+(CRR^+)^+$, providing a geometric interpretation of the mappings between the involved subspaces, (3) a new generalized randomized formula, $A^+_p = (P^TA)^+P^TAQ(AQ)^+$, which gives $A^+_p = A^+$ if and only if the sketching matrices $P$ and $Q$ preserve the rank of $A$, i.e., $\mathrm{rank}(P^TA) = \mathrm{rank}(AQ) = \mathrm{rank}(A)$. The framework is extended to generalized $\{1,2\}$-inverses and specialized forms, revealing the underlying structure of established randomized linear algebra algorithms, including randomized SVD, the Nystr\"om approximation, and CUR decomposition. We demonstrate applications in sparse sensor placement and effective resistance estimation. For the latter, we provide a rigorous quantitative analysis of an approximation scheme, establishing that it always underestimates the true resistance and deriving a worst-case spectral bound on the error of resistance differences.

math.NA↗

arXiv2023

The Pseudoinverse of $A=CR$ is $A^+=R^+C^+$ (?)

This paper gives three formulas for the pseudoinverse of a matrix product $A = CR$. The first is sometimes correct, the second is always correct, and the third is almost never correct. But that third randomized pseudoinverse $A^+_r$ may be very useful when $A$ is a very large matrix. 1. $A^+ = R^+C^+$ when $A = CR$ and $C$ has independent columns and $R$ has independent rows. 2. $A^+ = (C^+CR)^+(CRR^+)^+$ is always correct. 3. $A^+_r = (P^TCR)^+P^TCRQ(CRQ)^+ = A^+$ only when $\mathrm{rank}(P^TA) = \mathrm{rank}(AQ) = \mathrm{rank}(A)$ with $A = CR$.

math.NA↗

arXiv2023

Elimination and Factorization

If a matrix $A$ has rank $r$, then its row echelon form (from elimination) contains the identity matrix in its first $r$ independent columns. How do we \emph{interpret the matrix} $F$ that appears in the remaining columns of that echelon form\,? $F$ multiplies those first $r$ independent columns of $A$ to give its $n-r$ dependent columns. Then $F$ reveals bases for the row space and the nullspace of the original matrix $A$. And $F$ is the key to the column-row factorization $\boldsymbol{A}=\boldsymbol{CR}$.

math.NA↗

arXiv2021

A Neural Network Solves, Explains, and Generates University Math Problems by Program Synthesis and Few-Shot Learning at Human Level

We demonstrate that a neural network pre-trained on text and fine-tuned on code solves mathematics course problems, explains solutions, and generates new questions at a human level. We automatically synthesize programs using few-shot learning and OpenAI's Codex transformer and execute them to solve course problems at 81% automatic accuracy. We curate a new dataset of questions from MIT's largest mathematics courses (Single Variable and Multivariable Calculus, Differential Equations, Introduction to Probability and Statistics, Linear Algebra, and Mathematics for Computer Science) and Columbia University's Computational Linear Algebra. We solve questions from a MATH dataset (on Prealgebra, Algebra, Counting and Probability, Intermediate Algebra, Number Theory, and Precalculus), the latest benchmark of advanced mathematics problems designed to assess mathematical reasoning. We randomly sample questions and generate solutions with multiple modalities, including numbers, equations, and plots. The latest GPT-3 language model pre-trained on text automatically solves only 18.8% of these university questions using zero-shot learning and 30.8% using few-shot learning and the most recent chain of thought prompting. In contrast, program synthesis with few-shot learning using Codex fine-tuned on code generates programs that automatically solve 81% of these questions. Our approach improves the previous state-of-the-art automatic solution accuracy on the benchmark topics from 8.8% to 81.1%. We perform a survey to evaluate the quality and difficulty of generated questions. This work is the first to automatically solve university-level mathematics course questions at a human level and the first work to explain and generate university-level mathematics course questions at scale, a milestone for higher education.

cs.LG↗

arXiv2016

A Local Inverse Formula and a Factorization

When a matrix has a banded inverse there is a remarkable formula that quickly computes that inverse, using only local information in the original matrix. This local inverse formula holds more generally, for matrices with sparsity patterns that are examples of chordal graphs or perfect eliminators. The formula has a long history going back at least as far as the completion problem for covariance matrices with missing data. Maximum entropy estimates, log-determinants, rank conditions, the Nullity Theorem and wavelets are all closely related, and the formula has found wide applications in machine learning and graphical models. We describe that local inverse and explain how it can be understood as a matrix factorization.

math.NA↗

arXiv2015

Random Triangle Theory with Geometry and Applications

What is the probability that a random triangle is acute? We explore this old question from a modern viewpoint, taking into account linear algebra, shape theory, numerical analysis, random matrix theory, the Hopf fibration, and much much more. One of the best distributions of random triangles takes all six vertex coordinates as independent standard Gaussians. Six can be reduced to four by translation of the center to $(0,0)$ or reformulation as a 2x2 matrix problem. In this note, we develop shape theory in its historical context for a wide audience. We hope to encourage other to look again (and differently) at triangles. We provide a new constructive proof, using the geometry of parallelians, of a central result of shape theory: Triangle shapes naturally fall on a hemisphere. We give several proofs of the key random result: that triangles are uniformly distributed when the normal distribution is transferred to the hemisphere. A new proof connects to the distribution of random condition numbers. Generalizing to higher dimensions, we obtain the "square root ellipticity statistic" of random matrix theory. Another proof connects the Hopf map to the SVD of 2 by 2 matrices. A new theorem describes three similar triangles hidden in the hemisphere. Many triangle properties are reformulated as matrix theorems, providing insight to both. This paper argues for a shift of viewpoint to the modern approaches of random matrix theory. As one example, we propose that the smallest singular value is an effective test for uniformity. New software is developed and applications are proposed.

math.HO↗

arXiv2014

The Green's Function for the H\"uckel (Tight Binding) Model

Applications of the H\"uckel (tight binding) model are ubiquitous in quantum chemistry and solid state physics. The matrix representation of this model is isomorphic to an unoriented vertex adjacency matrix of a bipartite graph, which is also the Laplacian matrix plus twice the identity. In this paper, we analytically calculate the determinant and, when it exists, the inverse of this matrix in connection with the Green's function, $\mathbf{G}$, of the $N\times N$ H\"uckel matrix. A corollary is a closed form expression for a Harmonic sum (Eq. 12). We then extend the results to $d-$dimensional lattices, whose linear size is $N$. The existence of the inverse becomes a question of number theory. We prove a new theorem in number theory pertaining to vanishing sums of cosines and use it to prove that the inverse exists if and only if $N+1$ and $d$ are odd and $d$ is smaller than the smallest divisor of $N+1$. We corroborate our results by demonstrating the entry patterns of the Green's function and discuss applications related to transport and conductivity.

math-ph↗

arXiv2011

The Main Diagonal of a Permutation Matrix

By counting 1's in the "right half" of $2w$ consecutive rows, we locate the main diagonal of any doubly infinite permutation matrix with bandwidth $w$. Then the matrix can be correctly centered and factored into block-diagonal permutation matrices. Part II of the paper discusses the same questions for the much larger class of band-dominated matrices. The main diagonal is determined by the Fredholm index of a singly infinite submatrix. Thus the main diagonal is determined "at infinity" in general, but from only $2w$ rows for banded permutations.

math.NA↗

arXiv1993

Wavelet transforms versus Fourier transforms

This note is a very basic introduction to wavelets. It starts with an orthogonal basis of piecewise constant functions, constructed by dilation and translation. The ``wavelet transform'' maps each $f(x)$ to its coefficients with respect to this basis. The mathematics is simple and the transform is fast (faster than the Fast Fourier Transform, which we briefly explain), but approximation by piecewise constants is poor. To improve this first wavelet, we are led to dilation equations and their unusual solutions. Higher-order wavelets are constructed, and it is surprisingly quick to compute with them --- always indirectly and recursively. We comment informally on the contest between these transforms in signal processing, especially for video and image compression (including high-definition television). So far the Fourier Transform --- or its 8 by 8 windowed version, the Discrete Cosine Transform --- is often chosen. But wavelets are already competitive, and they are ahead for fingerprints. We present a sample of this developing theory.

math.NA↗