arXiv subjects

David Patterson

Publications and source records attributed to David Patterson.

At least 19 recordsLinked to original sources

arXiv2026

Google's Training Supercomputers from TPU v2 to Ironwood: Architectural Stability, Scale, Resilience, Power Efficiency, and Sustainability Across Five Generations

This paper (to appear in the July/August 2026 issue of IEEE Micro magazine) summarizes five generations of Google s TPUs, from TPU v2 to Ironwood, highlighting their evolution as scalable, resilient, power-efficient, sustainable supercomputers for AI training. It details the TPU s stable architecture, which has surprisingly easily accommodated the rapidly changing deep neural network workloads, such as the rise of Transformers. Key advancements over eight years include 10x increase in HBM capacity and bandwidth per node, a 100x increase in peak node performance, and a 3600x increase in supercomputer performance. The paper also discusses the role of optical circuit switches, built-in self test, and hardware replay in enhancing resilience and how TPU's environmental impact is reduced with substantial improvements in performance per Watt and in carbon emissions per floating point operation. It concludes by identifying six features that may well characterize successful training accelerators of this decade.

cs.AR↗

arXiv2026

Challenges and Research Directions for Large Language Model Inference Hardware

Large Language Model (LLM) inference is hard. The autoregressive Decode phase of the underlying Transformer model makes LLM inference fundamentally different from training. Exacerbated by recent AI trends, the primary challenges are memory and interconnect rather than compute. To address these challenges, we highlight four architecture research opportunities: High Bandwidth Flash for 10X memory capacity with HBM-like bandwidth; Processing-Near-Memory and 3D memory-logic stacking for high memory bandwidth; and low-latency interconnect to speedup communication. While our focus is datacenter AI, we also review their applicability for mobile devices.

cs.AR↗

arXiv2025

Measuring the environmental impact of delivering AI at Google Scale

The transformative power of AI is undeniable - but as user adoption accelerates, so does the need to understand and mitigate the environmental impact of AI serving. However, no studies have measured AI serving environmental metrics in a production environment. This paper addresses this gap by proposing and executing a comprehensive methodology for measuring the energy usage, carbon emissions, and water consumption of AI inference workloads in a large-scale, AI production environment. Our approach accounts for the full stack of AI serving infrastructure - including active AI accelerator power, host system energy, idle machine capacity, and data center energy overhead. Through detailed instrumentation of Google's AI infrastructure for serving the Gemini AI assistant, we find the median Gemini Apps text prompt consumes 0.24 Wh of energy - a figure substantially lower than many public estimates. We also show that Google's software efficiency efforts and clean energy procurement have driven a 33x reduction in energy consumption and a 44x reduction in carbon footprint for the median Gemini Apps text prompt over one year. We identify that the median Gemini Apps text prompt uses less energy than watching nine seconds of television (0.24 Wh) and consumes the equivalent of five drops of water (0.26 mL). While these impacts are low compared to other daily activities, reducing the environmental impact of AI serving continues to warrant important attention. Towards this objective, we propose that a comprehensive measurement of AI serving environmental metrics is critical for accurately comparing models, and to properly incentivize efficiency gains across the full AI serving stack.

cs.AI↗

arXiv2025

Life-Cycle Emissions of AI Hardware: A Cradle-To-Grave Approach and Generational Trends

Specialized hardware accelerators aid the rapid advancement of artificial intelligence (AI), and their efficiency impacts AI's environmental sustainability. This study presents the first publication of a comprehensive AI accelerator life-cycle assessment (LCA) of greenhouse gas emissions, including the first publication of manufacturing emissions of an AI accelerator. Our analysis of five Tensor Processing Units (TPUs) encompasses all stages of the hardware lifespan - from raw material extraction, manufacturing, and disposal, to energy consumption during development, deployment, and serving of AI models. Using first-party data, it offers the most comprehensive evaluation to date of AI hardware's environmental impact. We include detailed descriptions of our LCA to act as a tutorial, road map, and inspiration for other computer engineers to perform similar LCAs to help us all understand the environmental impacts of our chips and of AI. A byproduct of this study is the new metric compute carbon intensity (CCI) that is helpful in evaluating AI hardware sustainability and in estimating the carbon footprint of training and inference. This study shows that CCI improves 3x from TPU v4i to TPU v6e. Moreover, while this paper's focus is on hardware, software advancements leverage and amplify these gains.

cs.AR↗

arXiv2024

Shaping AI's Impact on Billions of Lives

Artificial Intelligence (AI), like any transformative technology, has the potential to be a double-edged sword, leading either toward significant advancements or detrimental outcomes for society as a whole. As is often the case when it comes to widely-used technologies in market economies (e.g., cars and semiconductor chips), commercial interest tends to be the predominant guiding factor. The AI community is at risk of becoming polarized to either take a laissez-faire attitude toward AI development, or to call for government overregulation. Between these two poles we argue for the community of AI practitioners to consciously and proactively work for the common good. This paper offers a blueprint for a new type of innovation infrastructure including 18 concrete milestones to guide AI research in that direction. Our view is that we are still in the early days of practical AI, and focused efforts by practitioners, policymakers, and other stakeholders can still maximize the upsides of AI and minimize its downsides. We talked to luminaries such as recent Nobelist John Jumper on science, President Barack Obama on governance, former UN Ambassador and former National Security Advisor Susan Rice on security, philanthropist Eric Schmidt on several topics, and science fiction novelist Neal Stephenson on entertainment. This ongoing dialogue and collaborative effort has produced a comprehensive, realistic view of what the actual impact of AI could be, from a diverse assembly of thinkers with deep understanding of this technology and these domains. From these exchanges, five recurring guidelines emerged, which form the cornerstone of a framework for beginning to harness AI in service of the public good. They not only guide our efforts in discovery but also shape our approach to deploying this transformative technology responsibly and ethically.

cs.AI↗

arXiv2024

Rotationally Resolved Spectroscopy of a Single Polyatomic Molecule

We report the rotationally resolved spectrum of a single polyatomic molecular ion in the gas phase. Building upon the recently developed inelastic recoil spectroscopy (IRS) technique, we have achieved a spectral resolution sufficient to observe resolved rotational-vibrational transitions of a trapped cyclopropenyl cation, c-C3H3+. The high precision of IRS shown in this work presents an attractive platform for astrochemistry, single molecule chiral detection, and tests of fundamental physics.

physics.chem-ph↗

arXiv2023

Non-destructive inelastic recoil spectroscopy of a single molecular ion: a versatile tool toward precision action spectroscopy

We demonstrate a novel single molecule technique that is compatible with high precision measurements and obtain the spectrum of two molecular ion species. While the current result yields modest spectral resolution due to a broad light source, we expect the method to ultimately provide resolution comparable to quantum logic methods with significantly less stringent requirements. Adaptations of this technique will prove useful in a wide range of precision spectroscopy arenas including the search for parity violating effects in chiral molecules.

physics.chem-ph↗

arXiv2023

Fifty Years of ISCA: A data-driven retrospective on key trends

Computer Architecture, broadly, involves optimizing hardware and software for current and future processing systems. Although there are several other top venues to publish Computer Architecture research, including ASPLOS, HPCA, and MICRO, ISCA (the International Symposium on Computer Architecture) is one of the oldest, longest running, and most prestigious venues for publishing Computer Architecture research. Since 1973, except for 1975, ISCA has been organized annually. Accordingly, this year will be the 50th year of ISCA. Thus, we set out to analyze the past 50 years of ISCA to understand who and what has been driving and innovating computing systems thus far. Our analysis identifies several interesting trends that reflect how ISCA, and Computer Architecture in general, has grown and evolved in the past 50 years, including minicomputers, general-purpose uniprocessor CPUs, multiprocessor and multi-core CPUs, general-purpose GPUs, and accelerators.

cs.AR↗

arXiv2023

TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings

In response to innovations in machine learning (ML) models, production workloads changed radically and rapidly. TPU v4 is the fifth Google domain specific architecture (DSA) and its third supercomputer for such ML models. Optical circuit switches (OCSes) dynamically reconfigure its interconnect topology to improve scale, availability, utilization, modularity, deployment, security, power, and performance; users can pick a twisted 3D torus topology if desired. Much cheaper, lower power, and faster than Infiniband, OCSes and underlying optical components are <5% of system cost and <3% of system power. Each TPU v4 includes SparseCores, dataflow processors that accelerate models that rely on embeddings by 5x-7x yet use only 5% of die area and power. Deployed since 2020, TPU v4 outperforms TPU v3 by 2.1x and improves performance/Watt by 2.7x. The TPU v4 supercomputer is 4x larger at 4096 chips and thus ~10x faster overall, which along with OCS flexibility helps large language models. For similar sized systems, it is ~4.3x-4.5x faster than the Graphcore IPU Bow and is 1.2x-1.7x faster and uses 1.3x-1.9x less power than the Nvidia A100. TPU v4s inside the energy-optimized warehouse scale computers of Google Cloud use ~3x less energy and produce ~20x less CO2e than contemporary DSAs in a typical on-premise data center.

cs.AR↗

arXiv2022

The Sky Above The Clouds

Technology ecosystems often undergo significant transformations as they mature. For example, telephony, the Internet, and PCs all started with a single provider, but in the United States each is now served by a competitive market that uses comprehensive and universal technology standards to provide compatibility. This white paper presents our view on how the cloud ecosystem, barely over fifteen years old, could evolve as it matures.

cs.DC↗

arXiv2022

The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink

Machine Learning (ML) workloads have rapidly grown in importance, but raised concerns about their carbon footprint. Four best practices can reduce ML training energy by up to 100x and CO2 emissions up to 1000x. By following best practices, overall ML energy use (across research, development, and production) held steady at <15% of Google's total energy use for the past three years. If the whole ML field were to adopt best practices, total carbon emissions from training would reduce. Hence, we recommend that ML papers include emissions explicitly to foster competition on more than just model quality. Estimates of emissions in papers that omitted them have been off 100x-100,000x, so publishing emissions has the added benefit of ensuring accurate accounting. Given the importance of climate change, we must get the numbers right to make certain that we work on its biggest challenges.

cs.LG↗

arXiv2021

Carbon Emissions and Large Neural Network Training

The computation demand for machine learning (ML) has grown rapidly recently, which comes with a number of costs. Estimating the energy cost helps measure its environmental impact and finding greener strategies, yet it is challenging without detailed information. We calculate the energy use and carbon footprint of several recent large models-T5, Meena, GShard, Switch Transformer, and GPT-3-and refine earlier estimates for the neural architecture search that found Evolved Transformer. We highlight the following opportunities to improve energy efficiency and CO2 equivalent emissions (CO2e): Large but sparsely activated DNNs can consume <1/10th the energy of large, dense DNNs without sacrificing accuracy despite using as many or even more parameters. Geographic location matters for ML workload scheduling since the fraction of carbon-free energy and resulting CO2e vary ~5X-10X, even within the same country and the same organization. We are now optimizing where and when large models are trained. Specific datacenter infrastructure matters, as Cloud datacenters can be ~1.4-2X more energy efficient than typical datacenters, and the ML-oriented accelerators inside them can be ~2-5X more effective than off-the-shelf systems. Remarkably, the choice of DNN, datacenter, and processor can reduce the carbon footprint up to ~100-1000X. These large factors also make retroactive estimates of energy cost difficult. To avoid miscalculations, we believe ML papers requiring large computational resources should make energy consumption and CO2e explicit when practical. We are working to be more transparent about energy use and CO2e in our future research. To help reduce the carbon footprint of ML, we believe energy usage and CO2e should be a key metric in evaluating models, and we are collaborating with MLPerf developers to include energy usage during training and inference in this industry standard benchmark.

cs.LG↗

arXiv2020

Ultralong spin coherence times for rubidium atoms in solid parahydrogen via dynamical decoupling

Coherence time is an essential parameter for quantum sensing, quantum information, and quantum computation. In this work, we demonstrate electron spin coherence times as long as 0.1 s for an ensemble of rubidium atoms trapped in a solid parahydrogen matrix. We explore the underlying physics limiting the coherence time. The properties of these matrix isolated atoms are very promising for future applications, including quantum sensing of nuclear spins. If combined with efficient single-atom readout, this would enable NMR and magnetic resonance imaging of single molecules cotrapped with alkali-metal atom quantum sensors within a parahydrogen matrix.

physics.atom-ph↗

arXiv2020

Benchmarking TinyML Systems: Challenges and Direction

Recent advancements in ultra-low-power machine learning (TinyML) hardware promises to unlock an entirely new class of smart applications. However, continued progress is limited by the lack of a widely accepted benchmark for these systems. Benchmarking allows us to measure and thereby systematically compare, evaluate, and improve the performance of systems and is therefore fundamental to a field reaching maturity. In this position paper, we present the current landscape of TinyML and discuss the challenges and direction towards developing a fair and useful hardware benchmark for TinyML workloads. Furthermore, we present our four benchmarks and discuss our selection methodology. Our viewpoints reflect the collective thoughts of the TinyMLPerf working group that is comprised of over 30 organizations.

cs.PF↗

arXiv2019

Spin coherence and optical properties of alkali-metal atoms in solid parahydrogen

We present a joint experimental and theoretical study of spin coherence properties of 39K, 85Rb, 87Rb, and 133Cs atoms trapped in a solid parahydrogen matrix. We use optical pumping to prepare the spin states of the implanted atoms and circular dichroism to measure their spin states. Optical pumping signals show order-of-magnitude differences depending on both matrix growth conditions and atomic species. We measure the ensemble transverse relaxation times (T2*) of the spin states of the alkali-metal atoms. Different alkali species exhibit dramatically different T2* times, ranging from sub-microsecond coherence times for high mF states of 87Rb, to ~100 microseconds for 39K. These are the longest ensemble T2* times reported for an electron spin system at high densities (n > 10^16 cm^-3). To interpret these observations, we develop a theory of inhomogenous broadening of hyperfine transitions of ^2S atoms in weakly-interacting solid matrices. Our calculated ensemble transverse relaxation times agree well with experiment, and suggest ways to longer coherence times in future work.

physics.atom-ph↗

arXiv2019

MLPerf Training Benchmark

Machine learning (ML) needs industry-standard performance benchmarks to support design and competitive evaluation of the many emerging software and hardware solutions for ML. But ML training presents three unique benchmarking challenges absent from other domains: optimizations that improve training throughput can increase the time to solution, training is stochastic and time to solution exhibits high variance, and software and hardware systems are so diverse that fair benchmarking with the same binary, code, and even hyperparameters is difficult. We therefore present MLPerf, an ML benchmark that overcomes these challenges. Our analysis quantitatively evaluates MLPerf's efficacy at driving performance and scalability improvements across two rounds of results from multiple vendors.

cs.LG↗

arXiv2019

High Sensitivity Microwave Spectroscopy in a Cryogenic Buffer Gas Cell

We describe an instrument which can be used to analyze complex chemical mixtures at high resolution and high sensitivity. Molecules are collisionally cooled with helium gas at cryogenic temperatures (~ 4-7 K), and subsequently detected using chirped pulse microwave spectroscopy. Here we demonstrate three significant improvements to the apparatus relative to an earlier version: (1) extension of its operating range by more than a factor of two, from 12-18 GHz to 12-26 GHz, which allows a much wider range of species to be characterized; (2) improved detection sensitivity owing to use of cryogenically-cooled low-noise amplifiers and protection switches, and (3) a versatile method of sample input that enables analysis of solids, liquids, gases, and solutions, without the need for chemical separation (as demonstrated with a 12-16 GHz spectrum of lemon oil). This instrument can record broadband microwave spectra at comparable sensitivity to high Q cavity spectrometers which use pulsed supersonic jets, but up to 3000 times faster with a modest increase in sample consumption rate.

physics.ins-det↗

arXiv2018

Automated, context-free assignment of asymmetric rotor microwave spectra

We present a new algorithm, Robust Automated Assignment of Rigid Rotors (RAARR), for assigning rotational spectra of asymmetric tops. The RAARR algorithm can automatically assign experimental spectra under a broad range of conditions, including spectra comprised of multiple mixture components, in about 100 seconds or less. The RAARR algorithm exploits constraints placed by the conservation of energy to find sets of connected lines in an unassigned spectrum. The highly constrained structure of these sets eliminates all but a handful of plausible assignments for a given set, greatly reducing the number of potential assignments that must be evaluated. We successfully apply our algorithm to automatically assign 15 experimental spectra, including 5 previously unassigned species, without prior estimation of molecular rotational constants. In 9 of the 15 cases, the RAARR algorithm successfully assigns two or more mixture components.

physics.chem-ph↗