arXiv subjects

J. Nathan Kutz

Publications and source records attributed to J. Nathan Kutz.

At least 145 records · Page 8Linked to original sources

arXiv2021

Extraction of instantaneous frequencies and amplitudes in nonstationary time-series data

Time-series analysis is critical for a diversity of applications in science and engineering. By leveraging the strengths of modern gradient descent algorithms, the Fourier transform, multi-resolution analysis, and Bayesian spectral analysis, we propose a data-driven approach to time-frequency analysis that circumvents many of the shortcomings of classic approaches, including the extraction of nonstationary signals with discontinuities in their behavior. The method introduced is equivalent to a {\em nonstationary Fourier mode decomposition} (NFMD) for nonstationary and nonlinear temporal signals, allowing for the accurate identification of instantaneous frequencies and their amplitudes. The method is demonstrated on a diversity of time-series data, including on data from cantilever-based electrostatic force microscopy to quantify the time-dependent evolution of charging dynamics at the nanoscale.

eess.SP↗

arXiv2021

Data-Driven Stabilization of Periodic Orbits

Periodic orbits are among the simplest non-equilibrium solutions to dynamical systems, and they play a significant role in our modern understanding of the rich structures observed in many systems. For example, it is known that embedded within any chaotic attractor are infinitely many unstable periodic orbits (UPOs) and so a chaotic trajectory can be thought of as `jumping' from one UPO to another in a seemingly unpredictable manner. A number of studies have sought to exploit the existence of these UPOs to control a chaotic system. These methods rely on introducing small, precise parameter manipulations each time the trajectory crosses a transverse section to the flow. Typically these methods suffer from the fact that they require a precise description of the Poincaré mapping for the flow, which is a difficult task since there is no systematic way of producing such a mapping associated to a given system. Here we employ recent model discovery methods for producing accurate and parsimonious parameter-dependent Poincaré mappings to stabilize UPOs in nonlinear dynamical systems. Specifically, we use the sparse identification of nonlinear dynamics (SINDy) method to frame model discovery as a sparse regression problem, which can be implemented in a computationally efficient manner. This approach provides an explicit Poincaré mapping that faithfully describes the dynamics of the flow in the Poincaré section and can be used to identify UPOs. For each UPO, we then determine the parameter manipulations that stabilize this orbit. The utility of these methods are demonstrated on a variety of differential equations, including the Rössler system in a chaotic parameter regime.

math.DS↗

arXiv2021

PySensors: A Python Package for Sparse Sensor Placement

PySensors is a Python package for selecting and placing a sparse set of sensors for classification and reconstruction tasks. Specifically, PySensors implements algorithms for data-driven sparse sensor placement optimization for reconstruction (SSPOR) and sparse sensor placement optimization for classification (SSPOC). In this work we provide a brief description of the mathematical algorithms and theory for sparse sensor optimization, along with an overview and demonstration of the features implemented in PySensors (with code examples). We also include practical advice for user and a list of potential extensions to PySensors. Software is available at https://github.com/dynamicslab/pysensors.

eess.SP↗

arXiv2021

Data-driven discovery of Koopman eigenfunctions for control

Data-driven transformations that reformulate nonlinear systems in a linear framework have the potential to enable the prediction, estimation, and control of strongly nonlinear dynamics using linear systems theory. The Koopman operator has emerged as a principled linear embedding of nonlinear dynamics, and its eigenfunctions establish intrinsic coordinates along which the dynamics behave linearly. Previous studies have used finite-dimensional approximations of the Koopman operator for model-predictive control approaches. In this work, we illustrate a fundamental closure issue of this approach and argue that it is beneficial to first validate eigenfunctions and then construct reduced-order models in these validated eigenfunctions. These coordinates form a Koopman-invariant subspace by design and, thus, have improved predictive power. We show then how the control can be formulated directly in these intrinsic coordinates and discuss potential benefits and caveats of this perspective. The resulting control architecture is termed Koopman Reduced Order Nonlinear Identification and Control (KRONIC). It is demonstrated that these eigenfunctions can be approximated with data-driven regression and power series expansions, based on the partial differential equation governing the infinitesimal generator of the Koopman operator. Validating discovered eigenfunctions is crucial and we show that lightly damped eigenfunctions may be faithfully extracted from EDMD or an implicit formulation. These lightly damped eigenfunctions are particularly relevant for control, as they correspond to nearly conserved quantities that are associated with persistent dynamics, such as the Hamiltonian. KRONIC is then demonstrated on a number of relevant examples, including 1) a nonlinear system with a known linear embedding, 2) a variety of Hamiltonian systems, and 3) a high-dimensional double-gyre model for ocean mixing.

math.OC↗

arXiv2021

Structured Time-Delay Models for Dynamical Systems with Connections to Frenet-Serret Frame

Time-delay embeddings and dimensionality reduction are powerful techniques for discovering effective coordinate systems to represent the dynamics of physical systems. Recently, it has been shown that models identified by dynamic mode decomposition (DMD) on time-delay coordinates provide linear representations of strongly nonlinear systems, in the so-called Hankel alternative view of Koopman (HAVOK) approach. Curiously, the resulting linear model has a matrix representation that is approximately antisymmetric and tridiagonal with a zero diagonal; for chaotic systems, there is an additional forcing term in the last component. In this paper, we establish a new theoretical connection between HAVOK and the Frenet-Serret frame from differential geometry, and also develop an improved algorithm to identify more stable and accurate models from less data. In particular, we show that the sub- and super-diagonal entries of the linear model correspond to the intrinsic curvatures in Frenet-Serret frame. Based on this connection, we modify the algorithm to promote this antisymmetric structure, even in the noisy, low-data limit. We demonstrate this improved modeling procedure on data from several nonlinear synthetic and real-world examples.

math.DS↗

arXiv2021

Learning dominant physical processes with data-driven balance models

Throughout the history of science, physics-based modeling has relied on judiciously approximating observed dynamics as a balance between a few dominant processes. However, this traditional approach is mathematically cumbersome and only applies in asymptotic regimes where there is a strict separation of scales in the physics. Here, we automate and generalize this approach to non-asymptotic regimes by introducing the idea of an equation space, in which different local balances appear as distinct subspace clusters. Unsupervised learning can then automatically identify regions where groups of terms may be neglected. We show that our data-driven balance models successfully delineate dominant balance physics in a much richer class of systems. In particular, this approach uncovers key mechanistic models in turbulence, combustion, nonlinear optics, geophysical fluids, and neuroscience.

physics.flu-dyn↗

arXiv2020

DeepGreen: Deep Learning of Green's Functions for Nonlinear Boundary Value Problems

Boundary value problems (BVPs) play a central role in the mathematical analysis of constrained physical systems subjected to external forces. Consequently, BVPs frequently emerge in nearly every engineering discipline and span problem domains including fluid mechanics, electromagnetics, quantum mechanics, and elasticity. The fundamental solution, or Green's function, is a leading method for solving linear BVPs that enables facile computation of new solutions to systems under any external forcing. However, fundamental Green's function solutions for nonlinear BVPs are not feasible since linear superposition no longer holds. In this work, we propose a flexible deep learning approach to solve nonlinear BVPs using a dual-autoencoder architecture. The autoencoders discover an invertible coordinate transform that linearizes the nonlinear BVP and identifies both a linear operator $L$ and Green's function $G$ which can be used to solve new nonlinear BVPs. We find that the method succeeds on a variety of nonlinear systems including nonlinear Helmholtz and Sturm--Liouville problems, nonlinear elasticity, and a 2D nonlinear Poisson equation. The method merges the strengths of the universal approximation capabilities of deep learning with the physics knowledge of Green's functions to yield a flexible tool for identifying fundamental solutions to a variety of nonlinear systems.

math.NA↗

arXiv2020

Shallow Neural Networks for Fluid Flow Reconstruction with Limited Sensors

In many applications, it is important to reconstruct a fluid flow field, or some other high-dimensional state, from limited measurements and limited data. In this work, we propose a shallow neural network-based learning methodology for such fluid flow reconstruction. Our approach learns an end-to-end mapping between the sensor measurements and the high-dimensional fluid flow field, without any heavy preprocessing on the raw data. No prior knowledge is assumed to be available, and the estimation method is purely data-driven. We demonstrate the performance on three examples in fluid mechanics and oceanography, showing that this modern data-driven approach outperforms traditional modal approximation techniques which are commonly used for flow reconstruction. Not only does the proposed method show superior performance characteristics, it can also produce a comparable level of performance with traditional methods in the area, using significantly fewer sensors. Thus, the mathematical architecture is ideal for emerging global monitoring technologies where measurement data are often limited.

physics.comp-ph↗

arXiv2020

Sparse Principal Component Analysis via Variable Projection

Sparse principal component analysis (SPCA) has emerged as a powerful technique for modern data analysis, providing improved interpretation of low-rank structures by identifying localized spatial structures in the data and disambiguating between distinct time scales. We demonstrate a robust and scalable SPCA algorithm by formulating it as a value-function optimization problem. This viewpoint leads to a flexible and computationally efficient algorithm. Further, we can leverage randomized methods from linear algebra to extend the approach to the large-scale (big data) setting. Our proposed innovation also allows for a robust SPCA formulation which obtains meaningful sparse principal components in spite of grossly corrupted input data. The proposed algorithms are demonstrated using both synthetic and real world data, and show exceptional computational efficiency and diagnostic performance.

stat.ML↗

arXiv2020

Bilinear dynamic mode decomposition for quantum control

Data-driven methods for establishing quantum optimal control (QOC) using time-dependent control pulses tailored to specific quantum dynamical systems and desired control objectives are critical for many emerging quantum technologies. We develop a data-driven regression procedure, bilinear dynamic mode decomposition (biDMD), that leverages time-series measurements to establish quantum system identification for QOC. The biDMD optimization framework is a physics-informed regression that makes use of the known underlying Hamiltonian structure. Further, the biDMD can be modified to model both fast and slow sampling of control signals, the latter by way of stroboscopic sampling strategies. The biDMD method provides a flexible, interpretable, and adaptive regression framework for real-time, online implementation in quantum systems. Further, the method has strong theoretical connections to Koopman theory, which approximates non-linear dynamics with linear operators. In comparison with many machine learning paradigms, it requires minimal data and the biDMD model is easily updated as new data is collected. We demonstrate the efficacy and performance of the approach on a number of representative quantum systems, showing that it also matches experimental results.

quant-ph↗

arXiv2020

Automatic Differentiation to Simultaneously Identify Nonlinear Dynamics and Extract Noise Probability Distributions from Data

The sparse identification of nonlinear dynamics (SINDy) is a regression framework for the discovery of parsimonious dynamic models and governing equations from time-series data. As with all system identification methods, noisy measurements compromise the accuracy and robustness of the model discovery procedure. In this work, we develop a variant of the SINDy algorithm that integrates automatic differentiation and recent time-stepping constrained motivated by Rudy et al. for simultaneously (i) denoising the data, (ii) learning and parametrizing the noise probability distribution, and (iii) identifying the underlying parsimonious dynamical system responsible for generating the time-series data. Thus within an integrated optimization framework, noise can be separated from signal, resulting in an architecture that is approximately twice as robust to noise as state-of-the-art methods, handling as much as 40% noise on a given time-series signal and explicitly parametrizing the noise probability distribution. We demonstrate this approach on several numerical examples, from Lotka-Volterra models to the spatio-temporal Lorenz 96 model. Further, we show the method can identify a diversity of probability distributions including Gaussian, uniform, Gamma, and Rayleigh.

eess.SP↗

arXiv2020

SINDy-PI: A Robust Algorithm for Parallel Implicit Sparse Identification of Nonlinear Dynamics

Accurately modeling the nonlinear dynamics of a system from measurement data is a challenging yet vital topic. The sparse identification of nonlinear dynamics (SINDy) algorithm is one approach to discover dynamical systems models from data. Although extensions have been developed to identify implicit dynamics, or dynamics described by rational functions, these extensions are extremely sensitive to noise. In this work, we develop SINDy-PI (parallel, implicit), a robust variant of the SINDy algorithm to identify implicit dynamics and rational nonlinearities. The SINDy-PI framework includes multiple optimization algorithms and a principled approach to model selection. We demonstrate the ability of this algorithm to learn implicit ordinary and partial differential equations and conservation laws from limited and noisy data. In particular, we show that the proposed approach is several orders of magnitude more noise robust than previous approaches, and may be used to identify a class of complex ODE and PDE dynamics that were previously unattainable with SINDy, including for the double pendulum dynamics and the Belousov Zhabotinsky (BZ) reaction.

cs.LG↗

arXiv2020

Built to Last: Functional and structural mechanisms in the moth olfactory network mitigate effects of neural injury

Most organisms suffer neuronal damage throughout their lives, which can impair performance of core behaviors. Their neural circuits need to maintain function despite injury, which in particular requires preserving key system outputs. In this work, we explore whether and how certain structural and functional neuronal network motifs act as injury mitigation mechanisms. Specifically, we examine how (i) Hebbian learning, (ii) high levels of noise, and (iii) parallel inhibitory and excitatory connections contribute to the robustness of the olfactory system in the Manduca sexta moth. We simulate injuries on a detailed computational model of the moth olfactory network calibrated to in vivo data. The injuries are modeled on focal axonal swellings, a ubiquitous form of axonal pathology observed in traumatic brain injuries and other brain disorders. Axonal swellings effectively compromise spike train propagation along the axon, reducing the effective neural firing rate delivered to downstream neurons. All three of the network motifs examined significantly mitigate the effects of injury on readout neurons, either by reducing injury's impact on readout neuron responses or by restoring these responses to pre-injury levels. These motifs may thus be partially explained by their value as adaptive mechanisms to minimize the functional effects of neural injury. More generally, robustness to injury is a vital design principle to consider when analyzing neural systems.

q-bio.NC↗

arXiv2020

Numerical differentiation of noisy data: A unifying multi-objective optimization framework

Computing derivatives of noisy measurement data is ubiquitous in the physical, engineering, and biological sciences, and it is often a critical step in developing dynamic models or designing control. Unfortunately, the mathematical formulation of numerical differentiation is typically ill-posed, and researchers often resort to an \textit{ad hoc} process for choosing one of many computational methods and its parameters. In this work, we take a principled approach and propose a multi-objective optimization framework for choosing parameters that minimize a loss function to balance the faithfulness and smoothness of the derivative estimate. Our framework has three significant advantages. First, the task of selecting multiple parameters is reduced to choosing a single hyper-parameter. Second, where ground-truth data is unknown, we provide a heuristic for automatically selecting this hyper-parameter based on the power spectrum and temporal resolution of the data. Third, the optimal value of the hyper-parameter is consistent across different differentiation methods, thus our approach unifies vastly different numerical differentiation methods and facilitates unbiased comparison of their results. Finally, we provide an extensive open-source Python library \texttt{pynumdiff} to facilitate easy application to diverse datasets (https://github.com/florisvb/PyNumDiff).

math.DS↗

arXiv2020

Data-Driven Aerospace Engineering: Reframing the Industry with Machine Learning

Data science, and machine learning in particular, is rapidly transforming the scientific and industrial landscapes. The aerospace industry is poised to capitalize on big data and machine learning, which excels at solving the types of multi-objective, constrained optimization problems that arise in aircraft design and manufacturing. Indeed, emerging methods in machine learning may be thought of as data-driven optimization techniques that are ideal for high-dimensional, non-convex, and constrained, multi-objective optimization problems, and that improve with increasing volumes of data. In this review, we will explore the opportunities and challenges of integrating data-driven science and engineering into the aerospace industry. Importantly, we will focus on the critical need for interpretable, generalizeable, explainable, and certifiable machine learning techniques for safety-critical applications. This review will include a retrospective, an assessment of the current state-of-the-art, and a roadmap looking forward. Recent algorithmic and technological trends will be explored in the context of critical challenges in aerospace design, manufacturing, verification, validation, and services. In addition, we will explore this landscape through several case studies in the aerospace industry. This document is the result of close collaboration between UW and Boeing to summarize past efforts and outline future opportunities.

cs.LG↗

arXiv2020

Hierarchical Deep Learning of Multiscale Differential Equation Time-Steppers

Nonlinear differential equations rarely admit closed-form solutions, thus requiring numerical time-stepping algorithms to approximate solutions. Further, many systems characterized by multiscale physics exhibit dynamics over a vast range of timescales, making numerical integration computationally expensive due to numerical stiffness. In this work, we develop a hierarchy of deep neural network time-steppers to approximate the flow map of the dynamical system over a disparate range of time-scales. The resulting model is purely data-driven and leverages features of the multiscale dynamics, enabling numerical integration and forecasting that is both accurate and highly efficient. Moreover, similar ideas can be used to couple neural network-based models with classical numerical time-steppers. Our multiscale hierarchical time-stepping scheme provides important advantages over current time-stepping algorithms, including (i) circumventing numerical stiffness due to disparate time-scales, (ii) improved accuracy in comparison with leading neural-network architectures, (iii) efficiency in long-time simulation/forecasting due to explicit training of slow time-scale dynamics, and (iv) a flexible framework that is parallelizable and may be integrated with standard numerical time-stepping algorithms. The method is demonstrated on a wide range of nonlinear dynamical systems, including the Van der Pol oscillator, the Lorenz system, the Kuramoto-Sivashinsky equation, and fluid flow pass a cylinder; audio and video signals are also explored. On the sequence generation examples, we benchmark our algorithm against state-of-the-art methods, such as LSTM, reservoir computing, and clockwork RNN. Despite the structural simplicity of our method, it outperforms competing methods on numerical integration.

cs.LG↗

arXiv2020

Data-driven Modeling of Rotating Detonation Waves

The direct monitoring of a rotating detonation engine (RDE) combustion chamber has enabled the observation of combustion front dynamics that are composed of a number of co- and/or counter-rotating coherent traveling shock waves whose nonlinear mode-locking behavior exhibit bifurcations and instabilities which are not well understood. Computational fluid dynamics simulations are ubiquitous in characterizing the dynamics of RDE's reactive, compressible flow. Such simulations are prohibitively expensive when considering multiple engine geometries, different operating conditions, and the long-time dynamics of the mode-locking interactions. Reduced-order models (ROMs) provide a critically enabling simulation framework because they exploit low-rank structure in the data to minimize computational cost and allow for rapid parameterized studies and long-time simulations. However, ROMs are inherently limited by translational invariances manifest by the combustion waves present in RDEs. In this work, we leverage machine learning algorithms to discover moving coordinate frames into which the data is shifted, thus overcoming limitations imposed by the underlying translational invariance of the RDE and allowing for the application of traditional dimensionality reduction techniques. We explore a diverse suite of data-driven ROM strategies for characterizing the complex shock wave dynamics and interactions in the RDE. Specifically, we employ the dynamic mode decomposition and a deep Koopman embedding to give new modeling insights and understanding of combustion wave interactions in RDEs.

math.DS↗

arXiv2020

Machine Learning and Feature Engineering for Predicting Pulse Status during Chest Compressions

Objective: Current resuscitation protocols require pausing chest compressions during cardiopulmonary resuscitation (CPR) to check for a pulse. However, pausing CPR during a pulseless rhythm can worsen patient outcome. Our objective is to design an ECG-based algorithm that predicts pulse status during uninterrupted CPR and evaluate its performance. Methods: We evaluated 383 patients being treated for out-of-hospital cardiac arrest using defibrillator data. We collected paired and immediately adjacent ECG segments having an organized rhythm. Segments were collected during the 10s period of ongoing CPR prior to a pulse check, and 5s segments without CPR during the pulse check. ECG segments with or without a pulse were identified by the audio annotation of a paramedic's pulse check findings and recorded blood pressures. We developed an algorithm to predict the clinical pulse status based on the wavelet transform of the bandpass-filtered ECG, applying principle component analysis. We then trained a linear discriminant model using 3 principle component modes. Model performance was evaluated on test group segments with and without CPR using receiver operating curves and according to the initial arrest rhythm. Results: There were 230 patients (540 pulse checks) in the training set and 153 patients (372 pulse checks) in the test set. Overall 38% (351/912) of checks had a spontaneous pulse. The areas under the receiver operating characteristic curve (AUCs) for predicting pulse status with and without CPR on test data were 0.84 and 0.89, respectively. Conclusion: A novel ECG-based algorithm demonstrates potential to improve resuscitation by predicting presence of a spontaneous pulse without pausing CPR. Significance: Our algorithm predicts pulse status during uninterrupted CPR, allowing for CPR to proceed unimpeded by pauses to check for a pulse and potentially improving resuscitation performance.

eess.SP↗