arXiv subjects

Remy Kusters

Publications and source records attributed to Remy Kusters.

16 recordsLinked to original sources

arXiv2023

Neural-based classification rule learning for sequential data

Discovering interpretable patterns for classification of sequential data is of key importance for a variety of fields, ranging from genomics to fraud detection or more generally interpretable decision-making. In this paper, we propose a novel differentiable fully interpretable method to discover both local and global patterns (i.e. catching a relative or absolute temporal dependency) for rule-based binary classification. It consists of a convolutional binary neural network with an interpretable neural filter and a training strategy based on dynamically-enforced sparsity. We demonstrate the validity and usefulness of the approach on synthetic datasets and on an open-source peptides dataset. Key to this end-to-end differentiable method is that the expressive patterns used in the rules are learned alongside the rules themselves.

cs.LG↗

arXiv2022

Differentiable Rule Induction with Learned Relational Features

Rule-based decision models are attractive due to their interpretability. However, existing rule induction methods often result in long and consequently less interpretable rule models. This problem can often be attributed to the lack of appropriately expressive vocabulary, i.e., relevant predicates used as literals in the decision model. Most existing rule induction algorithms presume pre-defined literals, naturally decoupling the definition of the literals from the rule learning phase. In contrast, we propose the Relational Rule Network (R2N), a neural architecture that learns literals that represent a linear relationship among numerical input features along with the rules that use them. This approach opens the door to increasing the expressiveness of induced decision models by coupling literal learning directly with rule learning in an end-to-end differentiable fashion. On benchmark tasks, we show that these learned literals are simple enough to retain interpretability, yet improve prediction accuracy and provide sets of rules that are more concise compared to state-of-the-art rule induction algorithms.

stat.ML↗

arXiv2021

Discovering PDEs from Multiple Experiments

Automated model discovery of partial differential equations (PDEs) usually considers a single experiment or dataset to infer the underlying governing equations. In practice, experiments have inherent natural variability in parameters, initial and boundary conditions that cannot be simply averaged out. We introduce a randomised adaptive group Lasso sparsity estimator to promote grouped sparsity and implement it in a deep learning based PDE discovery framework. It allows to create a learning bias that implies the a priori assumption that all experiments can be explained by the same underlying PDE terms with potentially different coefficients. Our experimental results show more generalizable PDEs can be found from multiple highly noisy datasets, by this grouped sparsity promotion rather than simply performing independent model discoveries.

stat.ML↗

arXiv2021

Sparsistent Model Discovery

Discovering the partial differential equations underlying spatio-temporal datasets from very limited and highly noisy observations is of paramount interest in many scientific fields. However, it remains an open question to know when model discovery algorithms based on sparse regression can actually recover the underlying physical processes. In this work, we show the design matrices used to infer the equations by sparse regression can violate the irrepresentability condition (IRC) of the Lasso, even when derived from analytical PDE solutions (i.e. without additional noise). Sparse regression techniques which can recover the true underlying model under violated IRC conditions are therefore required, leading to the introduction of the randomised adaptive Lasso. We show once the latter is integrated within the deep learning model discovery framework DeepMod, a wide variety of nonlinear and chaotic canonical PDEs can be recovered: (1) up to $\mathcal{O}(2)$ higher noise-to-sample ratios than state-of-the-art algorithms, (2) with a single set of hyperparameters, which paves the road towards truly automated model discovery.

stat.ML↗

arXiv2021

Fully differentiable model discovery

Model discovery aims at autonomously discovering differential equations underlying a dataset. Approaches based on Physics Informed Neural Networks (PINNs) have shown great promise, but a fully-differentiable model which explicitly learns the equation has remained elusive. In this paper we propose such an approach by integrating neural network-based surrogates with Sparse Bayesian Learning (SBL). This combination yields a robust model discovery algorithm, which we showcase on various datasets. We then identify a connection with multitask learning, and build on it to construct a Physics Informed Normalizing Flow (PINF). We present a proof-of-concept using a PINF to directly learn a density model from single particle data. Our work expands PINNs to various types of neural network architectures, and connects neural network-based surrogates to the rich field of Bayesian parameter inference.

stat.ML↗

arXiv2021

Model discovery in the sparse sampling regime

To improve the physical understanding and the predictions of complex dynamic systems, such as ocean dynamics and weather predictions, it is of paramount interest to identify interpretable models from coarsely and off-grid sampled observations. In this work, we investigate how deep learning can improve model discovery of partial differential equations when the spacing between sensors is large and the samples are not placed on a grid. We show how leveraging physics informed neural network interpolation and automatic differentiation, allow to better fit the data and its spatiotemporal derivatives, compared to more classic spline interpolation and numerical differentiation techniques. As a result, deep learning-based model discovery allows to recover the underlying equations, even when sensors are placed further apart than the data's characteristic length scale and in the presence of high noise levels. We illustrate our claims on both synthetic and experimental data sets where combinations of physical processes such as (non)-linear advection, reaction, and diffusion are correctly identified.

physics.comp-ph↗

arXiv2020

Sparsely constrained neural networks for model discovery of PDEs

Sparse regression on a library of candidate features has developed as the prime method to discover the partial differential equation underlying a spatio-temporal data-set. These features consist of higher order derivatives, limiting model discovery to densely sampled data-sets with low noise. Neural network-based approaches circumvent this limit by constructing a surrogate model of the data, but have to date ignored advances in sparse regression algorithms. In this paper we present a modular framework that dynamically determines the sparsity pattern of a deep-learning based surrogate using any sparse regression technique. Using our new approach, we introduce a new constraint on the neural network and show how a different network architecture and sparsity estimator improve model discovery accuracy and convergence on several benchmark examples. Our framework is available at \url{https://github.com/PhIMaL/DeePyMoD}

cs.LG↗

arXiv2019

Temporal Normalizing Flows

Analyzing and interpreting time-dependent stochastic data requires accurate and robust density estimation. In this paper we extend the concept of normalizing flows to so-called temporal Normalizing Flows (tNFs) to estimate time dependent distributions, leveraging the full spatio-temporal information present in the dataset. Our approach is unsupervised, does not require an a-priori characteristic scale and can accurately estimate multi-scale distributions of vastly different length scales. We illustrate tNFs on sparse datasets of Brownian and chemotactic walkers, showing that the inclusion of temporal information enhances density estimation. Finally, we speculate how tNFs can be applied to fit and discover the continuous PDE underlying a stochastic process.

physics.comp-ph↗

arXiv2019

Geometric localization in supported elastic struts

Localized deformation patterns are a common motif in morphogenesis and are increasingly finding widespread applications in materials science, for instance as memory devices. Here we describe the emergence of spatially localized deformations in a minimal mechanical system by exploring the impact of growth and shear on the conformation of a semi-flexible filament connected to a pliable shearable substrate. We combine numerical simulations of a discrete rod model with theoretical analysis of the differential equations recovered in the continuum limit to quantify (in the form of scaling laws) how geometry, mechanics, and growth act together to give rise to such localized structures in this system. We find that spatially localized deformations along the filament emerge for intermediate shear modulus and increasing growth. Finally, we use experiments on a 3D printed multi-material model system to demonstrate that external control of the amount of shear and growth regulates the spatial extent of the localized strain texture.

cond-mat.soft↗

arXiv2019

DeepMoD: Deep learning for Model Discovery in noisy data

We introduce DeepMoD, a Deep learning based Model Discovery algorithm. DeepMoD discovers the partial differential equation underlying a spatio-temporal data set using sparse regression on a library of possible functions and their derivatives. A neural network approximates the data and constructs the function library, but it also performs the sparse regression. This construction makes it extremely robust to noise, applicable to small data sets, and, contrary to other deep learning methods, does not require a training set. We benchmark our approach on several physical problems such as the Burgers', Korteweg-de Vries and Keller-Segel equations, and find that it requires as few as $\mathcal{O}(10^2)$ samples and works at noise levels up to $75\%$. Motivated by these results, we apply DeepMoD directly on noisy experimental time-series data from a gel electrophoresis experiment and find that it discovers the advection-diffusion equation describing this system.

physics.comp-ph↗

arXiv2017

Active elastohydrodynamics of vesicles in narrow, blind constrictions

Fluid-resistance limited transport of vesicles through narrow constrictions is a recurring theme in many biological and engineering applications. Inspired by the motor-driven movement of soft membrane-bound vesicles into closed neuronal dendritic spines, here we study this problem using a combination of passive three-dimensional simulations and a simplified semi-analytical theory for active transport of vesicles that are forced through such constrictions by molecular motors. We show that the motion of these objects is characterized by two dimensionless quantities related to the geometry and the strength of forcing relative to the vesicle elasticity. We use numerical simulations to characterize the transit time for a vesicle forced by fluid pressure through a constriction in a channel, and find that relative to an open channel, transport into a blind end leads to the formation of an effective lubrication layer that strongly impedes motion. When the fluid pressure forcing is complemented by forces due to molecular motors that are responsible for vesicle trafficking into dendritic spines, we find that the competition between motor forcing and fluid drag results in multistable dynamics reminiscent of the real system. Our study highlights the role of non-local hydrodynamic effects in determining the kinetics of vesicular transport in constricted geometries.

cond-mat.soft↗

arXiv2015

Actin Remodeling and Polymerization Forces Control Dendritic Spine Morphology

Dendritic spines are small membranous structures that protrude from the neuronal dendrite. Each spine contains a synaptic contact site that may connect its parent dendrite to the axons of neighboring neurons. Dendritic spines are markedly distinct in shape and size, and certain types of stimulation prompt spines to evolve, in fairly predictable fashion, from thin nascent morphologies to the mushroom-like shapes associated with mature spines. This striking progression is coincident with the (re)configuration of the neuronal network during early development, learning and memory formation, and has been conjectured to be part of the machinery that encodes these processes at the scale of individual neuronal connections. It is well established that the structural plasticity of spines is strongly dependent upon the actin cytoskeleton inside the spine. A general framework that details the precise role of actin in directing the transitions between the various spine shapes is lacking. We address this issue, and present a quantitative, model-based scenario for spine plasticity validated using realistic and physiologically relevant parameters. Our model points to a crucial role for the actin cytoskeleton. In the early stages of spine formation, the interplay between the elastic properties of the spine membrane and the protrusive forces generated in the actin cytoskeleton propels the incipient spine. In the maturation stage, actin remodeling in the form of the combined dynamics of branched and bundled actin is required to form mature, mushroom-like spines. Our model identifies additional factors that plausibly aid the stabilization and maintenance of spine morphology. Taken together, our model provides unique insights into the fundamental role of actin remodeling and polymerization forces during spine formation and maturation.

physics.bio-ph↗

arXiv2015

Crowding induced clustering under confinement

We present Langevin dynamics simulations that study the collective behavior of driven particles embedded in a densely packed background consisting of passive particles. Depending on the driving force, the densities of driven and passive particles, and the temperature we observe a dynamical phase separation of the driven particles, which cluster together in tight bands. We explore the mechanisms that drive this cluster formation, and determine the critical conditions for such phase separation. A simple physical picture explains the formation and subsequent growth of a jammed zone developing in front of the driven cluster. The model correctly captures the observed scaling with time. We analyze the implications of this clustering transition for the driven transport in dense particulate flows, which due to a non-monotonic dependence on the applied driving force is not straightforwardly optimized. We provide proof-of-concept for a direct application of the clustering effect, and propose a 'colloidal chromatograph'; a setup that permits the separation of colloids by mass or size.

cond-mat.soft↗

arXiv2014

A method for molecular dynamics on curved surfaces

Dynamics simulations of constrained particles can greatly aid in understanding the temporal and spatial evolution of biological processes such as lateral transport along membranes and self-assembly of viruses. Most theoretical efforts in the field of diffusive transport have focussed on solving the diffusion equation on curved surfaces, for which it is not tractable to incorporate particle interactions even though these play a crucial role in crowded systems. We show here that it is possible to combine standard constraint algorithms with the classical velocity Verlet scheme to perform molecular dynamics simulations of particles constrained to an arbitrarily curved surface, in which such interactions can be taken into account. Furthermore, unlike Brownian dynamics schemes in local coordinates, our method is based on Cartesian coordinates allowing for the reuse of many other standard tools without modifications, including parallelisation through domain decomposition. We show that by applying the schemes to the Langevin equation for various surfaces, confined Brownian motion is obtained, which has direct applications to many biological and physical problems. Finally we present two practical examples that highlight the applicability of the method: (i) the influence of crowding and shape on the lateral diffusion of proteins in curved membranes and (ii) the self-assembly of a coarse-grained virus capsid protein model.

physics.comp-ph↗

arXiv2014

Confinement without boundaries: Anisotropic diffusion on the surface of a cylinder

Densely packed systems of thermal particles in curved geometries are frequently encountered in biological and microfluidic systems. In 2D systems, at sufficiently high surface coverage, diffusive motion is widely known to be strongly affected by physical confinement, e.g., by the walls. In this Letter, we explore the effects of confinement by shape, not rigid boundaries, on the diffusion of particles by confining them to the surface of a cylinder. We find that both the magnitude and the directionality of lateral diffusion is strongly influenced by the radius of the cylinder. An anisotropy between diffusion in the longitudinal and circumferential direction of the cylinder develops. We demonstrate that the origin of this effect lies in the fact that screw-like packings of mono- and oligodisperse discs on the surface of a cylinder induce preferential collective motions in the circumferential direction, but also show that even in polydisperse systems lacking such order an intrinsic finite size confinement effect increases diffusivity in the circumferential direction.

cond-mat.stat-mech↗

arXiv2014

Forced transport of deformable containers through narrow constrictions

We study, numerically and analytically, the forced transport of deformable containers through a narrow constriction. Our central aim is to quantify the competition between the constriction geometry and the active forcing, regulating whether and at which speed a container may pass through the constriction and under what conditions it gets stuck. We focus, in particular, on the interrelation between the force that propels the container and the radius of the channel, as these are the external variables that may be directly controlled in both artificial and physiological settings. We present Lattice-Boltzmann simulations that elucidate in detail the various phases of translocation, and present simplified analytical models that treat two limiting types of these membrane containers: deformational energy dominated by the bending or stretching contribution. In either case we find excellent agreement with the full simulations, and our results reveal that not only the radius but also the length of the constriction determines whether or not the container will pass.

cond-mat.soft↗