Deep Neural Networks and Finite Elements of Any Order on Arbitrary Dimensions

Juncai He; Jinchao Xu

Home/Archive/arXiv

arXiv · 2312.14276

Deep Neural Networks and Finite Elements of Any Order on Arbitrary Dimensions

Juncai He·Jinchao Xu

View original source ↗Read source document ↗Export metadata ↓

Abstract

In this study, we establish that deep neural networks employing ReLU and ReLU$^2$ activation functions can effectively represent Lagrange finite element functions of any order on various simplicial meshes in arbitrary dimensions. We introduce two novel formulations for globally expressing the basis functions of Lagrange elements, tailored for both specific and arbitrary meshes. These formulations are based on a geometric decomposition of the elements, incorporating several insightful and essential properties of high-dimensional simplicial meshes, barycentric coordinate functions, and global basis functions of linear elements. This representation theory facilitates a natural approximation result for such deep neural networks. Our findings present the first demonstration of how deep neural networks can systematically generate general continuous piecewise polynomial functions on both specific or arbitrary simplicial meshes.

Explore related subjects

math.NA cs.LG cs.NA

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Juncai He, Jinchao Xu. 2024-01-11. Deep Neural Networks and Finite Elements of Any Order on Arbitrary Dimensions. https://arxiv.org/abs/2312.14276

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

$L^{p}$-convergence of Kantorovich-type Max-Min Neural Network Operators

In this work, we study the Kantorovich variant of max-min neural network operators, in which the operator kernel is defined in terms of sigmoidal functions. Our main aim is to demonstrate the $L^{p}$-convergence of these nonlinear operators for $1\leq p<\infty$, which makes it possible to obtain approximation results for functions that are not necessarily continuous. In addition, we will derive quantitative estimates for the rate of approximation in the $L^{p}$-norm. We will provide some explicit examples, studying the approximation of discontinuous functions with the max-min operator, and varying additionally the underlying sigmoidal function of the kernel. Further, we numerically compare the $L^{p}$-approximation error with the respective error of the Kantorovich variants of other popular neural network operators. As a final application, we show that the Kantorovich variant has advantages compared to the sampling variant of the max-min operator and Kantorovich variant of the max-product operator when it comes to approximate noisy functions as for instance biomedical ECG signals.

math.NA↗

arXiv2026

Quotient geometry of tensor ring decomposition

Differential geometries derived from tensor decompositions have been extensively studied and provided the foundations for a variety of efficient numerical methods. Despite the practical success of the tensor ring (TR) decomposition, its intrinsic geometry remains less understood, primarily due to the underlying ring structure and the resulting nontrivial gauge invariance. We establish the quotient geometry and immersed-submanifold structure of TR decomposition by imposing full-rank conditions on all unfolding matrices of the core tensors and capturing the gauge invariance. The intrinsic ring structure of TR leads to an analysis that is substantially different from other tensor formats. Additionally, for the uniform TR decomposition, where all core tensors are identical and the manifold structure is known, we derive explicit parameterizations for the vertical and horizontal spaces, which enable Riemannian optimization. Numerical experiments validate the developed geometries via tensor ring completion tasks.

math.NA↗

arXiv2026

Boundary elements for clamped Kirchhoff--Love plates

We present a Galerkin boundary element method for clamped Kirchhoff--Love plates with piecewise smooth boundary. It is a direct method based on the representation formula and requires the inversion of the single-layer operator, an application of the double-layer operator to the Dirichlet data, and, in the presence of a vertical load, an application of the Dirichlet trace of the Newton potential to that load. We present trace approximation spaces of arbitrary order, required for both the Dirichlet data and the unknown Neumann trace. Our boundary element method is quasi-optimal with respect to the natural trace norm and achieves optimal convergence order under minimal regularity assumptions. We provide explicit representations of all three integral operators and discuss the implementation of the appearing integrals. Numerical experiments for smooth and non-smooth domains confirm predicted convergence rates.

math.NA↗