Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 1906.06994

Neural network identifiability for a family of sigmoidal nonlinearities

Verner Vlačić·Helmut Bölcskei
View original source ↗Read source document ↗Export metadata ↓

Abstract

This paper addresses the following question of neural network identifiability: Does the input-output map realized by a feed-forward neural network with respect to a given nonlinearity uniquely specify the network architecture, weights, and biases? Existing literature on the subject Sussman 1992, Albertini, Sontag et al. 1993, Fefferman 1994 suggests that the answer should be yes, up to certain symmetries induced by the nonlinearity, and provided the networks under consideration satisfy certain "genericity conditions". The results in Sussman 1992 and Albertini, Sontag et al. 1993 apply to networks with a single hidden layer and in Fefferman 1994 the networks need to be fully connected. In an effort to answer the identifiability question in greater generality, we derive necessary genericity conditions for the identifiability of neural networks of arbitrary depth and connectivity with an arbitrary nonlinearity. Moreover, we construct a family of nonlinearities for which these genericity conditions are minimal, i.e., both necessary and sufficient. This family is large enough to approximate many commonly encountered nonlinearities to within arbitrary precision in the uniform norm.

Explore related subjects

math.COcs.AIcs.ITmath.CVmath.ITstat.ML

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Verner Vlačić, Helmut Bölcskei. 2020-09-02. Neural network identifiability for a family of sigmoidal nonlinearities. https://arxiv.org/abs/1906.06994

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Rooted Spider Embeddings and the Erd\H os-Sós Conjecture

Under a local density condition, we prove that every $k$-edge spider embeds at any prescribed center of degree at least $k$, unless all legs are even and the host graph has one of two specified structures. These structures contain complete bipartite subgraphs with prescribed neighborhoods. The proof uses path rerouting and three exchange lemmas that describe equality in neighborhood estimates. As a consequence, we recover the Erd\H os-Sós bound for all spiders.

math.CO↗
arXiv2026

Random algebraic constructions for extremal and Ramsey problems

Building on Bukh's random algebraic method, we develop a framework for extremal and Ramsey problems involving apex hypergraphs. If $\mathcal{H}$ is a $(d-1)$-partite $(d-1)$-uniform hypergraph with $S$ edges and $\mathcal{H}(t)$ is obtained by adjoining $t$ vertices with common link $\mathcal{H}$, we prove that $\operatorname{ex}(n,\mathcal{H}(t))=Ω_{\mathcal{H}}(n^{d-1/S})$ for $t>9^{S+o_d(S)}$, which is best possible when $\mathcal{H}$ is Sidorenko. Our framework also yields sharper sided Zarankiewicz bounds, quantitative generalized Tur'an bounds, and diagonal multicolor Ramsey constructions. For each fixed $s\geq 2$ and $K\geq 3$, we further prove $\operatorname{r}_K(\mathcal K_{s,t};\mathcal K_n) =Θ_{s,t,K}((n/\log n)^s)$ for $t>9^{s+o(s)}$, extending a theorem of Alon and Rödl from factorial to exponential $t$. The main ingredients are interpolation on $m$-independent varieties, control of the dependencies imposed by symmetry, and linear spaces of forms whose nonzero members remain regular after a common algebraic slice. Limited edge independence then gives the spectral and local-density estimates needed for the Ramsey application.

math.CO↗
arXiv2026

A note on vertex-critical induced subgraphs of shift graphs

Shift graphs, introduced by Erdős and Hajnal in 1964, form one of the simplest known non-recursive constructions of triangle-free graphs with arbitrarily large chromatic number. In this note, we identify a surprising property: for each integer $k \geq 1$, the smallest $k$-chromatic shift graph contains a \emph{unique} induced $k$-vertex-critical subgraph. We give an explicit description of this subgraph and prove its uniqueness. This provides a new family of vertex-critical triangle-free graphs of arbitrarily large chromatic number.

math.CO↗