Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · cond-mat/9710074

Phase transitions in the generalization behaviour of multilayer perceptrons: II. The influence of noise

B. Schottky·U. Krey
View original source ↗Read source document ↗Export metadata ↓

Abstract

We extend our study of phase transitions in the generalization behaviour of multilayer perceptrons with non-overlapping receptive fields to the problem of the influence of noise, concerning e.g. the input units and/or the couplings between the input units and the hidden units of the second layer (='input noise'), or the final output unit (='output noise'). Without output noise, the output itself is given by a general, permutation-invariant Boolean function of the outputs of the hidden units. As a result we find that the phase transitions, which we found in the deterministic case, mostly persist in the presence of noise. The influence of the noise on the position of the phase transition, as well as on the behaviour in other regimes of the loading parameter $α$, can often be described by a simple rescaling of $α$ depending on strength and type of the noise. We then consider the problem of the optimal noise level for Gibbsian and Bayesian learning, looking on replica symmetry breaking as well. Finally we consider the question why learning with errors is useful at all.

Explore related subjects

cond-mat.dis-nn

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

B. Schottky, U. Krey. 1997-10-08. Phase transitions in the generalization behaviour of multilayer perceptrons: II. The influence of noise. https://doi.org/10.1088/0305-4470%2F30%2F24%2F017

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

R-transforms for non-Hermitian matrices: a spherical integral approach

In this paper, we establish a connection between the formalism of $\mathcal{R}$-transforms for non-Hermitian random matrices and the framework of spherical integrals, using the replica method. This connection was previously proved in the Hermitian setting and in the case of bi-invariant random matrices. We show that the $\mathcal{R}$-transforms used in the non-Hermitian context in fact originate from a single scalar function of two variables. This provides a new and transparent way to compute $\mathcal{R}$-transforms, which until now had been known only in restricted cases such as bi-invariant, Hermitian, or elliptic ensembles.

cond-mat.dis-nn↗
arXiv2026

Spectral boundaries of deterministic matrices deformed by rotationally invariant random non-Hermitian ensembles

One of the great miracles of random matrix theory is that, in the $N \to \infty$ limit, many otherwise intractable matrix problems with horrendously complicated finite-$N$ expressions admit remarkably simple and elegant asymptotic solutions. In this paper, we illustrate this phenomenon in the context of spectral boundaries (or spectral edges) for deformed random matrices. Specifically, we consider matrices of the form $\mathbf{A} + \mathbf{B}$, where $\mathbf{A}$ is a deterministic $N\times N$ matrix (not necessarily Hermitian) and $\mathbf{B}$ is a rotationally invariant random matrix. In the large-$N$ limit, we show that the complex eigenvalue distribution of $\mathbf{A} + \mathbf{B}$ satisfies remarkably simple boundary equations that depend on the $\mathcal{R}_1$ and $\mathcal{R}_2$ transforms of $\mathbf{B}$. We illustrate our results on several explicit random matrix ensembles and support them with numerical simulations.

cond-mat.dis-nn↗
arXiv2026

Electrical conductivity of crack-template-based transparent conducting films: mean-field approximation, effective-medium theory, and simulation

In this work, crack-template-based transparent conducting films were modeled as networks corresponding to the edges of a two-dimensional Poisson--Voronoi diagram. Two types of networks were considered: the original one, in which the conductance of each edge was inversely proportional to its length, and the effective one, in which all edges had the same conductance obtained from the effective-medium theory. The mean-field approximation was used for analytical evaluation of the electrical conductivity. Direct numerical calculations for the Poisson--Voronoi diagram showed that the mean-field approximation overestimated the effective conductivity of the original network by approximately 13\%, and of the effective network by 79\%. In addition, a honeycomb network with an edge conductance distribution corresponding to the Poisson--Voronoi diagram was studied: for it, the predictions of the effective-medium theory turned out to be more accurate than for the Poisson--Voronoi diagram, which was explained by the greater structural homogeneity of the periodic honeycomb lattice. The results indicate that, when modeling crack-template-based transparent conducting films, the application of the mean-field approximation may lead to significant errors if the resistance of individual conductors is not simply proportional to their length. This possibility is discussed as a motivation for future studies of hierarchical cracks with variable width, which are not directly investigated here.

cond-mat.dis-nn↗