arXiv subjects

Alex Kunin

Publications and source records attributed to Alex Kunin.

5 recordsLinked to original sources

arXiv2022

Braid-based architecture search

In this article, we propose the approach to structural optimization of neural networks, based on the braid theory. The paper describes the basics of braid theory as applied to the description of graph structures of neural networks. It is shown how networks of various topologies can be built using braid structures between layers of neural networks. The operation of a neural network based on the braid theory is compared with a homogeneous deep neural network and a network with random intersections between layers that do not correspond to the ordering of the braids. Results are obtained showing the advantage of braid-based networks over comparable architectures in classification problems.

cs.LG↗

arXiv2021

The principle of weight divergence facilitation for unsupervised pattern recognition in spiking neural networks

Parallels between the signal processing tasks and biological neurons lead to an understanding of the principles of self-organized optimization of input signal recognition. In the present paper, we discuss such similarities among biological and technical systems. We propose adding the well-known STDP synaptic plasticity rule to direct the weight modification towards the state associated with the maximal difference between background noise and correlated signals. We use the principle of physically constrained weight growth as a basis for such weights' modification control. It is proposed that the existence and production of bio-chemical 'substances' needed for plasticity development restrict a biological synaptic straight modification. In this paper, the information about the noise-to-signal ratio controls such a substances' production and storage and drives the neuron's synaptic pressures towards the state with the best signal-to-noise ratio. We consider several experiments with different input signal regimes to understand the functioning of the proposed approach.

q-bio.NC↗

arXiv2021

BraidNet: procedural generation of neural networks for image classification problems using braid theory

In this article, we propose the approach to procedural optimization of a neural network, based on the combination of information theory and braid theory. The network studied in the article implemented with the intersections between the braid strands, as well as simplified networks (a network with strands without intersections and a simple convolutional deep neural network), are used to solve various problems of multiclass image classification that allow us to analyze the comparative effectiveness of the proposed architecture. The simulation results showed BraidNet's comparative advantage in learning speed and classification accuracy.

cs.NE↗

arXiv2021

Constrained plasticity reserve as a natural way to control frequency and weights in spiking neural networks

Biological neurons have adaptive nature and perform complex computations involving the filtering of redundant information. However, most common neural cell models, including biologically plausible, such as Hodgkin-Huxley or Izhikevich, do not possess predictive dynamics on a single-cell level. Moreover, the modern rules of synaptic plasticity or interconnections weights adaptation also do not provide grounding for the ability of neurons to adapt to the ever-changing input signal intensity. While natural neuron synaptic growth is precisely controlled and restricted by protein supply and recycling, weight correction rules such as widely used STDP are efficiently unlimited in change rate and scale. The present article introduces new mechanics of interconnection between neuron firing rate homeostasis and weight change through STDP growth bounded by abstract protein reserve, controlled by the intracellular optimization algorithm. We show how these cellular dynamics help neurons filter out the intense noise signals to help neurons keep a stable firing rate. We also examine that such filtering does not affect the ability of neurons to recognize the correlated inputs in unsupervised mode. Such an approach might be used in the machine learning domain to improve the robustness of AI systems.

q-bio.NC↗

arXiv2018

Hyperplane Neural Codes and the Polar Complex

Hyperplane codes are a class of convex codes that arise as the output of a one layer feed-forward neural network. Here we establish several natural properties of stable hyperplane codes in terms of the {\it polar complex} of the code, a simplicial complex associated to any combinatorial code. We prove that the polar complex of a stable hyperplane code is shellable and show that most currently known properties of the hyperplane codes follow from the shellability of the appropriate polar complex.

q-bio.NC↗