arXiv subjects

Barnabas Poczos

Publications and source records attributed to Barnabas Poczos.

97 records · Page 6Linked to original sources

arXiv2008

D-optimal Bayesian Interrogation for Parameter and Noise Identification of Recurrent Neural Networks

We introduce a novel online Bayesian method for the identification of a family of noisy recurrent neural networks (RNNs). We develop Bayesian active learning technique in order to optimize the interrogating stimuli given past experiences. In particular, we consider the unknown parameters as stochastic variables and use the D-optimality principle, also known as `\emph{infomax method}', to choose optimal stimuli. We apply a greedy technique to maximize the information gain concerning network parameters at each time step. We also derive the D-optimal estimation of the additive noise that perturbs the dynamical system of the RNN. Our analytical results are approximation-free. The analytic derivation gives rise to attractive quadratic update rules.

cs.NE↗

arXiv2007

Undercomplete Blind Subspace Deconvolution via Linear Prediction

We present a novel solution technique for the blind subspace deconvolution (BSSD) problem, where temporal convolution of multidimensional hidden independent components is observed and the task is to uncover the hidden components using the observation only. We carry out this task for the undercomplete case (uBSSD): we reduce the original uBSSD task via linear prediction to independent subspace analysis (ISA), which we can solve. As it has been shown recently, applying temporal concatenation can also reduce uBSSD to ISA, but the associated ISA problem can easily become `high dimensional' [1]. The new reduction method circumvents this dimensionality problem. We perform detailed studies on the efficiency of the proposed technique by means of numerical simulations. We have found several advantages: our method can achieve high quality estimations for smaller number of samples and it can cope with deeper temporal convolutions.

stat.ME↗

arXiv2007

Undercomplete Blind Subspace Deconvolution

We introduce the blind subspace deconvolution (BSSD) problem, which is the extension of both the blind source deconvolution (BSD) and the independent subspace analysis (ISA) tasks. We examine the case of the undercomplete BSSD (uBSSD). Applying temporal concatenation we reduce this problem to ISA. The associated `high dimensional' ISA problem can be handled by a recent technique called joint f-decorrelation (JFD). Similar decorrelation methods have been used previously for kernel independent component analysis (kernel-ICA). More precisely, the kernel canonical correlation (KCCA) technique is a member of this family, and, as is shown in this paper, the kernel generalized variance (KGV) method can also be seen as a decorrelation method in the feature space. These kernel based algorithms will be adapted to the ISA task. In the numerical examples, we (i) examine how efficiently the emerging higher dimensional ISA tasks can be tackled, and (ii) explore the working and advantages of the derived kernel-ISA methods.

math.ST↗

arXiv2007

Independent Process Analysis without A Priori Dimensional Information

Recently, several algorithms have been proposed for independent subspace analysis where hidden variables are i.i.d. processes. We show that these methods can be extended to certain AR, MA, ARMA and ARIMA tasks. Central to our paper is that we introduce a cascade of algorithms, which aims to solve these tasks without previous knowledge about the number and the dimensions of the hidden processes. Our claim is supported by numerical simulations. As a particular application, we search for subspaces of facial components.

math.ST↗

arXiv2006

Separation Theorem for K-Independent Subspace Analysis with Sufficient Conditions

Here, a Separation Theorem about K-Independent Subspace Analysis (K real or complex), a generalization of K-Independent Component Analysis (KICA) is proven. According to the theorem, KISA estimation can be executed in two steps under certain conditions. In the first step, 1-dimensional KICA estimation is executed. In the second step, optimal permutation of the KICA elements is searched for. We present sufficient conditions for the KISA Separation Theorem. Namely, we shall show that (i) spherically symmetric sources (both for real and complex cases), as well as (ii) real 2-dimensional sources invariant to 90 degree rotation, among others, satisfy the conditions of the theorem.

math.ST↗

arXiv2006

Separation Theorem for Independent Subspace Analysis with Sufficient Conditions

Here, a separation theorem about Independent Subspace Analysis (ISA), a generalization of Independent Component Analysis (ICA) is proven. According to the theorem, ISA estimation can be executed in two steps under certain conditions. In the first step, 1-dimensional ICA estimation is executed. In the second step, optimal permutation of the ICA elements is searched for. We present sufficient conditions for the ISA Separation Theorem. Namely, we shall show that (i) elliptically symmetric sources, (ii) 2-dimensional sources invariant to 90 degree rotation, among others, satisfy the conditions of the theorem.

math.ST↗

arXiv2003

Kalman-filtering using local interactions

There is a growing interest in using Kalman-filter models for brain modelling. In turn, it is of considerable importance to represent Kalman-filter in connectionist forms with local Hebbian learning rules. To our best knowledge, Kalman-filter has not been given such local representation. It seems that the main obstacle is the dynamic adaptation of the Kalman-gain. Here, a connectionist representation is presented, which is derived by means of the recursive prediction error method. We show that this method gives rise to attractive local learning rules and can adapt the Kalman-gain.

cs.AI↗