Gal Chechik — Page 8 | Search arXiv

arXiv2012

Adaptive Regularization for Weight Matrices

Algorithms for learning distributions over weight-vectors, such as AROW were recently shown empirically to achieve state-of-the-art performance at various problems, with strong theoretical guaranties. Extending these algorithms to matrix models pose challenges since the number of free parameters in the covariance of the distribution scales as $n^4$ with the dimension $n$ of the matrix, and $n$ tends to be large in real applications. We describe, analyze and experiment with two new algorithms for learning distribution of matrix models. Our first algorithm maintains a diagonal covariance over the parameters and can handle large covariance matrices. The second algorithm factors the covariance to capture inter-features correlation while keeping the number of parameters linear in the size of the original matrix. We analyze both algorithms in the mistake bound model and show a superior precision performance of our approach over other algorithms in two tasks: retrieving similar images, and ranking similar documents. The factored algorithm is shown to attain faster convergence rate.

cs.LG↗

arXiv2011

COLoR - Coordinated On-Line Rankers for Network Reconstruction

Predicting protein interactions is one of the more interesting challenges of the post-genomic era. Many algorithms address this problem as a binary classification problem: given two proteins represented as two vectors of features, predict if they interact or not. Importantly however, computational predictions are only one component of a larger framework for identifying PPI. The most promising candidate pairs can be validated experimentally by testing if they physical bind to each other. Since these experiments are more costly and error prone, the computational predictions serve as a filter, aimed to produce a small number of highly promising candidates. Here we propose to address this problem as a ranking problem: given a network with known interactions, rank all unknown pairs based on the likelihood of their interactions. In this paper we propose a ranking algorithm that trains multiple inter-connected models using a passive aggressive on-line approach. We show good results predicting protein-protein interactions for post synaptic density PPI network. We compare the precision of the ranking algorithm with local classifiers and classic support vector machine. Though the ranking algorithm outperforms the classic SVM classification, its performance is inferior compared to the local supervised method.

q-bio.MN↗