Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · math/0110103

The Generalized Spike Process, Sparsity, and Statistical Independence

Naoki Saito
View original source ↗Read source document ↗Export metadata ↓

Abstract

A basis under which a given set of realizations of a stochastic process can be represented most sparsely (the so-called best sparsifying basis (BSB)) and the one under which such a set becomes as less statistically dependent as possible (the so-called least statistically-dependent basis (LSDB)) are important for data compression and have generated interests among computational neuroscientists as well as applied mathematicians. Here we consider these bases for a particularly simple stochastic process called ``generalized spike process'', which puts a single spike--whose amplitude is sampled from the standard normal distribution--at a random location in the zero vector of length $\ndim$ for each realization. Unlike the ``simple spike process'' which we dealt with in our previous paper and whose amplitude is constant, we need to consider the kurtosis-maximizing basis (KMB) instead of the LSDB due to the difficulty of evaluating differential entropy and mutual information of the generalized spike process. By computing the marginal densities and moments, we prove that: 1) the BSB and the KMB selects the standard basis if we restrict our basis search within all possible orthonormal bases in ${\mathbb R}^n$; 2) if we extend our basis search to all possible volume-preserving invertible linear transformations, then the BSB exists and is again the standard basis whereas the KMB does not exist. Thus, the KMB is rather sensitive to the orthonormality of the transformations under consideration whereas the BSB is insensitive to that. Our results once again support the preference of the BSB over the LSDB/KMB for data compression applications as our previous work did.

Explore related subjects

math.PRcs.NAmath.NAmath.OCmath.RAmath.STq-biostat.TH

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Naoki Saito. 2001-10-09. The Generalized Spike Process, Sparsity, and Statistical Independence. https://arxiv.org/abs/math/0110103

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Selection in the Zero-Noise Limit for High-Dimensional Diffusions with Measurable Drift

This paper investigates the zero-noise limit of high-dimensional small-noise diffusion processes governed by the stochastic differential equation (SDE) \begin{equation*} \mathrm{d}X_{t}^{\varepsilon }=b(X_{t}^{\varepsilon })\,\mathrm{d}t+\varepsilon \,\mathrm{d}W_{t},\quad X_{0}^{\varepsilon }=0,\quad \varepsilon >0, \end{equation*} where the drift coefficient $b$ is assumed to be measurable and bounded. Under this condition, the associated deterministic ordinary differential equation (ODE) $\dot{x}_{t}=b(x_{t})$ may admit multiple Filippov solutions, i.e., solutions in the sense of differential inclusions, due to the lack of Lipschitz continuity or uniqueness criteria. However, the introduction of non-degenerate additive noise restores well-posedness: the perturbed system admits a unique strong (pathwise) solution for each $\varepsilon >0$. Our analysis is based on the radial-spherical decomposition. We prove that, under certain assumptions, the radius dominates a one-dimensional comparison process and escapes to infinity at a linear rate, and the extrinsic angular martingale converges. Under an explicit separable gradient structure, expressed in terms of a radial potential $V(r,ω)=\langle b(rω),ω\rangle $, the angle converges, for every fixed noise, almost surely to the set of global angular maximizers of $V$, and its distance from that set tends to zero in probability as $\varepsilon\rightarrow0$. The limiting measure is compact, carried by immediate-departure Filippov paths, and, under a Lipschitz radial-graph condition, purely singular with Hausdorff dimension at most $d-1$. A continuous-mapping criterion for exit-time convergence is also established.

math.PR↗
arXiv2026

Asymptotic optimality of dynamic first-fit packing on the half-axis

We revisit a classical problem in dynamic storage allocation. Items arrive in a linear storage medium, modeled as a half-axis, at a Poisson rate $r$ and depart after an independent exponentially distributed unit mean service time. The arriving item sizes (lengths) are assumed to be independent and identically distributed (i.i.d.) from a common distribution $H$. A widely employed algorithm for allocating the items is the "first-fit" discipline, namely, each arriving item is placed in the left-most vacant interval large enough to accommodate it. In a seminal 1985 paper, Coffman, Kadota, and Shepp ([6]) proved that in the special case of unit length items (i.e. degenerate $H$), as $r$ tends towards infinity, the first-fit algorithm is asymptotically optimal in the following sense: the steady-state ratio of expected "empty space" (gaps between items) to expected occupied space tends towards $0$. In a sequel to [6], Coffman, Kadota, and Shepp ([5]) conjectured that the first-fit discipline is also asymptotically optimal for non-degenerate $H$. In this paper we provide the first proof of first-fit asymptotic optimality for non-degenerate distributions $H$ of item sizes. Our main result is for the case when $H$ is concentrated on countably many positive real sizes forming an increasing sequence that is either finite or goes to infinity, with the average item size being finite. We prove that under the first-fit discipline, as $r$ tends towards infinity, the steady-state packing configuration (scaled down by $r$) converges in distribution to the limiting packing configuration with smaller items on the left, larger items on the right, and with no gaps between. In particular, this proves asymptotic optimality of first-fit in the following sense: if $P$ is the expected occupied space, then in steady-state the empty space (scaled down by $r$) in $[0,P]$ vanishes.

math.PR↗
arXiv2026

Mind the jumps: well-posedness of semi-martingale 2BSDEs

We develop a well-posedness theory for second-order backward stochastic differential equations (2BSDEs) with jumps. This work covers two complementary notions of solution: an extrinsic and an intrinsic one. We also discuss the obstruction to aggregating jump integrands. The chosen framework allows for controlled diffusions with jumps, pure-jump processes, and discrete-time processes in a unified setting.

math.PR↗