Search arXivSearch

arXiv subjects

Larry Shepp

Publications and source records attributed to Larry Shepp.

11 recordsLinked to original sources

Yule's "Nonsense Correlation" Solved!

In this paper, we resolve a longstanding open statistical problem. The problem is to mathematically confirm Yule's 1926 empirical finding of "nonsense correlation" (\cite{Yule}). We do so by analytically determining the second moment of the empirical correlation coefficient \beqn \theta := \frac{\int_0^1W_1(t)W_2(t) dt - \int_0^1W_1(t) dt \int_0^1 W_2(t) dt}{\sqrt{\int_0^1 W^2_1(t) dt - \parens{\int_0^1W_1(t) dt}^2} \sqrt{\int_0^1 W^2_2(t) dt - \parens{\int_0^1W_2(t) dt}^2}}, \eeqn of two {\em independent} Wiener processes, $W_1,W_2$. Using tools from Fred- holm integral equation theory, we successfully calculate the second moment of $\theta$ to obtain a value for the standard deviation of $\theta$ of nearly .5. The "nonsense" correlation, which we call "volatile" correlation, is volatile in the sense that its distribution is heavily dispersed and is frequently large in absolute value. It is induced because each Wiener process is "self-correlated" in time. This is because a Wiener process is an integral of pure noise and thus its values at different time points are correlated. In addition to providing an explicit formula for the second moment of $\theta$, we offer implicit formulas for higher moments of $\theta$.

math.ST

Revisiting a Theorem of L.A. Shepp on Optimal Stopping

Using a bondholder who seeks to determine when to sell his bond as our motivating example, we revisit one of Larry Shepp's classical theorems on optimal stopping. We offer a novel proof of Theorem 1 from from \cite{Shepp}. Our approach is that of guessing the optimal control function and proving its optimality with martingales. Without martingale theory one could hardly prove our guess to be correct.

q-fin.MF

On Optimal Retirement (How to Retire Early)

We pose an optimal control problem arising in a perhaps new model for retirement investing. Given a control function $f$ and our current net worth as $X(t)$ for any $t$, we invest an amount $f(X(t))$ in the market. We need a fortune of $M$ "superdollars" to retire and want to retire as early as possible. We model our change in net worth over each infinitesimal time interval by the Ito process $dX(t)= (1+f(X(t))dt+ f(X(t))dW(t)$. We show how to choose the optimal $f=f_0$ and show that the choice of $f_0$ is optimal among all nonanticipative investment strategies, not just among Markovian ones.

q-fin.ST

On occupation times of the first and third quadrants for planar Brownian motion

An open problem of interest, first infused into the applied probability community in the work of Bingham and Doney in 1988, (see \cite{Bingham}) is stated as follows: find the distribution of the quadrant occupation time of planar Brownian motion. In this short communication, we study an alternate formulation of this longstanding open problem: let $X(t), Y(t), t \geq 0$ be standard Brownian motions starting at $x,y$ respectively. Find the distribution of the total time $T=Leb\{t \in [0,1]: X(t) \times Y(t) >0\}$, when $x=y=0$, i.e., the occupation time of the union of the first and third quadrants. If two adjacent quadrants are used, the problem becomes much easier and the distribution of $T$ follows the arcsine law.

math.PR

Fiscal stimulus as an optimal control problem

During the Great Recession, Democrats in the United States argued that government spending could be utilized to "grease the wheels" of the economy in order to create wealth and to increase employment; Republicans, on the other hand, contended that government spending is wasteful and discouraged investment, thereby increasing unemployment. Today, in 2020, we find ourselves in the midst of another crisis where government spending and fiscal stimulus is again being considered as a solution. In the present paper, we address this question by formulating an optimal control problem generalizing the model of Radner & Shepp (1996). The model allows for the company to borrow continuously from the government. We prove that there exists an optimal strategy; rigorous verification proofs for its optimality are provided. We proceed to prove that government loans increase the expected net value of a company. We also examine the consequences of different profit-taking behaviors among firms who receive fiscal stimulus.

econ.GN

The General Stationary Gaussian Markov Process

We find the class, ${\cal{C}}_k, k \ge 0$, of all zero mean stationary Gaussian processes, $Y(t), ~t \in \reals$ with $k$ derivatives, for which \begin{equation} Z(t) \equiv (Y^{(0)}(t), Y^{(1)}(t), \ldots, Y^{(k)}(t) ), ~ t \ge 0 \end{equation} \noindent is a $(k+1)$-vector Markov process. (here, $Y^{(0)}(t) = Y(t)$).

math.PR

On the Time for Brownian Motion to Visit Every Point on a Circle

Consider a Wiener process $W$ on a circle of circumference $L$. We prove the rather surprising result that the Laplace transform of the distribution of the first time, $\theta_L$, when the Wiener process has visited every point of the circle can be solved in closed form using a continuous recurrence approach.

math.PR

Pick Up Sticks

We apply a classic result in Theoretical Computer Science to solve a sticky problem, and give a neat and slick quick proof of half of it

math.CO

Distinct counting with a self-learning bitmap

Counting the number of distinct elements (cardinality) in a dataset is a fundamental problem in database management. In recent years, due to many of its modern applications, there has been significant interest to address the distinct counting problem in a data stream setting, where each incoming data can be seen only once and cannot be stored for long periods of time. Many probabilistic approaches based on either sampling or sketching have been proposed in the computer science literature, that only require limited computing and memory resources. However, the performances of these methods are not scale-invariant, in the sense that their relative root mean square estimation errors (RRMSE) depend on the unknown cardinalities. This is not desirable in many applications where cardinalities can be very dynamic or inhomogeneous and many cardinalities need to be estimated. In this paper, we develop a novel approach, called self-learning bitmap (S-bitmap) that is scale-invariant for cardinalities in a specified range. S-bitmap uses a binary vector whose entries are updated from 0 to 1 by an adaptive sampling process for inferring the unknown cardinality, where the sampling rates are reduced sequentially as more and more entries change from 0 to 1. We prove rigorously that the S-bitmap estimate is not only unbiased but scale-invariant. We demonstrate that to achieve a small RRMSE value of $\epsilon$ or less, our approach requires significantly less memory and consumes similar or less operations than state-of-the-art methods for many common practice cardinality scales. Both simulation and experimental studies are reported.

stat.CO

ROC and the bounds on tail probabilities via theorems of Dubins and F. Riesz

For independent $X$ and $Y$ in the inequality $P(X\leq Y+μ)$, we give sharp lower bounds for unimodal distributions having finite variance, and sharp upper bounds assuming symmetric densities bounded by a finite constant. The lower bounds depend on a result of Dubins about extreme points and the upper bounds depend on a symmetric rearrangement theorem of F. Riesz. The inequality was motivated by medical imaging: find bounds on the area under the Receiver Operating Characteristic curve (ROC).

math.PR

Statistical thinking: From Tukey to Vardi and beyond

Data miners (minors?) and neural networkers tend to eschew modelling, misled perhaps by misinterpretation of strongly expressed views of John Tukey. I discuss Vardi's views of these issues as well as other aspects of Vardi's work in emision tomography and in sampling bias.

math.ST