Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 1512.06238

The Limitations of Optimization from Samples

Eric Balkanski·Aviad Rubinstein·Yaron Singer
View original source ↗Read source document ↗Export metadata ↓

Abstract

In this paper we consider the following question: can we optimize objective functions from the training data we use to learn them? We formalize this question through a novel framework we call optimization from samples (OPS). In OPS, we are given sampled values of a function drawn from some distribution and the objective is to optimize the function under some constraint. While there are interesting classes of functions that can be optimized from samples, our main result is an impossibility. We show that there are classes of functions which are statistically learnable and optimizable, but for which no reasonable approximation for optimization from samples is achievable. In particular, our main result shows that there is no constant factor approximation for maximizing coverage functions under a cardinality constraint using polynomially-many samples drawn from any distribution. We also show tight approximation guarantees for maximization under a cardinality constraint of several interesting classes of functions including unit-demand, additive, and general monotone submodular functions, as well as a constant factor approximation for monotone submodular functions with bounded curvature.

Explore related subjects

cs.DScs.DMcs.LG

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Eric Balkanski, Aviad Rubinstein, Yaron Singer. 2016-11-15. The Limitations of Optimization from Samples. https://arxiv.org/abs/1512.06238

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Online Flexible Busy Time Scheduling on Heterogeneous Machines

We study the online busy time scheduling model on heterogeneous machines. In our setting, jobs with uniform processing time arrive online with a deadline that becomes known to the algorithm at the job's arrival time. An algorithm has access to machines, each with different associated capacities and costs. The goal is to schedule jobs on machines by their deadline, so that the total cost incurred by the scheduling algorithm is minimized. While busy time scheduling has been well-studied, relatively little is known when machines are heterogeneous (i.e., have different costs and capacities), despite this natural theoretical generalization being the most practical model for clients using cloud computing services. We make significant progress in understanding this model by designing a deterministic online algorithm with competitive ratio 8(2p-1)/p < 16 when all jobs have uniform processing time p. A randomized version of this algorithm is 4(2p-1)/(p \ln 2)-competitive against an oblivious adversary. For unit-processing-time jobs, we give lower bounds of 4 and e (where e is Euler's number) on the competitive ratio of deterministic and randomized online algorithms, respectively. For unit-processing-time jobs with agreeable deadlines, we provide a deterministic 2-competitive online algorithm and a matching lower bound.

cs.DS↗
arXiv2026

The Semi-Oblivious Cup Game: an Imperfect Information Setting

In the cup game, an adversary distributes $1$ unit of water among $n$ initially empty cups during each time step. The player then selects a single cup from which to remove up to $1$ unit of water, with the goal of minimizing the backlog, i.e., the supremum of the height of the fullest cup over all time steps. In the cup flushing game, the player is additionally allowed to empty the chosen cup entirely. Past work has shown that the optimal backlog in both of these settings is $Θ(\log n)$. Furthermore, the \textbf{greedy} algorithm, which always removes water from the fullest cup, has been shown in previous work to be exactly optimal in both the cup game and the cup flushing game. We introduce a new model, the semi-oblivious cup game, in which the player is uncertain of the exact height of each cup. We analyze the performance of the \textbf{greedy} algorithm in this setting, which can be viewed as selecting an arbitrary cup within a constant multiplicative factor of the fullest cup. We prove matching upper and lower bounds showing that the \textbf{greedy} algorithm achieves a backlog of $Θ(n^{\frac{c-1}{c}})$ in the semi-oblivious cup game. We also establish matching upper and lower bounds of $2^{Θ(\sqrt{\log n})}$ in the semi-oblivious cup flushing game. Finally, we show that in an additive error setting, greedy is actually able to achieve backlog $Θ(\log n)$, via matching upper and lower bounds. All of our lower bounds apply for adaptive adversaries against any (even randomized) algorithm, proving that greedy is asymptotically optimal in the semi-oblivious model.

cs.DS↗
arXiv2026

The Binary Tree Mechanism is Optimal for Differentially Private Continual Counting

Private continual counting is a fundamental problem in differential privacy: given a binary stream of length $n$, where each $1$ corresponds to the contribution of one individual, the goal is to release all running counts while protecting the privacy of each individual. For fixed privacy parameters, the standard binary tree mechanism achieves expected $\ell_\infty$ error $O(\log^{3/2} n)$ under approximate differential privacy and $O(\log^2 n)$ under pure differential privacy. Whether these dependences on the stream length are necessary has remained a central open problem. For fixed $\varepsilon\in(0,1)$, we prove a lower bound of $Ω(\log^{3/2} n)$ under approximate DP with sufficiently small fixed $δ>0$, and a lower bound of $Ω(\log^2 n)$ under pure DP. These bounds establish the optimality of the binary tree mechanism in both settings. The bounds hold for arbitrary mechanisms, even when the entire stream is available in advance. Both proofs use the same decomposition and accumulation of residual noise along a tree. As a consequence of the approximate-DP bound, we also obtain a largest-possible separation between hereditary discrepancy and private $\ell_\infty$ error for linear queries, showing that the known general upper bound in terms of hereditary discrepancy has the optimal dependence on the number of queries.

cs.DS↗