Learning-aided Stochastic Network Optimization with Imperfect State Prediction

Abstract

We investigate the problem of stochastic network optimization in the presence of imperfect state prediction and non-stationarity. Based on a novel distribution-accuracy curve prediction model, we develop the predictive learning-aided control (PLC) algorithm, which jointly utilizes historic and predicted network state information for decision making. PLC is an online algorithm that requires zero a-prior system statistical information, and consists of three key components, namely sequential distribution estimation and change detection, dual learning, and online queue-based control. Specifically, we show that PLC simultaneously achieves good long-term performance, short-term queue size reduction, accurate change detection, and fast algorithm convergence. In particular, for stationary networks, PLC achieves a near-optimal $[O(ε)$, $O(\log(1/ε)^2)]$ utility-delay tradeoff. For non-stationary networks, \plc{} obtains an $[O(ε), O(\log^2(1/ε)$ $+ \min(ε^{c/2-1}, e_w/ε))]$ utility-backlog tradeoff for distributions that last $Θ(\frac{\max(ε^{-c}, e_w^{-2})}{ε^{1+a}})$ time, where $e_w$ is the prediction accuracy and $a=Θ(1)>0$ is a constant (the Backpressue algorithm \cite{neelynowbook} requires an $O(ε^{-2})$ length for the same utility performance with a larger backlog). Moreover, PLC detects distribution change $O(w)$ slots faster with high probability ($w$ is the prediction size) and achieves an $O(\min(ε^{-1+c/2}, e_w/ε)+\log^2(1/ε))$ convergence time. Our results demonstrate that state prediction (even imperfect) can help (i) achieve faster detection and convergence, and (ii) obtain better utility-delay tradeoffs.