Recurrent Neural Networks for Forecasting Autoregressive Noncausal Processes

Abstract

This paper examines the use of recurrent neural networks and their extensions for forecasting autoregressive causal-noncausal processes, characterized by local explosive patterns such as bubbles and spikes. We study the recurrent neural networks (RNN), long short-term memory networks (LSTM), gated recurrent unit (GRU), bidirectional LSTM, and also a simple feed-forward network (FNN) to reveal the difficulty of forecasting extreme values depending on the type of network and the length of the training set. We show that FNN outperforms other networks in predicting extreme values of a univariate noncausal time series during a bubble. We also examine the improvements of point predictions of extreme values resulting from pooling and a modified loss function.