State-Space Neural Network with Ordered Variance for Model Order Determination

Abstract

This paper addresses the problem of identifying a nonlinear state-space model, along with an adequate model order, from a given input-output training dataset. To this end, a novel framework, termed state-space neural network with ordered variance (SSNNO), is proposed. In SSNNO, the state variables are ordered according to their variances computed using the training data. This ordering is achieved by introducing a variance-regularization term into the loss function used for SSNNO training and it facilitates a distinction between significant states, which exhibit high variance from the other residual states with near-zero variance. The number of significant states is indicative of a suitable model order. The variance-regularization mechanism is designed to minimize the number of significant state variables, thereby promoting a minimal order of the identified state-space model without significantly compromising its prediction accuracy. A systematic procedure is then introduced to obtain a reduced-order state-space model from the trained SSNNO, yielding a reduced-order SSNNO (R-SSNNO). The existence of an SSNNO with variance-ordered states, based solely on input-output data, as well as an upper bound on its output prediction error, are formally established. A practical and robust method is proposed for ensuring variance-ordered states in an SSNNO, even when the network is trained using local optimization algorithms. The effectiveness of the proposed method for identification of nonlinear state space models is demonstrated through simulation studies on a nonlinear continuous stirred-tank reactor process. The identified model is further used for state estimation and prediction in a model predictive control implementation.