Forward and Backward Input Variable Selection for Polynomial Echo State Networks

Publication date: Available online 17 February 2020Source: NeurocomputingAuthor(s): Cuili Yang, Xinxin Zhu, Junfei Qiao, Kaizhe NieAbstractAs extension of traditional echo state networks (ESNs), the polynomial echo state networks (PESNs) have been proposed in our previous work (C.L. Yang, J.F. Qiao, H.G. Han, L. Wang, Design of polynomial echo state networks for time series prediction, Neurocomputing 290 (2018) 148-160.) by employing the polynomial function of complete input variable as output weight matrix. In practice, the generalization performance and computational burden of PESNs are perturbed by redundant or irrelevant inputs. To construct output weights with a suitable subset of input variables, the forward selection based PESN (FS-PESN) and backward selection based PESN (BS-PESN) are proposed. Firstly, the forward selection method is used in FS-PESN to choose the input variable which incurs the maximum reduction on objective function, and the backward selection shame is introduced in BS-PESN to remove the input variable which leads to the smallest increment on objective function. Then, the iterative updating strategies are designed to avoid repetitive computations in FS-PESN and BS-PESN. Specially, an accelerating scheme is introduced into BS-PESN to simplify training process. Finally, numerical simulations are carried out to illustrate effectiveness of the proposed techniques in terms of generalization ability and testing time.

Source: Neurocomputing - February 17, 2020 Category: Neuroscience Source Type: research