Contraction theory analysis of echo state networks

Abstract

Nowadays, the more and more intelligent and inter-disciplinary industrial tasks impose an increasingly strict requirement on the control system design, and thus, a more intensive research in the field of dynamic computation, control stability and robustness, as well as a deeper exploitation of implementing the ar- tificial intelligence methodology, for instance, recurrent neural networks (RNNs); such as precise control, motion planning and events detection for industry robots; stochastic events prediction in natural language processing. This report discusses the relationship between a nonlinear contraction control theory and echo state network (a specific type of neural network belonging to RNN), various proper- ties of echo state network (ESN), and applications of echo state network (ESN). Specifically, various sufficient conditions for a system to have echo state property (ESP) are investigated and compared, a sufficient condition for nonlinear con- traction theory was derived mathematically, the connections as well as nuances between these two properties are explored, and the short-term memory capacity of an echo state network is studied. It is discovered that with the contracting property, an echo state network is faster and easier to be trained to tackle com- plicated practical tasks, especially the nonlinear dynamical system.