Real-Time Computation at the Edge of Chaos in Recurrent Neural Networks

N. Bertschinger and T. Natschlaeger

Abstract:

Depending on the connectivity recurrent networks of simple computational units can show very different types of dynamics ranging from totally ordered to chaotic. We analyze how the type of dynamics (ordered or chaotic) exhibited by randomly connected networks of threshold gates driven by a time varying input signal depends on the parameters describing the distribution of the connectivity matrix. In particular we calculate the critical boundary in parameter space where the transition from ordered to chaotic dynamics takes places. Employing a recently developed framework for analyzing real-time computations we show that only near the critical boundary such networks can perform complex computations on time series. Hence, this result strongly supports conjectures that dynamical systems which are capable of doing complex computational tasks should operate near the edge of chaos, i.e. the transition from ordered to chaotic dynamics.

Reference: N. Bertschinger and T. Natschlaeger. Real-time computation at the edge of chaos in recurrent neural networks. Neural Computation, 16(7):1413-1436, 2004.