On Computational Power and the Order-Chaos Phase Transition in Reservoir
B. Schrauwen, L. Buesing, and R. Legenstein
Randomly connected recurrent neural circuits have proven to be very powerful
models for online computations when a trained memoryless readout function is
appended. Such Reservoir Computing (RC) systems are commonly used in
two flavors: with analog or binary (spiking) neurons in the recurrent
circuits. Previous work showed a fundamental difference between these two
incarnations of the RC idea. The performance of a RC system build from binary
neurons seems to depend strongly on the network connectivity structure. In
networks of analog neurons such dependency has not been observed. In this
article we investigate this apparent dichotomy in terms of the in-degree of
the circuit nodes. Our analyses based amongst others on the Lyapunov exponent
reveal that the phase transition between ordered and chaotic network behavior
of binary circuits qualitatively differs from the one in analog circuits.
This explains the observed decreased computational performance of binary
circuits of high node in-degree. Furthermore, a novel mean-field predictor
for computational performance is introduced and shown to accurately predict
the numerically obtained results.
Reference: B. Schrauwen, L. Buesing, and R. Legenstein.
On computational power and the order-chaos phase transition in reservoir
In Proc. of NIPS 2008, Advances in Neural Information Processing
Systems, volume 21, pages 1425-1432. MIT Press, 2009.