Efficient Temporal Processing with Biologically Realistic Dynamic
T. Natschlaeger, W. Maass, and A. Zador
Experimental data show that biological synapses behave quite differently from
the symbolic synapses in common artificial neural network models. Biological
synapses are dynamic, i.e., their ``weight'' changes on a short time scale by
several hundred percent in dependence of the past input to the synapse. Here
we describe a general model of computation that exploits dynamic synapses,
and use a backpropagation-like algorithm to adjust the synaptic parameters.
We show that such gradient descent suffices to approximate a given quadratic
filter by a rather small neural system with dynamic synapses. We demonstrate
that with this approach the nonlinear filter considered in (Back and Tsoi,
1993) can be approximated slightly better than by their model. Our numerical
results are complemented by theoretical analysis which show that even with
just a single hidden layer such networks can approximate a surprisingly large
class of nonlinear filters: all filters that can be characterized by Volterra
series. This result is robust with regard to various changes in the model for
Reference: T. Natschlaeger, W. Maass, and A. Zador.
Efficient temporal processing with biologically realistic dynamic synapses.
Network: Computation in Neural Systems, 12:75-87, 2001.