Neural Networks B, SS06 1

UA Dr. Robert Legenstein, WiMAus Prashant Joshi, M.S.

Institute for Theoretical Computer Science
Technische Universität Graz
A-8010 Graz, Austria
{legi, joshi}

NACHNAME Vorname Matrikelnmr Team

Exercise 1: Simulating a sigmoidal neuron with integrate-and-fire neurons

NOTE: You can download this exercise in pdf or postscript(ps) format here.

In this exercise you are going to simulate the output of a sigmoidal neuron using a feedforward circuit of intergrate-and-fire neurons. You are going to use the CSIM simulator for the purpose of this simulation.


Figure 1 shows a neuron that receives R inputs. The individual inputs $ p_1,p_2, \ldots,p_R$ are each weighted by corresponding elements $ w_{1,1},w_{1,2},\ldots,w_{1,R}$ of the weight matrix W.

Figure: Multiple input neuron

The neuron has a bias $ b$, which is summed with the weighted inputs to form the net input $ n$:

$\displaystyle n = w_{1,1}p_1 + w_{1,2}p_2+\ldots+w_{1,R}p_R +b$ (1)

This expression can be written in matrix form:

$\displaystyle n = \mathbf{Wp} +b$ (2)

where the matrix W for the single neuron case has only one row.

Now the output of this neuron can be written as

$\displaystyle a = f(\mathbf{Wp} +b)$ (3)

For this exercise, let: $ R = 10$, $ b = 0$, and the transfer function $ f$ of the neurons be the hyperbolic tangent sigmoid function that is defined as 2:

$\displaystyle f = \frac{e^n-e^{-n}}{e^n+e^{-n}}$ (4)

which is called in MATLAB as the $ logsig$ function.

In this problem, you have to simulate the output of the sigmoidal neuron with above mentioned properties using a feed-forward citcuit of integrate-and-fire neurons. An integrate-and-fire neuron is one of the simplest mathematical model of a spiking neuron whose membrane potential $ V_m$ is given by:

$\displaystyle \tau_m\frac{dV_m}{dt} = -(V_m - V_{resting}) + R_m\cdot(I_{syn}(t)
 + I_{inject} + I_{noise})$ (5)

where $ \tau_m = C_m \cdot R_m$ is the membrane time constant, $ R_m$ is the membrane resistance, $ I_{syn}(t)$ is the current supplied by the synapses, $ I_{inject}$ is a non-specific background current and $ I_{noise}$ is a Gaussian random variable with a zero mean and a given variance noise.

At time $ t = 0$ $ V_m$ is set to $ V_{init}$. If $ V_m$ exceeds the threshold voltage $ V_{thresh}$ it is reset to $ V_{reset}$ and held there for the length $ T_{refract}$, which is the absolute refractory period of this neuron.

Figure: Setup to simulate a sigmoidal neuron using a pool of integrate-and-fire neurons

In principle, you can simulate the output of a sigmoidal neuron using a feedforward circuit of integrate-and-fire neurons. In this exercise we are going to test this. The setup that acheives this is shown in figure 2. For this, each of the $ N$ integrate-and-fire neurons of the feedforward circuit receive the input $ \mathbf {p}$, and the output of this circuit at any time point is defined as:

$\displaystyle \xi(t) = \frac{n(t)}{N}$ (6)

where $ n(t)$ is the number of active neuron at time $ t$, and $ N$ is the total number of neurons in the feedforward circuit. This is an example of population coding, where the value of a variable at any point of time is encoded by the ratio of number of active neurons in the population to the total size of population.


Let the inputs to the sigmoidal neuron be a 10 dimensional vector, such that $ p_1,p_2,\ldots,p_{10} = \{0,1\}^{10}$. Let the feedforward circuit of integrate-and-fire neurons have $ N = 200$ neurons. Simulate the functioning of this neuron using the setup shown in figure 2 for the case when the weight vector from the inputs to the sigmoidal neuron has the values:
  1. ($ 10$ points)$ \mathbf W$ = ones(1,ipDim)/ipDim; % (ipDim = 10)
    (ipDim is the size of a single input pattern).
  2. ($ 5$ bonus points) $ \mathbf W$ = randperm(ipDim); %(ipDim = 10)
    $ \mathbf W$ = w/sum(w);

For this you will have to simulate the circuit once per input pattern followed by a reset.

NOTE: You are also required to submit the source code that you used to do this exercise.

About this document ...

Neural Networks B, SS06 1

This document was generated using the LaTeX2HTML translator Version 2002-2-1 (1.70)

Copyright © 1993, 1994, 1995, 1996, Nikos Drakos, Computer Based Learning Unit, University of Leeds.
Copyright © 1997, 1998, 1999, Ross Moore, Mathematics Department, Macquarie University, Sydney.

The command line arguments were:
latex2html -split=0 -no_navigation ex2.tex

The translation was initiated by Joshi Prashant on 2006-03-28


... SS061
Class Website:
... as2
We only consider positive activation of this function as all elements of $ \mathbf{p} \geq 0$.

Joshi Prashant 2006-03-28