next up previous
Next: RL theory I [3 Up: MLB_Exercises_2008 Previous: Small-world networks [3 P]

Robustness [3 P]

Compare the memory capacity and the robustness properties of five networks consisting of threshold gates with different connectivity graphs for a binary classification task.

a)
Networks:

The five types of networks are: regular 2-dimensional lattice, random networks, Barabasi-Albert scale-free networks, Watts-Strogatz small-world networks, and Kaiser-Hilgetag small-world networks. Each network consists of 100 threshold gates and 1000 weights, where each weight is drawn randomly from a Gaussian distribution with mean zero and standard deviation of one. The threshold of each gate is set to zero.

  1. Download the MATLAB code 5 and modify the MATLAB file robustness.m to generate the missing networks. Implement the small-world and scale-free networks as outlined in the exercises.
  2. For each network a function should be implemented that takes the input parameters as stated in the corresponding function header (nNodes,nLinks,$ \alpha$ ,...) and outputs a connectivity matrix $ W_{ij}$ of size 100$ \times$ 100 with elements 0 and 1, where 1 indicates a weight from gate $ j$ to gate $ i$ . All networks represent directed graphs and $ W_{ij}$ is therefore in general not symmetric.

b)
Memory capacity:

To estimate the memory capacity simulate the networks for 1000 time steps. During each time step binary input (1-dimensional with randomly chosen bit values) is injected into the first 20 threshold gates of the network. The input weights are drawn randomly from a Gaussian distribution with mean zero and a standard deviation of eight. A linear classifier should be trained with linear regression on the states of the last 20 gates of the network to output the input bit that was injected $ \Delta t \in \{1,2,3,...,50\}$ ms ago. For each $ \Delta t$ a different classifier should be trained, thereby estimating the memory capacity of the linear classifier for each network as a function of $ \Delta t$ .

  1. Perform 100 network simulations for each network type and generate for each simulation a novel input and a novel network.
  2. Train for each simulation a linear classifier for each $ \Delta t$ and determine the MAE. Calculate for all 100 simulations the average error and the standard error of the mean SEM to verify the significance of the results.
  3. Generate a plot that shows the average MAE and the SEM as a function of $ \Delta t$ for all network types. Note that the output of the linear classifier is 0 or 1. It is obtained by thresholding the output of the linear regression model at a value of 0.5. Compare and discuss the results for all networks.

c)
Robustness:

To estimate the robustnes properties train the networks for $ \Delta t = 2$ ms for which the performance of the linear classifier is highest. After training prune all weights of $ P=0,2,4,6,...,20, 30$ and 50 % of the network gates. (but not the input weights). Simulate the network again and test the performance of the linear classifier on the pruned network. Calculate the performance values thereby estimating the robustness of the performance of the linear classifier as a function of the number of gates with pruned weights.

  1. Perform 100 simulations for each network type and generate for each simulation a novel input and a novel network.
  2. Train for each simulation a linear classifier for each $ P$ and determine the MAE. Calculate for all 100 simulations the average error and the standard error of the mean SEM to verify the significance of the results.
  3. Generate a plot that shows the average MAE and the standard error of the mean SEM as a function of $ P$ (the % of pruned gates) for all network types. Compare the difference in performance for $ P=0$ , $ P=0.1$ and $ P=0.5$ for each network type. Which network has the smallest memory loss?

Present your results clearly, structured and legible. Document them in such a way that anybody can reproduce them effortless.


next up previous
Next: RL theory I [3 Up: MLB_Exercises_2008 Previous: Small-world networks [3 P]
Haeusler Stefan 2009-01-19