Computational Intelligence, SS08
2 VO 442.070 + 1 RU 708.070 last updated:
Course Notes (Skriptum)
Online Tutorials
Introduction to Matlab
Neural Network Toolbox
OCR with ANNs
Adaptive Filters
VC dimension
Gaussian Statistics
PCA, ICA, Blind Source Separation
Hidden Markov Models
Mixtures of Gaussians
Automatic Speech Recognition
Practical Course Slides
Animated Algorithms
Interactive Tests
Key Definitions
Literature and Links


A Simple logical problem

The task is to create and train a neural network that solves the XOR problem. XOR is a function that returns 1 when the two inputs are not equal,

Construct a Feed-Forward Network

To solve this we will need a feedforward neural network with two input neurons, and one output neuron. Because that the problem is not linearly separable it will also need a hidden layer with two neurons.To create a new feed forward neural network use the command newff. You have to enter the max and min of the input values, the number of neurons in each layer and optionally the activation functions.

>> net = newff([0 1; 0 1],[2 1],{'logsig','logsig'});
The variable net will now contain an untrained feedforward neural network with two neurons in the input layer, two neurons in the hidden layer and one output neuron, exactly as we want it. The [0 1; 0 1] tells matlab that the input values ranges between 0 and 1. The 'logsig','logsig' tells matlab that we want to use the logsig function as activation function in all layers. The first parameter tells the network how many nodes there should be in the input layer, hence you do not have to specify this in the second parameter. You have to specify at least as many transfer functions as there are layers, not counting the input layer. If you do not specify any transfer function Matlab will use the default settings.

First we construct a matrix of the inputs. The input to the network is always in the columns of the matrix. To create a matrix with the inputs "1 1", "1 0", "0 1" and "0 0" we enter:
>> input = [1 1 0 0; 1 0 1 0]

input =
     1     1     0     0
     1     0     1     0
Further we construct the target vector:
>> target = [0 1 1 0]

target =
     0     1     1     0

Train the Network via Backpropagation

In this example we do not need all the information that the training algorithms shows, so we turn it of by entering:

Let us apply the default training algorithm Levenberg-Marquardt backpropagation trainlm to our network. An additional training parameters is .min_grad. If the gradient of the performance is less than .min_grad the training is ended. To train the network enter:
>> net  = train(net,input,target);
Because of the small size of the network, the training is done in only a second or two. Now we simulate the network, to see how it reacts to the inputs:
>> output = sim(net,input)

output =

    0.0000    1.0000    1.0000    0.0000
That was exactly what we wanted the network to output! Now examine the weights that the training algorithm has set
>> net.IW{1,1}

ans =
   11.0358   -9.5595
   16.8909  -17.5570

>> net.LW{2,1}

ans =
   25.9797  -25.7624