Computational Intelligence, SS08
2 VO 442.070 + 1 RU 708.070 last updated:
Course Notes (Skriptum)
Online Tutorials
Introduction to Matlab
Neural Network Toolbox
OCR with ANNs
Adaptive Filters
VC dimension
Gaussian Statistics
PCA, ICA, Blind Source Separation
Hidden Markov Models
Mixtures of Gaussians
Automatic Speech Recognition
Practical Course Slides
Animated Algorithms
Interactive Tests
Key Definitions
Literature and Links


Using a Neural Network to sovle the problem

The script appcr1.m which is part of the Neural Network Toolbox demonstrates how character recognition can be done with a backpropagation network.

The twenty-six 35-element input vectors are defined in the function prprob as a matrix of input vectors called alphabet. The target vectors are also defined in this file with a variable called targets. Each target vector is a 26-element vector with a 1 in the position of the letter it represents, and 0's everywhere else. For example, the letter ``C'' is to be represented by a 1 in the third element (as ``C'' is the third letter of the alphabet), and 0's everywhere else.

The network receives the 5 x 7 real values as a 35-element input vector. It is then required to identify the letter by responding with a 26-element output vector. The 26 elements of the output vector each represent a letter. To operate correctly, the network should respond with a 1 in the position of the letter being presented to the network. All other values in the output vector should be 0.

In addition, the network should be able to handle noise. In practice, the network does not receive a perfect letter (see Fig. 1) as input. Specifically, the network should make as few mistakes as possible when classifying vectors with noise of mean 0 and standard deviation of 0.2 or less (see Fig. 2).

Network Architecture

The neural network needs 35 inputs and 26 neurons in its output layer to identify the letters. The network is a two-layer network. The log-sigmoid transfer function at the output layer was picked because its output range (0 to 1) is perfect for learning to output boolean values.

The hidden layer has 10 neurons. This number was picked by guesswork and experience. If the network has trouble learning, then neurons can be added to this layer.

The network is trained to output a 1 in the correct position of the output vector and to fill the rest of the output vector with 0's. However, noisy input vectors may result in the network not creating perfect 1's and 0's. After the network is trained the output is passed through the competitive transfer function compet. This makes sure that the output corresponding to the letter most like the noisy input vector takes on a value of 1, and all others have a value of 0. The result of this post-processing is the output that is actually used.


The two-layer network is created with newff.

   S1 = 10;
   [R,Q] = size(alphabet);
   [S2,Q] = size(targets);
   P = alphabet;
   net = newff(minmax(P),[S1 S2],{'logsig' 'logsig'},'traingdx');