To create a network that can handle noisy
input vectors it is best to train the network on both ideal and
noisy vectors. To do this, the network is first trained on ideal
vectors until it has a low sum-squared error.
Then, the network is trained on 10 sets of
ideal and noisy vectors. The network is trained on two copies of
the noise-free alphabet at the same time as it is trained on noisy
vectors. The two copies of the noise-free alphabet are used to
maintain the network's ability to classify ideal input
Unfortunately, after the training described
above the network may have learned to classify some difficult noisy
vectors at the expense of properly classifying a noise-free vector.
Therefore, the network is again trained on just ideal vectors. This
ensures that the network responds perfectly when presented with an
All training is done using backpropagation
with both adaptive learning rate and momentum with the function
The network is initially trained without
noise for a maximum of 5000 epochs or until the network sum-squared
error falls beneath 0.1.
P = alphabet;
T = targets;
net.performFcn = 'sse';
net.trainParam.goal = 0.1;
net.trainParam.show = 20;
net.trainParam.epochs = 5000;
net.trainParam.mc = 0.95;
[net,tr] = train(net,P,T);
To obtain a network not sensitive to noise,
we trained with two ideal copies and two noisy copies of the
vectors in alphabet. The target vectors consist of four copies of
the vectors in target. The noisy vectors have noise of std 0.1 and
0.2 added to them. This forces the neuron to learn how to properly
identify noisy letters, while requiring that it can still respond
well to ideal vectors.
To train with noise, the maximum number of
epochs is reduced to 300 and the error goal is increased to 0.6,
reflecting that higher error is expected because more vectors
(including some with noise), are being presented.
netn = net;
netn.trainParam.goal = 0.6;
netn.trainParam.epochs = 300;
T = [targets targets targets targets];
for pass = 1:10
P = [alphabet, alphabet, ...
(alphabet + randn(R,Q)*0.1), ...
(alphabet + randn(R,Q)*0.2)];
[netn,tr] = train(netn,P,T);
Once the network is trained with noise, it
makes sense to train it without noise once more to ensure that
ideal input vectors are always classified correctly. Therefore, the
network is again trained with code identical to the Training
Without Noise section.