Computational Intelligence, SS08
2 VO 442.070 + 1 RU 708.070 last updated:
Course Notes (Skriptum)
Online Tutorials
Practical Course Slides
Animated Algorithms
Interactive Tests
Key Definitions
Literature and Links

Homework 26: Digit classification with backprop

[Points: 8; Issued: 2005/03/18; Deadline: 2005/04/28; Tutor: Ilire Hasani; Infohour: 2005/04/25, 13:00-14:00, HSi13; Einsichtnahme: 2005/05/16, 13:00-14:00, HSi13; Download: pdf; ps.gz]

Similarly to the Optical Character Recognition tutorial you are asked to train a feed forward network to perform a digit classification task. The difference to the tutorial is the data set: here you use the Digits data set. The file digits.mat (which you get when unzipping contains about 4000 images (8 pixel $ \times$ 8 pixel $ \times$ 16 colors) of training samples (learn.P) of hand drawn digits (0,1,...,9) and their classification (learn.T) and about 2000 images of test samples (test.P, test.T).

  • The goal is to find a suitable network architecture (i.e. number of layers and number of hidden units) and training parameters (i.e. learning rate and momentum term) for the training function traingdx1 such that more then 95% of the test examples are correctly classified.
  • Report the performance (i.e. percentage of test examples correctly classified) of at least 3 network architectures you have tried. One of them must be a network with no hidden units.
  • Discuss the relationship between the network architecture and the performance on the test set.
  • What can one conclude about the complexity of the task if you consider the performance of the network without any hidden units?


Each member of the team handing in the network with the highest performance will get 3 *-points.


  • Normalize the data using prestd and trastd.
  • Use early stopping as termination criterion. Hence you have to choose a suitable subset of the training data as a validation set.
  • To see how the digits look like you can use the commands
      imagesc(x); colormap(1-gray);
    to produce an image of the i-th training example.


  • Present your results clearly, structured and legible. Document them in such a way that anybody can reproduce them effortless.
  • Please hand in the print out of the Matlab program you have used.


... traingdx1
Gradient descent with momentum and adaptive learning rate.