Computational Intelligence, SS08
2 VO 442.070 + 1 RU 708.070 last updated:
General
Course Notes (Skriptum)
Online Tutorials
Practical Course Slides
Homework
Exams
Animated Algorithms
Artificial Neuron
Perceptron
Multi Layer Perceptron
Generalisation
RBF Networks
Optical Character Recognition
Prediction
Gaussian Mixture Model
Principal Component Analysis
Interactive Tests
Key Definitions
Downloads
Literature and Links
News
mailto:webmaster

Principal Component Analysis

Introduction

A common method from statistics for analysing data is principal component analysis (PCA). In communication theory, it is known as the Karhunen-Loève transform. The aim is to find a set of M orthogonal vectors in data space that account for as much as possible of the data's variance. Projecting the data from their original N-dimensional space onto the M-dimensional subspace spanned by these vectors then performs a dimensionality reduction that often retains most of the intrinsic information in the data.

The first principal component is taken to be along the direction with the maximum variance. The second principal component is constrained to lie in the subspace perpendicular to the first. Within that subspace, it points in  the direction of the maximum variance. Then, the third principal component (if any) is taken in the maximum variance direction in the subspace perpendicular to the first two, and so on.

Credits

The original applet was written by Olivier Michel.

Implementation

Principal component analysis is implemented as a neural algorithm called APEX (Adaptive Principal component EXtraction) developed by kung and Diamantaras (1990).

Instructions

This applet allows the user to set a number of points in a two dimensional space by clicking with the mouse button. Then, the user may specify a number of iterations for the neural PCA algorithm. When pressing the PCA button, the following calculations occur:

  1. The origin O is computed as the gravity center of the set of points.
  2. The first eigen vector is computed by the neural PCA algorithm. The components of this vector correspond to the weiths between the input layer and the first output unit.
  3. The first eigen vector is displayed in the two dimensional space.

Applet

 

Questions

  • Define a cluster of data points. The cluster should be not perfectly circular but have a preferred direction. Click on PCA and watch whether the algorithm finds the preferred direction.
  • Now add more data points so that the cluster is banana shaped. Click on PCA. Is PCA useful in this case?