next up previous
Next: Outer product approximation [3 Up: NNA_Exercises_2012 Previous: Iterative Reweighted Least Squares

Logistic regression [3 P]

Apply logistic regression to the data set vehicle.mat2. The task is to classify a given silhouette as one of two types of vehicles, i.e. SAAB and BUS, using a set of features extracted from the silhouette. You are required to use MATLAB for this assignment.

a)

Use the MATLAB command mapstd to normalize the values of each feature so that the mean is 0 and the variance is 1.

b)
[1 P]

Perform gradient descent on the cross-entropy error function. Initialize the weights to small non-zero values (use the MATLAB command randn). Adjust the learning rate $ \eta$ used for the weight update step

$\displaystyle {\bf w}^{(new)} = {\bf w}^{(old)} - \eta \nabla E_{CE}({\bf w})$

with

$\displaystyle \nabla E_{CE}({\bf w}) = \sum_{n=1}^N (y_n - t_n) {\bf x}_n $

and $ y_n = \sigma({\bf w}^T {\bf x}_n)$ to minimize the number of epochs needed to reach close to optimal performance. Report the resulting learning rate $ \eta_{fast}$ , the performance after 10000 epochs. Hand in a plot showing the dependence of the error on the number of epochs for $ \eta = \eta_{fast}$ .

c)
[1 P]

Perform linear regression using the Moore-Penrose pseudo-inverse $ (\Phi^T \Phi)^{-1}\Phi^T$ of the design matrix $ \Phi$ . Compare the resulting cross-entropy error to the one obtained in b).

d)
[1 P]

Apply the iterative reweighted least squares algorithm to the dataset. Report the performance after 100 epochs and the number of epochs needed to reach close to optimal performance. Hand in a plot showing the dependence of the error on the number of epochs. Compare the results to the one obtained in b).

Present your results clearly, structured and legible.


next up previous
Next: Outer product approximation [3 Up: NNA_Exercises_2012 Previous: Iterative Reweighted Least Squares
Haeusler Stefan 2013-01-16