Computational Intelligence, SS08
2 VO 442.070 + 1 RU 708.070 last updated:
General
Course Notes (Skriptum)
Online Tutorials
Introduction to Matlab
Neural Network Toolbox
OCR with ANNs
Adaptive Filters
VC dimension
Gaussian Statistics
PCA, ICA, Blind Source Separation
Hidden Markov Models
Mixtures of Gaussians
Automatic Speech Recognition
Practical Course Slides
Homework
Exams
Animated Algorithms
Interactive Tests
Key Definitions
Downloads
Literature and Links
News
mailto:webmaster


Subsections

MATLAB Exercises



LMS Algorithm

Write a MATLAB function y=lms1(x,d,N,mu) which implements an adaptive transversal filter using the LMS algorithm. Use the following example file header as a starting point:

function y = lms1(x,d,N,mu)
% y = lms1(x,d,N,mu)
%   Adaptive transversal filter using LMS
% INPUT
%   x ... vector containing the samples of the input signal x[n]
%         size(x) = [xlen,1] ... column vector
%   d ... vector containing the samples of the desired output signal d[n]
%         size(d) = [xlen,1] ... column vector
%   N ... number of coefficients
%   mu .. step-size parameter
% OUTPUT
%   y ... vector containing the samples of the output signal y[n]
%         size(y) = [xlen,1] ... column vector
Test your function using the following setup:
$\displaystyle x[n]=2$   for$\displaystyle \quad n=0,1,\ldots,999 \qquad d[n]=1$   for$\displaystyle \quad n=0,1,\ldots,999 \qquad N=1 $
Try different values for $ \mu$. Plot $ x[n]$, $ y[n]$, and $ d[n]$ into the same figure by executing plot([x,y,d]) (note: x, y, and d should be column vectors).



Learning Curve

Often we are interested to see how the adaptation of the coefficients works or how the error behaves over time. Therefore we need the function [y,e,c]=lms2(x,d,N,mu) which provides us more output arguments. Extend the function from exercise 6.1 by the additional output arguments.

function [y,e,c] = lms2(x,d,N,mu)
% [y,e,c] = lms2(x,d,N,mu)
%   Adaptive transversal filter using LMS (for algorithm analysis)
% INPUT
%   x ... vector containing the samples of the input signal x[n]
%         size(x) = [xlen,1] ... column vector
%   d ... vector containing the samples of the desired output signal d[n]
%         size(d) = [xlen,1] ... column vector
%   N ... number of coefficients
%   mu .. step-size parameter
% OUTPUT
%   y ... vector containing the samples of the output signal y[n]
%         size(y) = [xlen,1] ... column vector
%   e ... vector containing the samples of the error signal e[n]
%         size(y) = [xlen,1] ... column vector
%   c ... matrix containing the coefficient vectors c[n]
%         size(c) = [N,xlen+1]
Test this function by applying the same input like in exercise 6.1 and plot the squared error $ e^2[n]$ (learning curve). Use a logarithmic scale for the squared error in the learning curve.

Add a random signal to the desired output $ d[n]$ (this would be the signal from a local speaker in an echo cancelation application):

>> dn = d + 1e-6*randn(length(d),1);
and plot the learning curve for the LMS algorithm using the noisy desired output signal. Compare to the learning curve found before. Do the coefficients converge?

System Identification

According to the system identification setup in fig. 3, investigate the ability of an adaptive filter to approximate an FIR filter (i.e., a transversal filter with fixed coefficients $ \mathbf{h}$) of length $ M$ with an impulse response $ \mathbf{h} = \left[ h_0, h_1, \ldots, h_{M-1} \right] ^{\mathsf T}$.

The desired (reference) output of the filter is

$\displaystyle d[n] = \mathbf{h}^{\mathsf H}\mathbf{x}[n]= \sum_{k=0}^{M-1} h_k^{\ast} \, x[n-k]. $

For the unknown system, you can take any $ \mathbf{h}$ and $ M$ you may wish. To calculate $ d[n]$, use the MATLAB function filter().

Use your function lms2() from exercise 6.2 and let $ x[n]$ be normally distributed random numbers with mean zero and variance one (use the MATLAB function randn()). Choose a proper value for the step-size parameter $ \mu$.

Compare the output of the adaptive filter using the LMS algorithm to the desired signal and to the output of an adaptive filter using the RLS algorithm. The RLS algorithm is provided in the MATLAB file rls1.m. The forgetting factor $ \rho $ should be chosen between $ 0.95 \leq \rho \leq 1$.

function [y,e,c1]=rls1(x,d,N,rho)
%Adaptive transversal filter using RLS algorithm
%
%   [y,e,c1]=rls1(x,d,N,rho)
%
% input:
%   x   column vector containing the input samples x[n] (size(x)=[xlen,1])
%   d   column vector containing the samples of the desired output
%       signal d[n] (size(d)=[xlen,1])
%   N   number of coefficients
%   rho forgetting factor
%
% output:
%   y   column vector containing the samples of the
%       output y[n] (size(y)=[xlen,1])
%   e   column vector containing the samples of the
%       error signal e[n] (size(e)=[xlen,1])
%   c   matrix containing the coefficient vectors c[n]
%         size(c) = [N,xlen+1]
%

For the system identification application, write a MATLAB script to visualize the adaptation process in the time domain.

Compare the LMS and the RLS algorithms regarding the time until the coefficients converge.

Additionally, modify your script and examine the cases $ N>M$ and $ N<M$.

Coefficient Adaptation Behavior

For the two-dimensional case ($ N=M=2$), visualize the adaptation path in the $ \mathbf{c}$-plane ( $ \mathbf{c}[n]= \Big[ c_0[n], c_1[n] \Big] ^{\mathsf T}$). Use both algorithms (lms2() and rls1()) and different input signals:

  1. $ x[n] = \mathrm{sign}\left(\mathrm{rand}[n]-0.5\right)$   and$ \quad \mu=0.5$
  2. $ x[n] = \mathrm{randn}[n]$   and$ \quad \mu=0.5$
  3. $ x[n] = \cos[ \pi n ]$   and$ \quad \mu=0.5$
  4. $ x[n] = \cos[ \pi n ] + 2$   and$ \quad \mu=0.1$
Compare the results of this four input signals. Describe your observations.