Subsections
Gaussian Mixtures are combinations of Gaussian, or `normal',
distributions. A mixture of Gaussians can be written as a weighted
sum of Gaussian densities.
Recall the dimensional
Gaussian probability density function (pdf):

(1)

with mean vector
and covariance matrix
.
A weighted mixture of
Gaussians can be written as

(2)

where the weights are all positive and sum to one:
In Figure 1 an example is given for an
one dimensional Gaussian mixture, consisting of three single
Gaussians.
Figure 1: One dimensional
Gaussian mixture pdf, consisting of 3 single Gaussians

By varying the number of Gaussians , the weights , and the parameters
and
of each Gaussian
density function, Gaussian mixtures can be used to describe any
complex probability density function.
The parameters of a probability density function are the number
of Gaussians , their
weighting factors ,
and the mean vector
and covariance matrix
of each Gaussian
function.
To find these parameters to optimally fit a certain probability
density function for a set of data, an iterative algorithm, the
expectationmaximization (EM) algorithm can be used. We have
introduced this algorithm in the tutorial about Gaussian statistics
to train parameters of the Gaussian pdfs for several classes in
unsupervised classification. Starting with initial values for all
parameters they are reestimated iteratively.
It is crucial to start with `good' initial parameters as the
algorithm only finds a local, and not a global optimum. Therefore
the solution (to where the algorithm converges) strongly depends on
the initial parameters.
The data used in this experiment have already been used in
previous tutorials. Load the data file BackFront.mat. The
file contains simulated 2dimensional speech features in the form
of artificially generated pairs of formant frequency values (the
first and the second spectral formants, ). The 5 vowels have been
grouped into two classes, the back
vowels containing /a/ and /o/, and the front vowels, containing /e/, /i/, /y/. Since the
distribution for each vowel is Gaussian (cf. tutorial `Gaussian
Statistics and Unsupervised Learning'), each of the two classes,
back vowels and front vowels, shall constitute a Gaussian mixture
distribution. The data samples for the back vowels and front vowels
are saved in backV and frontV.
You can visualize the data by making a 2dimensional plot of the
data:
» plot(frontV(1,:),frontV(2,:))
You can also plot a 2dimensional histogram, with the function
histo.
» histo(frontV(1,:),frontV(2,:))
Recall that you can change the viewpoint in a 3dimensional
MATLAB plot with the mouse, choosing the rotate
option in MATLAB.
It is possible to depict the `real' pdf of the Gaussian mixtures
according to equation 2, considering the
parameters for the pdfs of the single vowels. The mean vectors and
covariance matrices are saved in means_front and
vars_front.
The means are saved in a 3dimensional array. First dimension
covers the mean
of and , the second dimension is of size
1, and the third dimension contains the number of Gaussian
mixtures. The second dimension is used if the pdf is part of a
Hidden Markov Model (HMM) and denotes the number of the state in
the HMM (see BNT toolkit [1]).
To find the mean for for the second Gaussian for the front
vowels (i.e., the mean of the Gaussian distribution of vowel /i/)
use:
» means_front(:,:,2)
The covariance matrices are stored in a 4dimensional array.
First and second dimension contains the covariance matrix
, the third dimension
is of size 1 (for HMMs, see above), and the forth dimension gives
the number of Gaussian mixtures. To get the covariance matrix for
and for the second Gaussian for the
front vowels use:
» vars_front(:,:,:,2)
You can verify the values given in means_front and
vars_front, resp. means_back and
vars_back, by computing the mean vector and the covariance
matrix for each vowel using the MATLAB commands
mean ans cov.
The weighting factors (see equation 2) for
the front/back vowels are stored in
mm_front/mm_back, in the form of a row
vector.
With these parameters we can plot the `true' pdf using the
function mgaussv. It is advisable to specify an xrange
and a yrange for plotting.
» % we take the x and yrange according to the
2dimensional histogram
»
figure;mgaussv(means_front,vars_front,mm_front,71:20:980,633:45:2953)
We like to estimate the parameters of our vowel groups with the
EM algorithm. Use the GaussianMixturesEMExplorer to do that. To
call the explorer type
» BW(data,initial_parameters)
The first input argument are the data that should be modeled by the
pdf. The second input argument is either a set of initial
parameters (consisting of initial means
, covariances
, and weighting
factors ) as a
cell (see the data structure of hmmF or type help
BW in MATLAB), or the number of Gaussian
mixtures that should
be used for the pdf. In the latter case a first guess of the
initial parameters
,
, and is done automatically (using the
command init_mhmm from the Bayes net
toolbox [1]).
Do several runs of the GaussianMixturesEMExplorer with
different initializations:
 Choose some initial parameters for the means, variances and
weighting factors. You have to generate them as specified above,
and connect them as a cell object. See the data structure of
hmmF which gives the true parameter set for the front
vowels.
» BW(frontV,hmmIni)
 The true parameters:
» BW(frontV,hmmF)
 Let the initial parameters be determined automatically and
specify only the number of Gaussian mixtures:
» BW(frontV,3)
Compare with the plots you generated before (histogram, pdf with
true parameters). Describe your observations.
All the examples here were given for the front vowels. Do the
same things for the back vowels.
