Apply the four optimization algorithms *gradient descent*, *genetic algorithms*, *simulated annealing*, and *particle swarm optimization* to minimize each of the following four 2-dimensional functions;

- 1)
- Rastrigin:
- 2)
- Rosenbrock:
- 3)
- Ackley:
- 4)
- Chasm:

- a)
- Download the MATLAB code for the genetic algorithm toolbox (
`gatoolbox`), simulated annealing (`anneal.m`) and the calculation of the gradient (`gradient.m`).^{1} - b)
- Implement the particle swarm optimization algorithm. Complete the MATLAB function
`pso.m`. - c)
- Implement the gradient descent algorithm by estimating the gradient with finite differences as provided in the MATLAB function
`gradient.m`. - d)
- In order to carry out the optimization you can write your own MATLAB code or use the code template provided in
`compare.m`. Missing code fragements that have to be completed are marked with "`... HOMEWORK ...`". - e)
- Repeat the minimization for each of the four functions with each of the four algorithms 10 times and calculate the mean and the standard error of the mean (SEM) of the 20 resulting minimum function values and the corresponding run times.
- f)
- Compare the results obtained in e) and interpret possible strengths and weaknesses of each optimization algorithm with respect to the optimized functions. Hand in graphical illustrations of your results that support all your statements.