Maschinelles Lernen B, WS 2004  
Institut für Grundlagen der Informationsverarbeitung (708)  last changes: 01.02.05 
Aim of the Course:
The goal is to give an overview of machine learning methods for
artificial agents, and to apply such methods for the solution of
exercise problems.
Teaching Methods:
Students are expected to learn how to solve autonomous learning problems, and which algorithms are suitable for what kind of application.
Course Prerequisites:
Computational Intelligence, Statistical Methods
This page lists all updates of this course homepage. It will be kept uptodate during the semester. 
01.02.05  The results of the test and the final grades are now online. Happy holidays! 
26.01.05  The results for the 3rd problem set are now online. The last presentation hour will take place on Friday, January 26. There also topics for projects and diploma theses are presented if you are interested in continuing with machine learning. 
26.01.05  The fastest times for the sail challenge can be found here. 
14.01.05  Please register via TUGOnline for the test (1st February) until 28th of January. 
14.01.05  The examples for the GA toolbox that were presented in the lab session are now available for download here. 
10.01.05  Several new links to material on genetic algorithms has been added to the Literature and Links sections. 
10.01.05  The next lab session will take place on Friday, January 14th. 
23.12.04  The simulation environment for the sailchallenge task (task 3) is now available here: sail_challenge.zip 
14.12.04  The third and last problem set is now available. The environment files for task 3 will be made available soon. 
25.11.04  An alternative program for exercise 2 is available here: multipole.zip. You can use also this program instead of the MultipoleTutorial that comes with the RLToolbox. Please indicate on your solution which program you used to solve this task. 
12.11.04  The next lab session will be on Friday November 26th. 
12.11.04  The new version of the Reinforcement Learning Toolbox is now available at www.igi.tugraz.at/riltoolbox. There you will also find demos presented during the tutorial. 
29.10.04  The next lab session will be on Friday November 5th. This will be a tutorial for the Reinforcement Learning Toolbox that you will use for Problem Set 2. The first presentation hour will take place on November 12th, 13:15  14:45. 
27.10.04  A compilation of all the theorems and corollaries that were covered in lecture 3 is now available here and in the section Course Material. 
27.10.04  A reminder: The next lab session will take place Friday, 29th October at 13:15 in HS i11. Use this hour to ask any questions regarding problem set 1. 
19.10.04  Since there seems to be a problem with the website of the Sutton and Barto book, I have added a link to a full PDF version of the book in the Course Material section. The direct link is http://www.cs.ttu.edu/~pazhayan/rlBook.pdf. 
18.10.04  The first problem set is now available here. Solutions have to be submitted before November 2nd 2004. 
08.10.04  There will be no practicals on 15th and 22nd of October. The next practicals will take place on October 29th. 
05.10.04  IMPORTANT: From next week on (12.10.04) the lecture will take place in Seminarraum IGI, Inffeldgasse 16b, 1st floor. The practicals will still be held in lecture hall i11. 
04.10.04  This homepage is created. I hope you will make use of the services that we offer you here. If you have any suggestions or complaints concerning this homepage please send us an email. 
Nr.  Issued  Submission  Link  Additional Material 
1  18.10.04  02.11.04  Problem Set 1  
2  02.11.04  14.12.04  Problem Set 2  multipole.zip Alternative Program for Exercise 2 
3  14.12.04  25.01.05  Problem Set 3  sail_challenge.zip: Simulation environment for Exercise 3 ga.zip: GA Toolbox Sail Challenge Hall of Fame 
If you have any questions or problems, please do not hesitate to contact one of the above persons. 
Lecture  Date  Topic 
1  08.10.04  Organization, Exploration/Exploitation Dilemma 
2  29.10.04  Problem Set 1, Value Functions 
3  05.11.04  Tutorial for Reinforcement Learning Toolbox 
4  12.11.04  First Presentation Hour (13:15  14:45) 
5  26.11.04  Problem Set 2 
6  10.12.04  Problem Set 2 
7  17.12.04  Second Presentation Hour (13:15  14:45) 
8  14.01.05  Genetic Algorithms 
9  28.01.05  Third Presentation Hour 
Genetic Algorithms:
Lecture  Date  Topic  Course Material 
1  05.10.04  Introduction to Reinforcement Learning  Slides from Sutton / Barto: 
2  12.10.04  The Reinforcement Learning Problem  Slides from Sutton / Barto: 
3  19.10.04  Theory of Reinforcement Learning 1/2  Theorems (PDF) 
4  02.11.04  Theory of Reinforcement Learning 2/2  
5  09.11.04  Temporal Difference Learning, Eligibility Traces  Slides from Sutton / Barto: 
6  16.11.04  Adaptive Control, Humanoid Robots  
7  23.11.04  Function Approximation in RL  Slides from Sutton / Barto: 
8  30.11.04  Hierarchical Reinforcement Learning  Papers:

9  07.12.04  Policy Gradient RL  Papers:

10  14.12.04  RL for Motor Control  Slides: 
11  11.01.05  Evolutionary Algorithms  
12  18.01.05  Learning and Evolution, GA in nature 
Lecture  Date  Topic  Slides 
1  08.10.04  Organization, Exploration/Exploitation Dilemma  (PDF) 
2  29.10.04  Problem Set 1, Value Functions  (PDF) 
3  05.11.04  Tutorial: Reinforcement Learning Tutorial  Old Tutorial (PDF) 
4  12.11.04  Presentation Problem Set 1  
5  26.11.04  Problem Set 2, On / offpolicy Learning, Selfplay Learning  (PDF) 
6  10.12.04  Problem Set 2, Policy Gradient RL  (PDF) 
7  17.12.04  Presentation Problem Set 2  
8  14.01.05  Genetic Algorithms and Toolbox 
Chapter  Topic  Slides 
1  Introduction  Chapter 1 
2  Evaluative Feedback  Chapter 2 
3  The Reinforcement Learning Problem  Chapter 3 
4  Dynamic Programming  Chapter 4 
5  Monte Carlo Methods  Chapter 5 
6  Temporal Difference Learning  Chapter 6 
7  Eligibility Traces  Chapter 7 
8  Generalization and Function Approximation  Chapter 8 
9  Planning and Learning  Chapter 9 
10  Dimensions of Reinforcement Learning  Chapter 10 
11  Case Studies  Chapter 11 