Machine Learning B

LV-Nr: 708.061 VO, 708.062 KU
Course Homepage:
TUGonline link: Machine Learning B


Note: Since Machine Learning A is not offered in WS 2007/08, students can instead attend Machine Learning B as a core course.


Design of Autonomously Learning and Evolving Systems

The current methods for the design of robustly functioning large computing systems are challenged by the need to design larger software systems with very large numbers of interacting components, and also by the need to design hardware devices that carry out computations with very large numbers of somewhat unreliable components on the nanoscale. In addition one needs to develop computing systems (including robots) that can continuously and autonomously learn, i.e., learn without a supervisor which tells them during training at each instance what the ideal action would be. An important task for computer science is to develop tools that will be able to provide this new generation of system design principles and algorithms. Whereas one may doubt whether this task can be solved, nature provides a positive example: Biological organisms consist of extremely large number of communicating components that evolve without a supervisor, and carry out extremely complex computations not only within brains, but also within the gene regulation network of each cell. In addition, biological systems are highly adaptive, i.e., they can learn to cope with changing problems and environments.

The goal of this course is to examine the most promising ideas for the design of autonomously learning and evolving systems, and to introduce the students to software that allows them to ealuate the potential and limitations of these ideas, and to develop better ideas. Whereas the so-called genetic or evolutionary algorithms have dominated research on methods for evolving well-functioning articifial organisms in previous decades (which has lead to some limited success), one has now realized that nature actually uses quite different principles for evolving, improving and maintaining biological organisms than those which were implemented in these algorithms: Since there is no master-agent that supervises the development of biological organisms, nature has invented powerful distributed methods for developing functioning multi-cell organisms, based on gene regulation networks within each cell that follow a complex set of rules (encoded in the DNA) that tell the cell how to respond to external signals (from other cells, or from the environment), and when to build and renew cell components.We will discuss in this course simple mathematical models that capture essential aspects of these design principles, and which provide new methods for the design of robustly functioning, self-repairing, and autonomously learning artificial organisms. In addition we present the main methods of reinforcement learning, which is apparently the primary type of learning that is used by autonomously learning biological organisms.

Covered topics:

* Genetic algorithms (= evolutionary algorithms), Karl Sims demo
* Simulated annealing as an alternative to genetic algorithms
* What is known about the way in which biological species have evolved, and how individual
   organisms develop
* Simple mathematical models for gene regulation networks
* Principles of biological gene regulation networks (mainly from the book by Uri Alon)
* First examples for self-generating artificial circuits and organisms with self-repair capabilities
* Concepts and paradigms for self-organization
* Options for the design of autonomously learning systems
* Algorithms for Reinforcement Learning
* Open research problems