Seminar Computational Intelligence E (708.115)
Grundlagen der Informationsverarbeitung (708)
Lecturer: O.Univ.-Prof. Dr. Wolfgang Maass
Office hours: by appointment (via e-mail)
Location: IGI-seminar room,
Inffeldgasse 16b/I, 8010 Graz
Date: starting from
Oct. 10, 2005 every Tuesday,
13:30 - 15.00 p.m.
Content of the seminar:
We will discuss in this seminar research articles that represent
the current state-of-the-art in the design of autonomous learners in
machine learning, and the current state of knowledge regarding
biological mechanisms that enable autonomous learning in biological
We will understand in this seminar autonomous learning in a broad
sense, as any mechanisms and algorithms that support learning without
teacher (supervisor). There exist some articles on autonomous
learning of robots, which we will discuss (although they are not too
instructive, as far as I can tell).
But also unsupervised learning is an essential component of any
powerful autonomous learners, and will be review in this seminar the
unsupervised learning of independent components etc.
Apart from unsupervised learning, an autonomous learner can make use of
freely available supervision for prediction learning (where the
environment automatically serves as supervisor for learning
predictions), and from rewards and punishments that the learner
receives from its actions. The latter is usually studied in the context
of Reinforcement Learning, which is covered in the course Machine
Learning B http://www.igi.tugraz.at/maass/lehre.html
(it will be taught again in the year 2006/07).
In this seminar we will focus on aspects of reinforcement learning that
are usually not covered in books and courses on that topic, but which
give a better idea of how reinforcement learning works in biological
In this context we will discuss also some of the research results of
the Austrian-borne Nobel Price Winner Eric Kandel http://almaz.com/nobel/medicine/2000c.html
He and his coworkers have analyzed the learning algorithms in one
particular biological autonomous learner (Aplysia), going down to the
molecular biology of learning mechanisms and the signals that are
involved in controling its learning. Of particular interest are there
results on "heterosynaptic plasticity", which suggest that the commonly
considered Hebbian learning rules are incomplete. We will also present
the controversy about the putative role of dopamine for reward based
learning in more complex biological organisms, and look at abstract
models for the role neuromodulators in the control of learning.
The research results that are presented in this seminar provide a
good introduction to our currently beginning work for the new
There we will focus on the understanding of learning algorithms and
learning mechanisms in increasingly more realistic (and larger) models
of cortical microcircuits and cortical areas.
Required background for active participants of this seminar:
Courses on machine learning and neural networks.
Talks about related current research by team members (most of these
talks should come rather early in the seminar, if possible)
a) Stefan Klampfl on applications of the BCM rule for
extracting independent components from a spiking network
(he could also consider to add material from the new book about
BCM-learning: Cooper, Intrator, Blais, Shouval: Theory of Cortical
b) Rafaela Hechl on applications of learning
rules for slow feature extraction to readouts from circuits of spiking
c) Robert Legenstein about independent component analysis
via rules for competitive Hebbian learning, as well as by a rule
proposed by Foel
Schedule of talks:
Gerhard Neumann - "Report on the recent 3rd International
Adaptive Motion in Animals and Machines"
Amir Saffari - "overcomplete representations"
Learning higher-order structures in natural images
Y Karklin, MS Lewicki - Network Computation in Neural Systems, 2003
A Hierarchical Bayesian Model for Learning Nonlinear Statistical
Regularities in Nonstationary ...
Y Karklin, MS Lewicki - Neural Computation, 2005
Prashant Joshi - Mini-Introduction to PCA:
More detailed material on PCA:
pp. 310 in the book Bishop: Neural Networks for Pattern
Oja rule and Sanger rule as possible neural implementations of PCA:
pp.210-209 (in particular the methods for proving convergence of
these rules should be presented in detail)
The discussion of a possible implementation of Oja's rule via synaptic
Rogert Legenstein - "Some problems and results on nonlinear ICA in
neural networks via rules for competitive Hebbian learning "
Andreas Juffinger - "A review of linear ICA"
Reviews of classical ICA can be found on
(the essential difficulty is to extract the most important
ideas/algorithms for a SHORT presentation in the current seminar)
Martin Bachler - " Variations and applications of ICA for
vision problems, and possible neural implementations"
The original paper by Herault and Jutten proposes at the end an
implementation by a neural network (hardcopy available from me)
Recent work on ICA in the context with neural systems (especially
Malte Rasch - "Nonnegative matrix factorization"
Learning the parts of objects by non-negative matrix factorization
DD Lee, HS Seung - Nature, 1999
Unsupervised Learning by Convex and Conic Coding
DD Lee, HS Seung - NIPS, 1996
Algorithms for Non-negative Matrix Factorization
DD Lee, HS Seung - NIPS, 2000
Ashley Mills - "Gating of Hebbian learning by
reinforcement signals in biological organisms"
Is heterosynaptic modulation essential for stabilizing Hebbian
plasticity and memory ?
CH Bailey, M Giustetto, YY Huang, RD Hawkins, ER, Kandel, Nat Rev
internal use only: file:///home/mammoth/ashley/presentations/ap/ap.html
Possibly one could add mterial from the book
Squire, Kandel: Memory: From Mind to Molecules
Jonathan Gutschi - "Bootstrap learning for object discovery"
J. Modayil and B. Kuipers. 2004.
Bootstrap learning for object discovery.
IEEE/RSJ International Conference on Intelligent Robots and Systems
Martin Ebner - "Learning to autonomously select landmarks for
navigation and communication"
by J. Fleischer and S Marsland
Stefan Häusler - "Cascade models of synaptically stored memories"
Fusi, Drew, Abbott, Neuron, 2005, 599-611
Michael Pfeiffer: "Dopamine and other neuromodulators as signals for
reinforcement learning in
biological organisms (and alternative interpretations of their