Seminar Computational Intelligence B (708.112)

SS 2015

Institut für Grundlagen der Informationsverarbeitung (708)

Lecturer:
O.Univ.-Prof. Dr. Wolfgang Maass

Office hours: by appointment (via e-mail)

E-mail: maass@igi.tugraz.at
Homepage: www.igi.tugraz.at/maass/


Assoc. Prof. Dr. Robert Legenstein

Office hours: by appointment (via e-mail)

E-mail: robert.legenstein@igi.tugraz.at
Homepage: www.igi.tugraz.at/legi/




Location: IGI-seminar room, Inffeldgasse 16b/I, 8010 Graz
Date: starting on Monday, March 2nd 2015, 16:15 - 18.00 p.m.
  (TUGonline)

Content of the seminar:  "New Methods in Machine Learning"


This seminar is designed for master students in their second year, or also in the first year, if they take simultaneously NN B (or have already taken one of the courses listed below).

The papers and talks will be on an introductory level, and will be accessible to an audience with minimal technical background. One didactical goal
of the seminar is to provide experience in giving presentation, which can be practiced here in a relaxed setting.
In addition, students can explore research topics in which they might be interested for a master thesis or -project.

One of the courses NN A, NN B, ML A, ML B suffices as background.



Talks:

Date

Time

Speaker

Topic

Monday, 20.04.2015

16:15

Fuchs Horst  Slides

Colovic Aleksander  Slides

Recurrent Neural Networks (Chapters 3.2 and 3.3 from the upcoming book "Supervised Sequence Labelling with Recurrent Neural Networks" by Alex Graves).

LSTMs (Chapter 4 from the upcoming book "Supervised Sequence Labelling with Recurrent Neural Networks" by Alex Graves)


Monday, 04.05.2015


16:15


Oberweger Markus  Slides


Do Deep Nets Really Need to be Deep? (Paper by Ba, J., & Caruana, R. (2014))


Monday, 18.05.2015


16:15


Anil Armagan  Slides

Mahdi Rad  Slides


Cross-generalization: Learning novel classes from a single example by feature replacement (Paper by
Bart, E., & Ullman, S. (2005))

Recurrent Model of Visual Attention (Paper by Mnih V., Heess N., Graves A., Kavukcuoglu K. (2014))


Monday, 01.06.2015


16:15


Urak Günter  Slides


More on LSTMs: vanishing gradients and metaparameter analysis


Options for Talks:

username: lehre
password: on request robert.legenstein@igi.tugraz.at


Talks on sequence-processing by recurrent neural networks:

        1.) Assorted chapters from the upcoming book "Supervised Sequence Labelling with Recurrent Neural Networks" by Alex Graves.
               preprint: http://www.cs.toronto.edu/~graves/preprint.pdf
               Talk 1: Chapter 3.2 and 3.3 on recurrent neural networks
               Talk 2: Chapter 4 on Long-Short-Term-Memory networks (LSTMs)
               Talk 3: Chapter 5 on a comparison of network architectures

        2.) Graves, A. Generating sequences with recurrent neural networks. (2013). In Arxiv preprint arXiv:1308.0850 (possibly two talks)

        3.) Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Advances in Neural Information Processing System
              http://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf

Talks on deep reinforcement learning:

        4.) Guo, X., Singh, S., Lee, H., Lewis, R. L., & Wang, X. (2014). Deep learning for real-time Atari game play using offline Monte-Carlo tree search planning. In Advances in Neural Information Processing Systems
              http://papers.nips.cc/paper/5421-deep-learning-for-real-time-atari-game-play-using-offline-monte-carlo-tree-search-planning.pdf


        5.) R. Legenstein, N. Wilbert, and L. Wiskott (2010). Reinforcement learning on slow features of high-dimensional input streams. PLoS Computational Biology, 6(8):e1000894.
              http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1000894

           
                    This paper has material for two talks.
                    Talk 1: on slow feature analysis (SFA)
                    Talk 2: on the combination of SFA with reinforcement learning.
       
        6.) Mnih V., Heess N., Graves A., Kavukcuoglu K. (2014). Recurrent Models of visual attention. In Advances in Neural Information Processing Systems 27 (NIPS 2014)
              http://papers.nips.cc/paper/5542-recurrent-models-of-visual-attention


Talks on one-shot and transfer learning:

        7.) Bart, E., & Ullman, S. (2005). Cross-generalization: Learning novel classes from a single example by feature replacement. In Computer Vision and Pattern Recognition, 2005. CVPR 2005.
              http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.167.3732&rep=rep1&type=pdf


        8.) Ba, J., & Caruana, R. (2014). Do Deep Nets Really Need to be Deep?. In Advances in Neural Information Processing Systems
              http://papers.nips.cc/paper/5484-do-deep-nets-really-need-to-be-deep.pdf