Seminar Computational Intelligence B (708.112)

SS 2015

Institut für Grundlagen der Informationsverarbeitung (708)

Lecturer:
O.Univ.-Prof. Dr. Wolfgang Maass

Office hours: by appointment (via e-mail)

E-mail: maass@igi.tugraz.at
Homepage: https://igi-web.tugraz.at/people/maass/

Assoc. Prof. Dr. Robert Legenstein

Office hours: by appointment (via e-mail)

E-mail: robert.legenstein@igi.tugraz.at
Homepage: www.igi.tugraz.at/legi/

Location: IGI-seminar room, Inffeldgasse 16b/I, 8010 Graz
Date: starting on Monday, March 2nd 2015, 16:15 - 18.00 p.m. (TUGonline)

Content of the seminar: "New Methods in Machine Learning"

This seminar is designed for master students in their second year, or also in the first year, if they take simultaneously NN B (or have already taken one of the courses listed below).

The papers and talks will be on an introductory level, and will be accessible to an audience with minimal technical background. One didactical goal
of the seminar is to provide experience in giving presentation, which can be practiced here in a relaxed setting.
In addition, students can explore research topics in which they might be interested for a master thesis or -project.

One of the courses NN A, NN B, ML A, ML B suffices as background.

Talks:

Date	Time	Speaker	Topic
Monday, 20.04.2015	16:15	Fuchs Horst Slides Colovic Aleksander Slides	Recurrent Neural Networks (Chapters 3.2 and 3.3 from the upcoming book "Supervised Sequence Labelling with Recurrent Neural Networks" by Alex Graves). LSTMs (Chapter 4 from the upcoming book "Supervised Sequence Labelling with Recurrent Neural Networks" by Alex Graves)
Monday, 04.05.2015	16:15	Oberweger Markus Slides	Do Deep Nets Really Need to be Deep? (Paper by Ba, J., & Caruana, R. (2014))
Monday, 18.05.2015	16:15	Anil Armagan Slides Mahdi Rad Slides	Cross-generalization: Learning novel classes from a single example by feature replacement (Paper by Bart, E., & Ullman, S. (2005)) Recurrent Model of Visual Attention (Paper by Mnih V., Heess N., Graves A., Kavukcuoglu K. (2014))
Monday, 01.06.2015	16:15	Urak Günter Slides	More on LSTMs: vanishing gradients and metaparameter analysis

Options for Talks:

username: lehre
password: on request robert.legenstein@igi.tugraz.at

Talks on sequence-processing by recurrent neural networks:

    1.) Assorted chapters from the upcoming book "Supervised Sequence Labelling with Recurrent Neural Networks" by Alex Graves.
           preprint: http://www.cs.toronto.edu/~graves/preprint.pdf
           Talk 1: Chapter 3.2 and 3.3 on recurrent neural networks
               Talk 2: Chapter 4 on Long-Short-Term-Memory networks (LSTMs)
             Talk 3: Chapter 5 on a comparison of network architectures

    2.) Graves, A. Generating sequences with recurrent neural networks. (2013). In Arxiv preprint arXiv:1308.0850 (possibly two talks)

      3.) Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Advances in Neural Information Processing System
            http://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf

Talks on deep reinforcement learning:

4.) Guo, X., Singh, S., Lee, H., Lewis, R. L., & Wang, X. (2014). Deep learning for real-time Atari game play using offline Monte-Carlo tree search planning. In Advances in Neural Information Processing Systems
http://papers.nips.cc/paper/5421-deep-learning-for-real-time-atari-game-play-using-offline-monte-carlo-tree-search-planning.pdf

5.) R. Legenstein, N. Wilbert, and L. Wiskott (2010). Reinforcement learning on slow features of high-dimensional input streams. PLoS Computational Biology, 6(8):e1000894.
http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1000894

                This paper has material for two talks.
                  Talk 1: on slow feature analysis (SFA)
                  Talk 2: on the combination of SFA with reinforcement learning.

        6.) Mnih V., Heess N., Graves A., Kavukcuoglu K. (2014). Recurrent Models of visual attention. In Advances in Neural Information Processing Systems 27 (NIPS 2014)
              http://papers.nips.cc/paper/5542-recurrent-models-of-visual-attention

Talks on one-shot and transfer learning:

7.) Bart, E., & Ullman, S. (2005). Cross-generalization: Learning novel classes from a single example by feature replacement. In Computer Vision and Pattern Recognition, 2005. CVPR 2005.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.167.3732&rep=rep1&type=pdf

8.) Ba, J., & Caruana, R. (2014). Do Deep Nets Really Need to be Deep?. In Advances in Neural Information Processing Systems
http://papers.nips.cc/paper/5484-do-deep-nets-really-need-to-be-deep.pdf