|
|
Overview
The RL Toolbox is a C++ based, open-source, framework for all kinds
of reinforcement learning (RL) algorithms. It can be used for all
types of reinforcement learning tasks, it provides most of the
common RL algorithms but also general interfaces for implementing
new algorithms. In the design of the classes we focused our
attention on the user-friendliness and extensibility, thus it is
easy to implement new learning tasks or also learning algorithms.
This tool allows the user to try out many RL algorithms with many
different parameter settings (which is typically needed in
reinforcement learning to find an algorithm which has a good
performance) without much effort.
Basic Features
- Easy to extend
- Different kind of Learning Algorithms There are many different
kinds of learning algorithm implemented, Q-Learning learning,
TD(\lambda) Learning, Actor critic learning, Advantage Learning,
model based reinforcement learning (prioritized sweeping, value
iteration), policy search algorithms (CONJPOMDP).
- A special focus is on algorithms for continuous control
problems, these algorithms include the residual algorithm,
continuous time RL, LSTD or also batch RL methods like fitted
Q-Iteration
- An huge variety of possibilities to build your v- or
q-functions. Thew toolbox provides tools to easily create a compact
discrete state space for your learning tasks. For continuous
problemens, it also provides several function approximators like
RBF networks, tilings, neural networks or regression trees.
- Computation Speed: The implementation of the toolbox is
optimized for computation speed, depending on the task, we can get
a speed up to comparable matlab programs of a factor up to
100.
- Learning from other Controllers: You can code you own
controller for a learning problem and then try to improve the
controller with reinforcement learning.
- Tools for Logging and Error Recognition and analyzing the v- or
q-functions
- Learning of Semi MDP's with options. The toolbox offers you a
flexible framewokr to implement options, i.e. actions which take
longer than just one step. The most learning algorithms implemented
in the toolbox also consider the duration of the action in their
updates.
- Hierarchical Reinforcement Learning: The toolbox provides a
flexible framework to define subgoals and behaviors in order to
bring more structure in the learning task, which can improve the
performance of RL considerably.
- Batch Mode RL: Batch mode RL usually uses the training data
much more efficiently by storing episodes from the past. The
toolbox provides fast mechanisms to do that, it supports fitted
Q-Iteration with all kinds of function approximators (regression
Trees, local Regression, Neural Networks...) and also Experience
Replay.
Introduction to the System
The manual of the new version is part of the
Master Thesis :
"Reinforcement Learning for optimal control tasks". It contains a
comprehensive description of the class framework, a more detailed
description of the implementation details is given in the
class reference.
Unfortunately not all classes are really good documented, but these
classes are usually not needed by the "standard user". For those
who don't want to read the whole manual, there are some tutorials,
see the example section.
Getting and installing the RL toolbox
The toolbox is written in OS independent c++ code, but the latest
version of the toolbox is based on qmake and only tested on Linux
machines. Windows is not supported any more. Windows users either
have to use an older version of the toolbox or try to port the
project to Windows on their own. You can download the RL Toolbox
here. To install the toolbox,
qmake is needed on your system.
The toolbox has been tested on Fedora Linux
(2.6.16-1.2108_FC4smp) with gcc 4.02 and on SUSE Linux. If you have
found any problems with other distributions, please contact me. For
further installation notes please consult the README.txt file in
the RL_Toolbox directory.
Contact
If you have any comments, suggestions or even have found a bug,
please contact me at
gerhard.neumann@igi.tu-graz.ac.at
Getting involved
You are interested in using or have already used the RL toolbox for
a learning task? Then write us about your learning problem and your
results so we can post it on our homepage.
If you want to implement another learning algorithm, which is
not implemented in the RL Toolbox, please contact us. The toolbox
can be easily extended with the advantage that you can use all the
tools for logging, error recognition and analyzation from the
toolbox.
Visits since 18.07.03:
|
