CGradientVETraces Class Reference
ETraces for gradient
functions. More...
#include <cvetraces.h>
Inheritance diagram for CGradientVETraces:
List of all
members.
Detailed Description
ETraces for gradient functions.
This class mantains the etraces for gradient V-Functions. For
the ETraces it uses a feature list, every time an etrace is added
with addETrace the e-trace object calculates the gradient with
respect to the weights from the current state of the Q-Function and
adds this gradient to the e-trace feature list. For gradient
E-Traces there are 2 different ways of adding a gradient, wether
you want to have replacing or non-replacing etraces:
- Non-Replacing ETraces : The gradient is just added to the
current ETraces.
- Replacing ETraces : If the current gradient has the same sign
as the current ETrace, the greater value of both remains the new
E-Trace value. If the signs are different, the current gradient's
value is taken. For updating the gradient Q-Function the E-Trace
object naturally uses the CGradientUpdateFunction
interface of the V-Function.
CGradientQETraces has the
following Parameters: -"Lambda", 0.9 : attenuation factor
-"DiscountFactor", 0.95 : gamma -"ReplacingETraces", see
description for adding a gradient to the etrace list.
-"ETraceTreshold", 0.001 : smallest value of an etrace, the etrace
will be deleted from the list if its lower than this value. Used
for performance reasons. -"ETraceMaxListSize", 100 : Maximum size
of the etrace-list, if the list exceeds this value, the smalles
etraces will be deleted.
Constructor & Destructor Documentation
| virtual
CGradientVETraces::~CGradientVETraces
|
( |
|
) |
[virtual] |
|
Member Function Documentation
| virtual void
CGradientVETraces::addETrace
|
( |
CStateCollection *
|
State,
|
|
|
double |
factor =
1.0 |
|
) |
[virtual] |
|
| virtual void
CGradientVETraces::addGradientETrace
|
( |
CFeatureList *
|
gradient,
|
|
|
double |
factor |
|
) |
[virtual] |
|
| |
Every time an etrace is added with addETrace the e-trace object
calculates the gradient with respect to the weights from the
current state of the V-Function and adds this gradient to the
e-trace feature list. For gradient E-Traces there are 2 different
ways of adding a gradient, wether you want to have replacing or
non-replacing etraces:
- Non-Replacing ETraces : The gradient is just added to the
current ETraces.
- Replacing ETraces : If the current gradient has the same sign
as the current ETrace, the greater value of both remains the new
E-Trace value. If the signs are different, the gradients are
added.
|
| virtual void
CGradientVETraces::multETraces
|
( |
double |
factor |
) |
[virtual] |
|
| virtual void
CGradientVETraces::resetETraces
|
( |
|
) |
[virtual] |
|
| |
Clear the etrace-feature list.
All elements are deleted from the etraces list and added to the
resource list.
Implements CAbstractVETraces.
|
| virtual void
CGradientVETraces::updateETraces
|
( |
int |
duration
= 1 |
) |
[virtual] |
|
| |
Updates all etrace factors.
All etrace factors get multiplied by lambda * gamma^N, where the
duration parameter is N.
Implements CAbstractVETraces.
|
| virtual void
CGradientVETraces::updateVFunction
|
( |
double |
td |
) |
[virtual] |
|
| |
All discrete states in the etrace list gets updated.
The update of the V-Function for state s_i is td * e_i, where
e_i is the etrace factor of s_i.
Implements CAbstractVETraces.
|
Member Data Documentation
The documentation for this class was generated from the following
file:
|
