Reinforcement Learning Toolbox 2.0 last updated:

General Documentation Manual Tutorial Class Reference Master Thesis Examples Related Papers Downloads Links News mailto:webmaster Main Page     Class Hierarchy   Compound List   File List   Compound Members   File Members CGradientQETraces Class Reference E-Trace class for all gradient Q-Functions. More... #include <cqetraces.h> Inheritance diagram for CGradientQETraces: List of all members. Public Member Functions   CGradientQETraces (CGradientQFunction *qfunction) virtual  ~CGradientQETraces () virtual void  resetETraces ()   Resets the etrace object, clears the etrace feature list. virtual void  addETrace (CStateCollection *State, CAction *action, double factor=1.0, CActionData *data=NULL)   Calls addGradientETrace with the current gradient. virtual void  addGradientETrace (CFeatureList *gradient, double factor=1.0)   Adds the current gradient to the etrace list. virtual void  updateETraces (CAction *action, CActionData *data=NULL)   Multiplies as usaul all etraces with lambda * gamma. virtual void  updateQFunction (double td)   Updates the QFunction with its CGradientUpdateFunction interface. CFeatureList *  getGradientETraces ()   returns the current gradient list. Protected Attributes CGradientQFunction *  gradientQFunction CFeatureList *  eTrace CFeatureList *  gradient Detailed Description E-Trace class for all gradient Q-Functions. This class mantains the etraces for gradient q-Functions. For the ETraces it uses a feature list, every time an etrace is added with addETrace the e-trace object calculates the gradient with respect to the weights from the current state of the Q-Function and adds this gradient to the e-trace feature list. For gradient E-Traces there are 2 different ways of adding a gradient, wether you want to have replacing or non-replacing etraces: Non-Replacing ETraces : The gradient is just added to the current ETraces. Replacing ETraces : If the current gradient has the same sign as the current ETrace, the greater value of both remains the new E-Trace value. If the signs are different, the current gradient's value is taken. For updating the gradient Q-Function the E-Trace object naturally uses the CGradientUpdateFunction interface of the Q-Function. CGradientQETraces has the following Parameters: -"Lambda", 0.9 : attenuation factor -"DiscountFactor", 0.95 : gamma -"ReplacingETraces", see description for adding a gradient to the etrace list. -"ETraceTreshold", 0.001 : smallest value of an etrace, the etrace will be deleted from the list if its lower than this value. Used for performance reasons. -"ETraceMaxListSize", 100 : Maximum size of the etrace-list, if the list exceeds this value, the smalles etraces will be deleted. Constructor & Destructor Documentation CGradientQETraces::CGradientQETraces (  CGradientQFunction *  qfunction  )  virtual CGradientQETraces::~CGradientQETraces (   )  [virtual] Member Function Documentation virtual void CGradientQETraces::addETrace (  CStateCollection *  State, CAction *  action, double  factor = 1.0, CActionData *  data = NULL )  [virtual]   Calls addGradientETrace with the current gradient. Implements CAbstractQETraces. virtual void CGradientQETraces::addGradientETrace (  CFeatureList *  gradient, double  factor = 1.0 )  [virtual]   Adds the current gradient to the etrace list. Every time an etrace is added with addETrace the e-trace object calculates the gradient with respect to the weights from the current state of the Q-Function and adds this gradient to the e-trace feature list. For gradient E-Traces there are 2 different ways of adding a gradient, wether you want to have replacing or non-replacing etraces: Non-Replacing ETraces : The gradient is just added to the current ETraces. Replacing ETraces : If the current gradient has the same sign as the current ETrace, the greater value of both remains the new E-Trace value. If the signs are different, the gradient's values are added. CFeatureList* CGradientQETraces::getGradientETraces (   )  [inline]   returns the current gradient list. virtual void CGradientQETraces::resetETraces (   )  [virtual]   Resets the etrace object, clears the etrace feature list. Implements CAbstractQETraces. virtual void CGradientQETraces::updateETraces (  CAction *  action, CActionData *  data = NULL )  [virtual]   Multiplies as usaul all etraces with lambda * gamma. Implements CAbstractQETraces. virtual void CGradientQETraces::updateQFunction (  double  td  )  [virtual]   Updates the QFunction with its CGradientUpdateFunction interface. Implements CAbstractQETraces. Member Data Documentation CFeatureList* CGradientQETraces::eTrace [protected] CFeatureList* CGradientQETraces::gradient [protected] CGradientQFunction* CGradientQETraces::gradientQFunction [protected] The documentation for this class was generated from the following file: cqetraces.h