Reinforcement Learning Toolbox 2.0
last updated:
General
Documentation
Manual
Tutorial
Class Reference
Master Thesis
Examples
Related Papers
Downloads
Links
News
mailto:webmaster
Main Page     Class Hierarchy   Compound List   File List   Compound Members   File Members

CAbstractQETraces Class Reference

Interface for Q-ETraces. More...

#include <cqetraces.h>

Inheritance diagram for CAbstractQETraces:

CParameterObject CParameters CComposedQETraces CGradientQETraces CQETraces CCALinearFAQETraces List of all members.


Public Member Functions
  CAbstractQETraces (CAbstractQFunction *qFunction)
virtual  ~CAbstractQETraces ()
virtual void  resetETraces ()=0
  Interface function for reseting the ETraces.

virtual void  updateETraces (CAction *action, CActionData *data=NULL)=0
  Interface function for updating the ETraces.

virtual void  addETrace (CStateCollection *State, CAction *action, double factor=1.0, CActionData *data=NULL)=0
  Interface function for adding a State-Action pair with the given factor to the ETraces.

virtual void  updateQFunction (double td)=0
  Interface function for updating the Q-Values of all State-Action Pairs in the ETraces.

virtual void  setLambda (double lambda)
  sets the Parameter "Lambda"

virtual double  getLambda ()
  returns the Parameter "Lambda"

virtual void  setReplacingETraces (bool bReplace)
  sets the parameter "ReplacingETraces"

virtual bool  getReplacingETraces ()
  returns the parameter "ReplacingETraces"



Protected Attributes
CAbstractQFunction *  qFunction
  The assigned Q-Function for updating.

Detailed Description

Interface for Q-ETraces.

Q-ETraces store additionally the action to the state, so you can trace back the episode an make updates to past states. The class provides functions for reseting, updating and add Q-ETraces. In contains the attenuation factor lambda as Parameter "Lambda", which is used to attenuate E-Traces from the past. You can also set, wether the ETraces should be replacing ETraces or not (Parameter "ReplacingETraces"). This parameter is handled by the subclasses slight differently, for more details see the subclasses.

CAbstractQETraces has following parameters:

  • "Lambda", 0.9 : attenuation factor
  • "DiscountFactor", 0.95 : gamma
  • "ReplacingETraces", true

Constructor & Destructor Documentation

CAbstractQETraces::CAbstractQETraces (  CAbstractQFunction *  qFunction  ) 
 
virtual CAbstractQETraces::~CAbstractQETraces (   )  [inline, virtual]
 

Member Function Documentation

virtual void CAbstractQETraces::addETrace (  CStateCollection *  State,
CAction *  action,
double  factor = 1.0,
CActionData *  data = NULL
)  [pure virtual]
 

Interface function for adding a State-Action pair with the given factor to the ETraces.

Implemented in CCALinearFAQETraces, CQETraces, CComposedQETraces, and CGradientQETraces.

virtual double CAbstractQETraces::getLambda (   )  [virtual]
 

returns the Parameter "Lambda"
virtual bool CAbstractQETraces::getReplacingETraces (   )  [virtual]
 

returns the parameter "ReplacingETraces"
virtual void CAbstractQETraces::resetETraces (   )  [pure virtual]
 

Interface function for reseting the ETraces.

Implemented in CQETraces, CComposedQETraces, and CGradientQETraces.

virtual void CAbstractQETraces::setLambda (  double  lambda  )  [virtual]
 

sets the Parameter "Lambda"
virtual void CAbstractQETraces::setReplacingETraces (  bool  bReplace  )  [virtual]
 

sets the parameter "ReplacingETraces"

Reimplemented in CQETraces, and CComposedQETraces.

virtual void CAbstractQETraces::updateETraces (  CAction *  action,
CActionData *  data = NULL
)  [pure virtual]
 

Interface function for updating the ETraces.

I.e. all stored ETraces-factors get multiplied by lambda * gamma, gamma is taken from the Q-Function. If the action is a multistep action, the factor lambda * gamma is exponentiated with the duration.

Implemented in CQETraces, CComposedQETraces, and CGradientQETraces.

virtual void CAbstractQETraces::updateQFunction (  double  td  )  [pure virtual]
 

Interface function for updating the Q-Values of all State-Action Pairs in the ETraces.

Implemented in CQETraces, CComposedQETraces, and CGradientQETraces.

Member Data Documentation

CAbstractQFunction* CAbstractQETraces::qFunction [protected]
 

The assigned Q-Function for updating.
The documentation for this class was generated from the following file: