Reinforcement Learning Toolbox 2.0 last updated:

General Documentation Manual Tutorial Class Reference Master Thesis Examples Related Papers Downloads Links News mailto:webmaster Main Page     Class Hierarchy   Compound List   File List   Compound Members   File Members CAbstractVFunction Class Reference Interface reprenting a Value Function. More... #include <cvfunction.h> Inheritance diagram for CAbstractVFunction: List of all members. Public Member Functions   CAbstractVFunction (CStateProperties *properties)   constructor, the properties are needed to fetch the state from the state collection. virtual  ~CAbstractVFunction () virtual void  resetData () virtual void  updateValue (CStateCollection *state, double td)   Calls updateValue(CState *state, double td) with the state assigned to the value function. virtual void  setValue (CStateCollection *state, double qValue)   Calls setValue(CState *state, double qValue) with the state assigned to the value function. virtual double  getValue (CStateCollection *state)   Calls setValue(CState *state, double qValue) with the state assigned to the value function. virtual void  updateValue (CState *state, double td)   sets the value of the state to the current value + td virtual void  setValue (CState *, double)   sets the value of the state, has to be implemented by the other V-Functions virtual double  getValue (CState *state)=0   returns the value of the state, has to be implemented by the other V-Functions virtual void  saveData (FILE *file)   Saves the Paramters of the Value Function. virtual void  loadData (FILE *file)   Loads the Paramters of the Value Function. virtual void  printValues ()   Prints the Paramters of the Value Function. int  getType () bool  isType (int isT) virtual CAbstractVETraces *  getStandardETraces ()   Returns a standard VETraces object. Public Attributes bool  mayDiverge Protected Member Functions void  addType (int newType) Protected Attributes int  type Detailed Description Interface reprenting a Value Function. Value functions return for each state the total discount reward which they expect to get from that state following a specific policy. Usually this is a Greedy Policy, greedy on the value function. In the RIL toolbox the Q-Functions are composed of v-functions, one v-function for each action, so the Value Function is the essential part part of each learning algorithm. The kind of the value function is for the most learning algorithm independent, so it does'nt matter what value functions you use for the Q-Function, the algorthm just works with the inteface of its v-function. The class CAbstractVFunction is the interface for all value functions. The interface provides a gamma-value for each value function, serving as discount factor. The value functions have to implement functions for getting V-Values , setting V-Values and updating V-Values for an specific state. These three functions are: getValue(CState *). The function returns the expected total discount reward for the given state. setState(CState *, double value). This function is usually used for initialisation of th value function. It has to set the total discount reward of the state to the specified value as good as it is possible (for function approximators). updateValue(CState *, double td) is the function usually used for learning. Adds the td value to the current value of the function. All these function have an companion piece with state collections, which are used by the learning algorithm. So each value function maintains an own state properties object to retrieve the required state from the given state collection and call the demanded function with a state as parameter. In the RIL toolbox there are 3 different kinds of value functions, V-Tables, V-FeatureFunctions and V-Functions using Neural Networks (The Torch toolbox is used for the neural networks). All these value functions support the possibilty to save and load the learned values. The class is subclass of CStateObject, with the consequent state properties object the desired state is fetches from the state collection. The class also has a function getStandardETraces to determine which E-Traces should be used. The function has to return a new instantiated CAbstractVETraces object for the V-Function, which is used to compose the CQETtraces object. The function returns CStateVETraces as standard. Constructor & Destructor Documentation CAbstractVFunction::CAbstractVFunction (  CStateProperties *  properties  )    constructor, the properties are needed to fetch the state from the state collection. virtual CAbstractVFunction::~CAbstractVFunction (   )  [virtual] Member Function Documentation void CAbstractVFunction::addType (  int  newType  )  [protected] virtual CAbstractVETraces* CAbstractVFunction::getStandardETraces (   )  [virtual]   Returns a standard VETraces object. The function has to return a new instantiated CAbstractVETraces object, which is used to compose the CQETtraces object. The function returns CStateVETraces as standard. Reimplemented in CVFunctionFromGradientFunction, CVFunctionSum, CGradientVFunction, CFeatureVFunction, and COptimalVFunctionFromQFunction. int CAbstractVFunction::getType (   )  virtual double CAbstractVFunction::getValue (  CState *  state  )  [pure virtual]   returns the value of the state, has to be implemented by the other V-Functions Implemented in CTorchVFunction, CVFunctionFromGradientFunction, CRegressionTreeVFunction, CFeatureVRegressionTreeFunction, CZeroVFunction, CVFunctionSum, CFeatureVFunction, CRewardAsVFunction, COptimalVFunctionFromQFunction, and CVFunctionFromQFunction. virtual double CAbstractVFunction::getValue (  CStateCollection *  state  )  [virtual]   Calls setValue(CState *state, double qValue) with the state assigned to the value function. Reimplemented in CVFunctionSum, COptimalVFunctionFromQFunction, and CVFunctionFromQFunction. bool CAbstractVFunction::isType (  int  isT  )  virtual void CAbstractVFunction::loadData (  FILE *  file  )  [virtual]   Loads the Paramters of the Value Function. Implements CLearnDataObject. Reimplemented in CGradientVFunction, and CFeatureVFunction. virtual void CAbstractVFunction::printValues (   )  [inline, virtual]   Prints the Paramters of the Value Function. Reimplemented in CFeatureVFunction. virtual void CAbstractVFunction::resetData (   )  [inline, virtual]   Implements CLearnDataObject. Reimplemented in CVFunctionFromGradientFunction, CRegressionTreeVFunction, CGradientVFunction, and CFeatureVFunction. virtual void CAbstractVFunction::saveData (  FILE *  file  )  [virtual]   Saves the Paramters of the Value Function. Implements CLearnDataObject. Reimplemented in CRegressionTreeVFunction, CGradientVFunction, and CFeatureVFunction. virtual void CAbstractVFunction::setValue (  CState *  , double  )  [inline, virtual]   sets the value of the state, has to be implemented by the other V-Functions Reimplemented in CVFunctionFromGradientFunction, and CFeatureVFunction. virtual void CAbstractVFunction::setValue (  CStateCollection *  state, double  qValue )  [virtual]   Calls setValue(CState *state, double qValue) with the state assigned to the value function. virtual void CAbstractVFunction::updateValue (  CState *  state, double  td )  [virtual]   sets the value of the state to the current value + td Reimplemented in CGradientVFunction, and CFeatureVFunction. virtual void CAbstractVFunction::updateValue (  CStateCollection *  state, double  td )  [virtual]   Calls updateValue(CState *state, double td) with the state assigned to the value function. Reimplemented in CGradientVFunction. Member Data Documentation bool CAbstractVFunction::mayDiverge int CAbstractVFunction::type [protected] The documentation for this class was generated from the following file: cvfunction.h