Reinforcement Learning Toolbox 2.0
last updated:
General
Documentation
Manual
Tutorial
Class Reference
Master Thesis
Examples
Related Papers
Downloads
Links
News
mailto:webmaster
Main Page     Class Hierarchy   Compound List   File List   Compound Members   File Members

CQFunction Class Reference

Compounded Q-Function consisting of V-Funcions. More...

#include <cqfunction.h>

Inheritance diagram for CQFunction:

CGradientQFunction CAbstractQFunction CGradientUpdateFunction CActionObject CLearnDataObject CParameterObject CLearnDataObject CParameterObject CParameters CParameterObject CParameters CParameters CFeatureQFunction List of all members.


Public Member Functions
  CQFunction (CActionSet *actions)
  Creates a composed Q-Function for the given actions.

virtual  ~CQFunction ()
virtual void  updateValue (CStateCollection *state, CAction *action, double td, CActionData *data=NULL)
  Updates the Value of the value function assigned to the given action.

virtual void  setValue (CStateCollection *state, CAction *action, double qValue, CActionData *data=NULL)
  Sets the Value of the value function assigned to the given action.

virtual double  getValue (CStateCollection *state, CAction *action, CActionData *data=NULL)
  Returns the Value of the value function assigned to the given action.

virtual void  updateValue (CState *state, CAction *action, double td, CActionData *data=NULL)
  Updates the Value of the value function assigned to the given action.

virtual void  setValue (CState *state, CAction *action, double qValue, CActionData *data=NULL)
  Sets the Value of the value function assigned to the given action.

virtual double  getValue (CState *state, CAction *action, CActionData *data=NULL)
  Returns the Value of the value function assigned to the given action.

virtual void  saveData (FILE *file)
  Saves the Value Functoins.

virtual void  loadData (FILE *file)
  Loads the Value Functions.

virtual void  printValues ()
  Calls saveValues with stdout as outputstream.

CAbstractVFunction *  getVFunction (CAction *action)
  Returns the value function assigned to the given action.

CAbstractVFunction *  getVFunction (int index)
  Returns the indexth value function (so the value function assigned to the indexth action).

void  setVFunction (CAction *action, CAbstractVFunction *vfunction, bool bDeleteOld=true)
  Sets the Value-Function of the specified action.

void  setVFunction (int index, CAbstractVFunction *vfunction, bool bDeleteOld=true)
  Sets the Value-Function of the indexth action.

int  getNumVFunctions ()
  Returns the number of V-Functions, which is always the number of Actions.

virtual CAbstractQETraces *  getStandardETraces ()
virtual void  getGradient (CStateCollection *state, CAction *action, CActionData *data, CFeatureList *gradient)
virtual int  getNumWeights ()
  Returns the number of weights.

virtual void  getWeights (double *weights)
  Function for getting all weights.

virtual void  setWeights (double *weights)
  Function for setting all weights.

virtual void  resetData ()
  Interface for resetting the weights.

virtual void  copy (CLearnDataObject *qFunction)


Protected Member Functions
virtual int  getWeightsOffset (CAction *action)
virtual void  updateWeights (CFeatureList *features)
  Interface for updating the weights.



Protected Attributes
std::map< CAction *, CAbstractVFunction * > *  vFunctions
  The list of V-Functions.

Detailed Description

Compounded Q-Function consisting of V-Funcions.

For Q-Functions there is one value function for each action of the Q-Function, so its obvious to compose Q-Functions of value Functions. RIL toolbox gives you the possibility to do this. This has the advantage that you can choose an own value functions for each action, so its possible to create own discretizations or even other kinds of value functions for an action. This possibility is only available for certain algorithm, e.g. the model based prioritized sweeping algorithm expects an Q-Function consisting of feature value functions with the same feature calculator (at least with the same number of features). The composition of value function is modeled by the class CQFunction. The class maintains a list of value functions, which has the same size as the action set of the Q-Function. You can assign specific value functions to specific actions. This is done by setVFunction.

If one of the functions for accessing the Q-Values is called (getValue, setValue, updateValue) the composed Q-Functions refers the call to the value function of the specified action. There are 2 subclasses of CQFunction, one for feature Q-Functions and one for Q-Tables.

Constructor & Destructor Documentation

CQFunction::CQFunction (  CActionSet *  actions  ) 
 

Creates a composed Q-Function for the given actions.

The Value-Functions list is initialized with NULL with the same size of the action set, so the V-Functions has to be set by the user with the function setVFunction.

virtual CQFunction::~CQFunction (   )  [virtual]
 

Member Function Documentation

virtual void CQFunction::copy (  CLearnDataObject *  qFunction  )  [virtual]
 

Reimplemented from CGradientQFunction.

virtual void CQFunction::getGradient (  CStateCollection *  state,
CAction *  action,
CActionData *  data,
CFeatureList *  gradient
)  [virtual]
 

Implements CGradientQFunction.

int CQFunction::getNumVFunctions (   ) 
 

Returns the number of V-Functions, which is always the number of Actions.

virtual int CQFunction::getNumWeights (   )  [virtual]
 

Returns the number of weights.

Implements CGradientUpdateFunction.

virtual CAbstractQETraces* CQFunction::getStandardETraces (   )  [virtual]
 

Reimplemented from CGradientQFunction.

virtual double CQFunction::getValue (  CState *  state,
CAction *  action,
CActionData *  data = NULL
)  [virtual]
 

Returns the Value of the value function assigned to the given action.

Returns the value of the getValue Function of the specified value function. The given state must be the state used by the value function (at least a state that can be used by the value function.

virtual double CQFunction::getValue (  CStateCollection *  state,
CAction *  action,
CActionData *  data = NULL
)  [virtual]
 

Returns the Value of the value function assigned to the given action.

Returns the value of the getValue Function of the specified value function.

Implements CAbstractQFunction.

CAbstractVFunction* CQFunction::getVFunction (  int  index  ) 
 

Returns the indexth value function (so the value function assigned to the indexth action).

CAbstractVFunction* CQFunction::getVFunction (  CAction *  action  ) 
 

Returns the value function assigned to the given action.

virtual void CQFunction::getWeights (  double *  weights  )  [virtual]
 

Function for getting all weights.

The double array is assumed to be large enough. This isn't checked!

Implements CGradientUpdateFunction.

virtual int CQFunction::getWeightsOffset (  CAction *  action  )  [protected, virtual]
 

Reimplemented from CGradientQFunction.

virtual void CQFunction::loadData (  FILE *  file  )  [virtual]
 

Loads the Value Functions.

Calls the loadValues method of all Value Functions. So the value Function list has already to be initialized and the V-Functions has to be in the same order as they were when the Q-Function was save.

Reimplemented from CGradientQFunction.

virtual void CQFunction::printValues (   )  [virtual]
 

Calls saveValues with stdout as outputstream.

Reimplemented from CAbstractQFunction.

virtual void CQFunction::resetData (   )  [virtual]
 

Interface for resetting the weights.

Reimplemented from CGradientQFunction.

virtual void CQFunction::saveData (  FILE *  file  )  [virtual]
 

Saves the Value Functoins.

Calls the saveValues method of all Value Functions

Reimplemented from CGradientQFunction.

virtual void CQFunction::setValue (  CState *  state,
CAction *  action,
double  qValue,
CActionData *  data = NULL
)  [virtual]
 

Sets the Value of the value function assigned to the given action.

Calls the setValue Function of the specified value function. The given state must be the state used by the value function (at least a state that can be used by the value function.

virtual void CQFunction::setValue (  CStateCollection *  state,
CAction *  action,
double  qValue,
CActionData *  data = NULL
)  [virtual]
 

Sets the Value of the value function assigned to the given action.

Calls the setValue Function of the specified value function.

Reimplemented from CAbstractQFunction.

void CQFunction::setVFunction (  int  index,
CAbstractVFunction *  vfunction,
bool  bDeleteOld = true
) 
 

Sets the Value-Function of the indexth action.

If bDeleteOld is true (default) the old Value Function is deleted

void CQFunction::setVFunction (  CAction *  action,
CAbstractVFunction *  vfunction,
bool  bDeleteOld = true
) 
 

Sets the Value-Function of the specified action.

If bDeleteOld is true (default) the old Value Function is deleted

virtual void CQFunction::setWeights (  double *  weights  )  [virtual]
 

Function for setting all weights.

The double array is assumed to be large enough. This isn't checked!

Implements CGradientUpdateFunction.

virtual void CQFunction::updateValue (  CState *  state,
CAction *  action,
double  td,
CActionData *  data = NULL
)  [virtual]
 

Updates the Value of the value function assigned to the given action.

Calls the updateValue Function of the specified value function. The given state must be the state used by the value function (at least a state that can be used by the value function)

virtual void CQFunction::updateValue (  CStateCollection *  state,
CAction *  action,
double  td,
CActionData *  data = NULL
)  [virtual]
 

Updates the Value of the value function assigned to the given action.

Calls the updateValue Function of the specified value function.

Reimplemented from CGradientQFunction.

virtual void CQFunction::updateWeights (  CFeatureList *  features  )  [protected, virtual]
 

Interface for updating the weights.

Implements CGradientUpdateFunction.

Member Data Documentation

std::map<CAction *, CAbstractVFunction *>* CQFunction::vFunctions [protected]
 

The list of V-Functions.

Has the same order as the actionset, so the first qFunction coresponds to the first action.

The documentation for this class was generated from the following file: