Reinforcement Learning Toolbox 2.0 last updated:

General Documentation Manual Tutorial Class Reference Master Thesis Examples Related Papers Downloads Links News mailto:webmaster Main Page     Class Hierarchy   Compound List   File List   Compound Members   File Members CFeatureQFunction Class Reference Composed feature Q-Function. More... #include <cqfunction.h> Inheritance diagram for CFeatureQFunction: List of all members. Public Member Functions   CFeatureQFunction (CActionSet *actions, CStateProperties *discretizer)   Creates an Q-Function with the specified discretizer.   CFeatureQFunction (CFeatureVFunction *vfunction, CAbstractFeatureStochasticModel *model, CFeatureRewardFunction *rewardFunction, double gamma)   initializes the Value Functions with the values comming from a V-Function combined with a model and a reward funcion virtual  ~CFeatureQFunction () void  updateValue (CFeature *state, CAction *action, double td, CActionData *data=NULL)   Calls updateValue from the specified V-Function. void  setValue (int state, CAction *action, double qValue, CActionData *data=NULL)   Calls setValue from the specified V-Function. double  getValue (int feature, CAction *action, CActionData *data=NULL)   Returns the Value of a feature for a specific actoin. void  setFeatureCalculator (CStateModifier *discretizer) CStateProperties *  getFeatureCalculator () int  getNumFeatures () void  saveFeatureActionValueTable (FILE *stream)   Saves the Values of the actions for each state in a readable tabular form. void  saveFeatureActionTable (FILE *stream)   Saves the index of the best action for each state in a readable tabular form. Protected Member Functions virtual void  init ()   initializes the V-Function list with CFeatureVFunction objects void  initVFunctions (CFeatureVFunction *vfunction, CAbstractFeatureStochasticModel *model, CFeatureRewardFunction *rewardFunction, double gamma)   initializes the V-Functions with the Values calculated by a V-Function, a theoretical model and a reward function Protected Attributes CStateProperties *  discretizer   Discretizer used for retrieving the state from the state collection. unsigned int  features   number of features from the Value-Functions std::list< CFeatureVFunction * > *  featureVFunctions Detailed Description Composed feature Q-Function. The class CFeatureQFunction is a composed Q-Function which consists of feature value function with the same feature calculator (or the same discretizer). Very often this ia all you need for learning, and CFeatureQFunction objects are easier to create. Its also needed because some learning algorithm expect the value functions to be all of the same kind (prioritized sweeping) and The class also provides methods for manipulating the features directly, without accessing the value functions explicitly. For creation of an feature Q-Function object you only need a state properties object of a feature calculator or discretizer. In addition you have the possibility to initialise you Q-Function with the values from a V-Function combined with a theoretical model (see CQFunctionFromStochasticModel). The difference to CQFunctionFromStochasticModel is, that all Action-Values for all states get calculated and stored in the value Function. With CQFunctionFromStochasticModel the value of the state gets calculated directly by the model and V-Function. So you can convert a Value Function to a Q-Function. Constructor & Destructor Documentation CFeatureQFunction::CFeatureQFunction (  CActionSet *  actions, CStateProperties *  discretizer )    Creates an Q-Function with the specified discretizer. CFeatureQFunction::CFeatureQFunction (  CFeatureVFunction *  vfunction, CAbstractFeatureStochasticModel *  model, CFeatureRewardFunction *  rewardFunction, double  gamma )    initializes the Value Functions with the values comming from a V-Function combined with a model and a reward funcion The actionset is taken from the model, the discretizer from the feature V-Function. The V-Funcions are initialized by the Function initVFunctions. virtual CFeatureQFunction::~CFeatureQFunction (   )  [virtual] Member Function Documentation CStateProperties* CFeatureQFunction::getFeatureCalculator (   )  int CFeatureQFunction::getNumFeatures (   )  double CFeatureQFunction::getValue (  int  feature, CAction *  action, CActionData *  data = NULL )    Returns the Value of a feature for a specific actoin. Allows direct feature manipulation without the need of state objects. virtual void CFeatureQFunction::init (   )  [protected, virtual]   initializes the V-Function list with CFeatureVFunction objects void CFeatureQFunction::initVFunctions (  CFeatureVFunction *  vfunction, CAbstractFeatureStochasticModel *  model, CFeatureRewardFunction *  rewardFunction, double  gamma )  [protected]   initializes the V-Functions with the Values calculated by a V-Function, a theoretical model and a reward function The action values are calculated for each action in each state by the function CDynmaicProgramming::getActionValue and then they aer stored in the V-Functions. void CFeatureQFunction::saveFeatureActionTable (  FILE *  stream  )    Saves the index of the best action for each state in a readable tabular form. Tool for debugging and tracing the learning results void CFeatureQFunction::saveFeatureActionValueTable (  FILE *  stream  )    Saves the Values of the actions for each state in a readable tabular form. Tool for debugging and tracing the learning results void CFeatureQFunction::setFeatureCalculator (  CStateModifier *  discretizer  )  void CFeatureQFunction::setValue (  int  state, CAction *  action, double  qValue, CActionData *  data = NULL )    Calls setValue from the specified V-Function. Allows direct feature manipulation without the need of state objects. void CFeatureQFunction::updateValue (  CFeature *  state, CAction *  action, double  td, CActionData *  data = NULL )    Calls updateValue from the specified V-Function. Allows direct feature manipulation without the need of state objects. Member Data Documentation CStateProperties* CFeatureQFunction::discretizer [protected]   Discretizer used for retrieving the state from the state collection. unsigned int CFeatureQFunction::features [protected]   number of features from the Value-Functions std::list<CFeatureVFunction *>* CFeatureQFunction::featureVFunctions [protected] The documentation for this class was generated from the following file: cqfunction.h