Reinforcement Learning Toolbox 2.0 last updated:

General Documentation Manual Tutorial Class Reference Master Thesis Examples Related Papers Downloads Links News mailto:webmaster Main Page     Class Hierarchy   Compound List   File List   Compound Members   File Members CQFunctionFromStochasticModel Class Reference Converts a VFunction and a Model to a Q-Function. More... #include <cqfunction.h> Inheritance diagram for CQFunctionFromStochasticModel: List of all members. Public Member Functions   CQFunctionFromStochasticModel (CFeatureVFunction *vfunction, CAbstractFeatureStochasticModel *model, CFeatureRewardFunction *rewardfunction)   Creates a new QFunction from VFunction object for the given V-Function and the given model, the discretizer is take nfrom the V-Function. virtual  ~CQFunctionFromStochasticModel () virtual void  updateValue (CStateCollection *, CAction *, double, CActionData *=NULL)   Does nothing. virtual void  setValue (CStateCollection *, CAction *, double, CActionData *=NULL)   Does nothing. virtual double  getValue (CStateCollection *state, CAction *action, CActionData *data=NULL)   getValue function for state collections virtual double  getValue (CState *featState, CAction *action, CActionData *data=NULL)   getValue functoin for states virtual double  getValue (int feature, CAction *action, CActionData *data=NULL)   calculates the Action-Value for a specific discrete state number virtual CAbstractQETraces *  getStandardETraces () Protected Attributes CFeatureVFunction *  vfunction   The given V-Function. CAbstractFeatureStochasticModel *  model   The model. CStateProperties *  discretizer   Discretizer used by the V-Function. CFeatureRewardFunction *  rewardfunction   feature Reward Function for the learning problem. CState *  discState   state buffer Detailed Description Converts a VFunction and a Model to a Q-Function. The class calculates the Q-Value by combining the information from a model and a feature V-Function. So the class obviuosly only provides the functions for getting a Q-Value. The Q-Value of an action is calculated the following way: Q(s,a)=sum_{s'} P(s'|s,a)*(R(s,a,s') + gamma * V(s')). This class is used for the policies if you only have a V-Function (e.g. model based learning), since policies can only handle Q-Functions. Constructor & Destructor Documentation CQFunctionFromStochasticModel::CQFunctionFromStochasticModel (  CFeatureVFunction *  vfunction, CAbstractFeatureStochasticModel *  model, CFeatureRewardFunction *  rewardfunction )    Creates a new QFunction from VFunction object for the given V-Function and the given model, the discretizer is take nfrom the V-Function. virtual CQFunctionFromStochasticModel::~CQFunctionFromStochasticModel (   )  [virtual] Member Function Documentation virtual CAbstractQETraces* CQFunctionFromStochasticModel::getStandardETraces (   )  [inline, virtual]   Reimplemented from CAbstractQFunction. virtual double CQFunctionFromStochasticModel::getValue (  int  feature, CAction *  action, CActionData *  data = NULL )  [virtual]   calculates the Action-Value for a specific discrete state number The Q-Value of an action is calculated the following way: Q(s,a)= sum_{s'} P(s'|s,a)*(R(s,a,s') + gamma * V(s')). virtual double CQFunctionFromStochasticModel::getValue (  CState *  featState, CAction *  action, CActionData *  data = NULL )  [virtual]   getValue functoin for states Decomposes the feature state in its discrete state variables and calls the getValue(int, CAction *) function. The results are wheighted by the feature factors and summed up. For discrete states obviously just the getValue(int, CAction *) with the discrete state number is called. virtual double CQFunctionFromStochasticModel::getValue (  CStateCollection *  state, CAction *  action, CActionData *  data = NULL )  [virtual]   getValue function for state collections Calls the getValue function for the specific state (retrieved from the collection by the discretizer) Implements CAbstractQFunction. virtual void CQFunctionFromStochasticModel::setValue (  CStateCollection *  , CAction *  , double  , CActionData *  = NULL )  [inline, virtual]   Does nothing. Reimplemented from CAbstractQFunction. virtual void CQFunctionFromStochasticModel::updateValue (  CStateCollection *  , CAction *  , double  , CActionData *  = NULL )  [inline, virtual]   Does nothing. Reimplemented from CAbstractQFunction. Member Data Documentation CStateProperties* CQFunctionFromStochasticModel::discretizer [protected]   Discretizer used by the V-Function. CState* CQFunctionFromStochasticModel::discState [protected]   state buffer CAbstractFeatureStochasticModel* CQFunctionFromStochasticModel::model [protected]   The model. CFeatureRewardFunction* CQFunctionFromStochasticModel::rewardfunction [protected]   feature Reward Function for the learning problem. CFeatureVFunction* CQFunctionFromStochasticModel::vfunction [protected]   The given V-Function. The documentation for this class was generated from the following file: cqfunction.h