Reinforcement Learning Toolbox 2.0 last updated:

General Documentation Manual Tutorial Class Reference Master Thesis Examples Related Papers Downloads Links News mailto:webmaster Main Page     Class Hierarchy   Compound List   File List   Compound Members   File Members CFeatureVFunction Class Reference #include <cvfunction.h> Inheritance diagram for CFeatureVFunction: List of all members. Public Member Functions   CFeatureVFunction (int numFeatures)   CFeatureVFunction (CStateProperties *featureFact)   Creates a feature v-function with the specific feature state.   CFeatureVFunction (CFeatureQFunction *qfunction, CStochasticPolicy *policy)   ~CFeatureVFunction () virtual void  setVFunctionFromQFunction (CFeatureQFunction *qfunction, CStochasticPolicy *policy) virtual void  updateWeights (CFeatureList *gradientFeatures)   Interface for updating the weights. virtual void  updateValue (CState *state, double td)   Updates the value function given a feature or discrete state. virtual void  setValue (CState *state, double qValue)   Sets the value given a feature or discrete state. virtual double  getValue (CState *state)   Returns the value given a feature or discrete state. virtual void  saveData (FILE *file)   Calls the saveFeatures function of CFeatureFunction. virtual void  loadData (FILE *file)   Calls the loadFeatures function of CFeatureFunction. virtual void  printValues ()   Prints the Paramters of the Value Function. virtual CAbstractVETraces *  getStandardETraces ()   Returns a new CFeatureVETraces object. virtual void  getGradient (CStateCollection *state, CFeatureList *gradientFeatures) virtual int  getNumWeights ()   Returns the number of weights. virtual void  resetData ()   Interface for resetting the weights. virtual void  getWeights (double *parameters)   Function for getting all weights. virtual void  setWeights (double *parameters)   Function for setting all weights. void  setFeatureCalculator (CFeatureCalculator *featCalc) Detailed Description Feature Functions can be used as linear approximators, like tile-coding and RBF-networks, the exact usage of a feature function depends on the feature state it uses. Feature states are a very common possibility to discretize continuous states. A feature consists of its index and a feature activation factor. The sum of all these factors should sum up to 1. Feature value function are modeled by the class CFeatureVFunction. For each feature it stores a feature value, so its just a table of features, the only difference to the tabular case is the calculation of the values (the sum of feature value * feature factor). CFeatureVFunctions are supposed to get a feature calculator or discretizer as state properties object. With this state property object the value function is able retrieve its feature state from the state collections. CFeatureVFunction inherits all access functions for the features from CFeatureFunction. Additionaly it implements the functions for setting and getting the values with states. These functions decompose the feature state into its discrete states and call the companion pieces of the functions for features (e.g. integer values instead of states) and multiply the values by the feature factors. To create a feature value function you have to pass a state properties object to the constructor. The number of features for the function is calculated by the discrete state size of the properties object. Of course the properties object is passed to the the super calss CAbstractVFunction and is used to retrieve the wanted state from the state collection. You also have an additional contructor at yours disposal, which can be used to calculate the value function from a stochastic policy given a Q-Function. The Values of the features are calculated as the expectation of the action propabilities multiplied with the Q-Values. The standard VETraces object of feature value functions are CVFeatureETraces, which store the features in a feature list. Constructor & Destructor Documentation CFeatureVFunction::CFeatureVFunction (  int  numFeatures  )  CFeatureVFunction::CFeatureVFunction (  CStateProperties *  featureFact  )    Creates a feature v-function with the specific feature state. The number of features for the function is calculated by the discrete state size of the properties object. Of course the properties object is passed to the the super calss CAbstractVFunction and is used to retrieve the wanted state from the state collection. The properties are supposed to be from a feature or a discrete state. CFeatureVFunction::CFeatureVFunction (  CFeatureQFunction *  qfunction, CStochasticPolicy *  policy )    Can be used to calculate the value function from a stochastic policy given a Q-Function. The Values of the features are calculated as the expectation of the action propabilities multiplied with the Q-Values. The constrcutor calls setVFunctionFromQFunction to do this. The state properties and so the number of features are taken from the Q-Function. CFeatureVFunction::~CFeatureVFunction (   )  Member Function Documentation virtual void CFeatureVFunction::getGradient (  CStateCollection *  state, CFeatureList *  gradientFeatures )  [virtual]   Implements CGradientVFunction. virtual int CFeatureVFunction::getNumWeights (   )  [virtual]   Returns the number of weights. Implements CGradientUpdateFunction. virtual CAbstractVETraces* CFeatureVFunction::getStandardETraces (   )  [virtual]   Returns a new CFeatureVETraces object. Reimplemented from CGradientVFunction. virtual double CFeatureVFunction::getValue (  CState *  state  )  [virtual]   Returns the value given a feature or discrete state. Decomposes the feature state in its discrete state variable and calculates the value by summing up the feature values of the aktiv features multiplied with their factor. Implements CAbstractVFunction. Reimplemented in CFeatureVRegressionTreeFunction. virtual void CFeatureVFunction::getWeights (  double *  parameters  )  [virtual]   Function for getting all weights. The double array is assumed to be large enough. This isn't checked! Implements CGradientUpdateFunction. virtual void CFeatureVFunction::loadData (  FILE *  file  )  [virtual]   Calls the loadFeatures function of CFeatureFunction. Reimplemented from CGradientVFunction. virtual void CFeatureVFunction::printValues (   )  [virtual]   Prints the Paramters of the Value Function. Reimplemented from CAbstractVFunction. virtual void CFeatureVFunction::resetData (   )  [virtual]   Interface for resetting the weights. Implements CGradientVFunction. virtual void CFeatureVFunction::saveData (  FILE *  file  )  [virtual]   Calls the saveFeatures function of CFeatureFunction. Reimplemented from CGradientVFunction. void CFeatureVFunction::setFeatureCalculator (  CFeatureCalculator *  featCalc  )  virtual void CFeatureVFunction::setValue (  CState *  state, double  qValue )  [virtual]   Sets the value given a feature or discrete state. Decomposes the feature state in its discrete state variable and sets the values of the features to the "qValue" value. Each value is multiplied with the coresponding feature factor. Reimplemented from CAbstractVFunction. virtual void CFeatureVFunction::setVFunctionFromQFunction (  CFeatureQFunction *  qfunction, CStochasticPolicy *  policy )  [virtual]   Can be used to calculate the value function from a stochastic policy given a Q-Function. The Values of the features are calculated as the expectation of the action propabilities multiplied with the Q-Values virtual void CFeatureVFunction::setWeights (  double *  parameters  )  [virtual]   Function for setting all weights. The double array is assumed to be large enough. This isn't checked! Implements CGradientUpdateFunction. virtual void CFeatureVFunction::updateValue (  CState *  state, double  td )  [virtual]   Updates the value function given a feature or discrete state. Decomposes the feature state in its discrete state variable and adds the "td" value to the values of the features. Each update is multiplied with the coresponding feature factor. Reimplemented from CGradientVFunction. virtual void CFeatureVFunction::updateWeights (  CFeatureList *  gradientFeatures  )  [virtual]   Interface for updating the weights. Implements CGradientUpdateFunction. The documentation for this class was generated from the following file: cvfunction.h