Reinforcement Learning Toolbox 2.0 last updated:

General Documentation Manual Tutorial Class Reference Master Thesis Examples Related Papers Downloads Links News mailto:webmaster Main Page     Class Hierarchy   Compound List   File List   Compound Members   File Members CDeterministicController Class Reference Controller class makes a given controller deterministic. More... #include <cagentcontroller.h> Inheritance diagram for CDeterministicController: List of all members. Public Member Functions   CDeterministicController (CActionSet *actions)   creates a new deterministic controller,   CDeterministicController (CAgentController *controller)   creates a new deterministic controller, with the given controller to make deterministic   CDeterministicController (CAgentStatisticController *controller)   creates a new deterministic controller, with the given statistc controller to make deterministic virtual  ~CDeterministicController () virtual CAction *  getNextAction (CStateCollection *state, CActionDataSet *data=NULL)   returns the action stored in nextAction virtual void  nextStep (CStateCollection *state1, CAction *action, CStateCollection *state2)   Clears the nextAction, forcing a new calculation of the action. virtual void  newEpisode ()   Clears the nextAction, forcing a new calculation of the action. CActionStatistics *  getLastActionStatistics ()   returns the last statistics got from the statistic controller. bool  isUsingStatisticController () void  setController (CAgentController *controller)   set the controller, no statistic controller is used void  setController (CAgentStatisticController *controller)   set the controller CAgentController *  getController () void  setNextAction (CAction *action, CActionData *data=NULL) Protected Member Functions void  initStatistics () Protected Attributes CAction *  nextAction   field for storing the calculated action CAgentController *  controller   the stochastic controller CAgentStatisticController *  statisticController   the stochastic statistic controller CActionStatistics *  statistics   last statistics returned by the statistic controller, not used when using a normal controller bool  useStatisticController CActionDataSet *  actionDataSet   action Data set of all actions, the controller uses this data set to store the changeable actoin data until the nextstep. Detailed Description Controller class makes a given controller deterministic. Many other controllers are stochastic controllers, which means that they don't return the same action for the same state (even in the same step). So if getNextAction is called more than once during a step, the resulting action don't have to be the same. For some algorithm a deterministic behavior (at least for one step) is needed, so they can ask for the action which has been chosen in the current step (e.g. CSarsaLearner needs the exact policy of the agent). The CDeterministicController isn't deterministic for states (same state -> same action) but for steps (same step -> same action), it stores the action computed by the given controller each time the getNextAction is called and a newStep begins. It always returns the stored action until a new step begins again. The class CDeterministicController makes the stochastic controller returning the same action while being in the same step. Therefore it stores the calculated action in the nextAction field, returning that action when further requests for the action occure. When the step is finished (nextStep event occurs) the nextAction is cleared and with the first request newly calculated. To gather the nextStep and newEpisode events the controller HAS TO BE ADDED to the agent's listeners. The deterministic controller can handle statistic controllers and normal controllers. When using a statistic controller the statistic object is stored in the statistics field and can be obtained by getLastActionStatistics(). For actions with changeable action data, the action data is stored in actionDataSet, so the changeable action data is calculated only once too. See also: CSemiMarkovDecisionProcess Constructor & Destructor Documentation CDeterministicController::CDeterministicController (  CActionSet *  actions  )    creates a new deterministic controller, CDeterministicController::CDeterministicController (  CAgentController *  controller  )    creates a new deterministic controller, with the given controller to make deterministic CDeterministicController::CDeterministicController (  CAgentStatisticController *  controller  )    creates a new deterministic controller, with the given statistc controller to make deterministic virtual CDeterministicController::~CDeterministicController (   )  [virtual] Member Function Documentation CAgentController* CDeterministicController::getController (   )  [inline] CActionStatistics* CDeterministicController::getLastActionStatistics (   )    returns the last statistics got from the statistic controller. object is empty or uninitialized when no statistic controller is used. virtual CAction* CDeterministicController::getNextAction (  CStateCollection *  state, CActionDataSet *  data = NULL )  [virtual]   returns the action stored in nextAction if the nextAction was cleared, a new action is calculated and stored in the nextAction field Implements CAgentController. Reimplemented in CBatchLearningPolicy. void CDeterministicController::initStatistics (   )  [protected] bool CDeterministicController::isUsingStatisticController (   )  virtual void CDeterministicController::newEpisode (   )  [virtual]   Clears the nextAction, forcing a new calculation of the action. Reimplemented from CSemiMDPListener. Reimplemented in CHierarchicalSemiMarkovDecisionProcess. virtual void CDeterministicController::nextStep (  CStateCollection *  state1, CAction *  action, CStateCollection *  state2 )  [virtual]   Clears the nextAction, forcing a new calculation of the action. Reimplemented from CSemiMDPListener. void CDeterministicController::setController (  CAgentStatisticController *  controller  )    set the controller so a statistic controller is used and the statistics of the controller can be obtained by getLastStatistics void CDeterministicController::setController (  CAgentController *  controller  )    set the controller, no statistic controller is used void CDeterministicController::setNextAction (  CAction *  action, CActionData *  data = NULL )  Member Data Documentation CActionDataSet* CDeterministicController::actionDataSet [protected]   action Data set of all actions, the controller uses this data set to store the changeable actoin data until the nextstep. Reimplemented in CBatchLearningPolicy. CAgentController* CDeterministicController::controller [protected]   the stochastic controller only one of controller and statisticController is used. CAction* CDeterministicController::nextAction [protected]   field for storing the calculated action Reimplemented in CBatchLearningPolicy. CAgentStatisticController* CDeterministicController::statisticController [protected]   the stochastic statistic controller only one of controller and statisticController is used. CActionStatistics* CDeterministicController::statistics [protected]   last statistics returned by the statistic controller, not used when using a normal controller bool CDeterministicController::useStatisticController [protected] The documentation for this class was generated from the following file: cagentcontroller.h