Reinforcement Learning Toolbox 2.0 last updated:

General Documentation Manual Tutorial Class Reference Master Thesis Examples Related Papers Downloads Links News mailto:webmaster Main Page     Class Hierarchy   Compound List   File List   Compound Members   File Members CHierarchicalSemiMarkovDecisionProcess Class Reference Subclass of CSemiMarkovDecisionProcess, used for hierarchical Learning. More... #include <cagent.h> Inheritance diagram for CHierarchicalSemiMarkovDecisionProcess: List of all members. Public Member Functions   CHierarchicalSemiMarkovDecisionProcess (CEpisode *currentEpisode)   CHierarchicalSemiMarkovDecisionProcess (CStateProperties *modelProperties, std::list< CStateModifier * > *modifiers=NULL)   ~CHierarchicalSemiMarkovDecisionProcess () virtual void  setLoggedEpisode (CEpisode *loggedEpisode) virtual void  nextStep (CStateCollection *oldState, CHierarchicalStack *actionStack, CStateCollection *newState) virtual void  newEpisode () virtual void  sendNextStep (CAction *action)   Sends the nextStep to the listeners. virtual bool  isFinished (CStateCollection *, CStateCollection *) virtual CAction *  getNextHierarchyLevel (CStateCollection *stateCollection, CActionDataSet *actionDataSet=NULL)   Virtual function for determining the next action in the next hierarchie level. virtual void  addStateModifier (CStateModifier *modifier)   Add a state Modifier to the StateCollections. virtual void  removeStateModifier (CStateModifier *modifier)   remove a state Modifier from the StateCollections Protected Member Functions virtual CAction *  getExecutedAction (CHierarchicalStack *actionStack)   Returns the action done by the SMDP. Protected Attributes CEpisode *  currentEpisode CStateCollectionImpl *  pastState CStateCollectionImpl *  currentState Detailed Description Subclass of CSemiMarkovDecisionProcess, used for hierarchical Learning. This abstract class provides full Hierarchical learning functionality. It implements the CHierarchicalStackListener interface, so the HierarchicalController can inform the SMDP about a hierarchical Step. Than the SMDP sends the State-Action-State Tuple with the action done by the specific hierarchical SMDP. In order to provide hierarchical Functionality the class also represents an ExtendedAction, so it can be used as Action for another hierarchical SMDP. It can't be used as action for the agent, you have to use CHierarchicalController to create the HierarchicalStructure. Use the hierarchical controller as controller for the agent, the agent itself does'nt know anything about the hierarchical structure of the learning Problem. The class is abstract because the isFinished Method from CMultiStepAction remains to be implemented. See also: CHierarchicalController Constructor & Destructor Documentation CHierarchicalSemiMarkovDecisionProcess::CHierarchicalSemiMarkovDecisionProcess (  CEpisode *  currentEpisode  )    Creates a new hierarchical SMDP. The episode is needed for reconstruction of the intermediate and hierarchical steps. Parameters: currentEpisode  Pointer to the current Episode. It is recommended to use the currentEpisode Object of the Agent. CHierarchicalSemiMarkovDecisionProcess::CHierarchicalSemiMarkovDecisionProcess (  CStateProperties *  modelProperties, std::list< CStateModifier * > *  modifiers = NULL )  CHierarchicalSemiMarkovDecisionProcess::~CHierarchicalSemiMarkovDecisionProcess (   )  Member Function Documentation virtual void CHierarchicalSemiMarkovDecisionProcess::addStateModifier (  CStateModifier *  modifier  )  [virtual]   Add a state Modifier to the StateCollections. Reimplemented from CStateModifiersObject. virtual CAction* CHierarchicalSemiMarkovDecisionProcess::getExecutedAction (  CHierarchicalStack *  actionStack  )  [protected, virtual]   Returns the action done by the SMDP. virtual CAction* CHierarchicalSemiMarkovDecisionProcess::getNextHierarchyLevel (  CStateCollection *  stateCollection, CActionDataSet *  actionDataSet = NULL )  [virtual]   Virtual function for determining the next action in the next hierarchie level. Implements CExtendedAction. virtual bool CHierarchicalSemiMarkovDecisionProcess::isFinished (  CStateCollection *  , CStateCollection *  )  [inline, virtual]   This method is normally used by a hierarchical controller to determine wether the action has finished or not. The finsished method may depend only on the current state transition, so you get the old state and the new state as parameters. The controller sets the finished flag according to isFinished, so other Listeners only have to look at this flag. This method must be implemented by all (non-abstract) sub-classes. Implements CMultiStepAction. Reimplemented in CSubGoalBehaviour, and CTaxiHierarchicalBehaviour. virtual void CHierarchicalSemiMarkovDecisionProcess::newEpisode (   )  [virtual]   Reimplemented from CHierarchicalStackListener. virtual void CHierarchicalSemiMarkovDecisionProcess::nextStep (  CStateCollection *  oldState, CHierarchicalStack *  actionStack, CStateCollection *  newState )  [virtual]   Implements CHierarchicalStackListener. virtual void CHierarchicalSemiMarkovDecisionProcess::removeStateModifier (  CStateModifier *  modifier  )  [virtual]   remove a state Modifier from the StateCollections Reimplemented from CStateModifiersObject. virtual void CHierarchicalSemiMarkovDecisionProcess::sendNextStep (  CAction *  action  )  [virtual]   Sends the nextStep to the listeners. If the action is an extended action, all intermediated steps and the double step itself get recovered from the Episode object, and send to the listeners (intermediate Steps gets send with the "intermediateStep" method). If the action is not an extended action, the nextSend Method from the super class gets called. Reimplemented in CSubGoalBehaviour. virtual void CHierarchicalSemiMarkovDecisionProcess::setLoggedEpisode (  CEpisode *  loggedEpisode  )  [virtual] Member Data Documentation CEpisode* CHierarchicalSemiMarkovDecisionProcess::currentEpisode [protected]   Pointer to the currentEpisode, the currenEpisode must be updated before the sendNextStep method is called. So currentEpisode has to be the first Element of the agent's Listener-List. Needed for determining the intermediate Steps. CStateCollectionImpl* CHierarchicalSemiMarkovDecisionProcess::currentState [protected] CStateCollectionImpl* CHierarchicalSemiMarkovDecisionProcess::pastState [protected] The documentation for this class was generated from the following file: cagent.h