Reinforcement Learning Toolbox 2.0 last updated:

General Documentation Manual Tutorial Class Reference Master Thesis Examples Related Papers Downloads Links News mailto:webmaster Main Page     Class Hierarchy   Compound List   File List   Compound Members   File Members CHierarchicalController Class Reference Class for calculating a hierarchical execution of a hierarchical structure. More... #include <chierarchiccontroller.h> Inheritance diagram for CHierarchicalController: List of all members. Public Member Functions   CHierarchicalController (CActionSet *agentActions, CActionSet *allActions, CExtendedAction *rootAction)   creates the hierarchical controller with the actionset he can choose from and the root of the hierarchical structure   ~CHierarchicalController () int  getMaxHierarchicalExecution ()   get maximum duration of hierarchical execution void  setMaxHierarchicalExecution (int maxExec)   set maximum duration of hierarchical execution virtual CAction *  getNextAction (CStateCollection *state, CActionDataSet *actionDataSet)   returns the primitiv action from the action stack virtual void  nextStep (CStateCollection *oldState, CAction *action, CStateCollection *newState)   sends the states and action stacks to the listeners. virtual void  intermediateStep (CStateCollection *oldState, CAction *action, CStateCollection *newState)   only calls nextStep virtual void  newEpisode ()   tells the Listener that a new Episode has startet. Protected Member Functions virtual CAction *  getAgentAction (CHierarchicalStack *stack, CActionDataSet *actionDataSet)   returns the action for the agent from the stack (the last action on stack) Protected Attributes CActionSet *  agentActions   The agent's actions. CHierarchicalStack *  actionStack   The actual hierarchical stack. CExtendedAction *  rootAction   The root of the hierarchical structure. CActionDataSet *  hierarchichActionDataSet Detailed Description Class for calculating a hierarchical execution of a hierarchical structure. The hierarchical controller calculates a hierarchical policy, given the root element from an hierarchical learning structure. The hierarchical learning structure is build by the Hierarchical Semi Markov Decision Processes. The hierarchical stores an hierarchical action stack and executes each hierarchical action as long as it (or an action with higher hierarchy in the stack) is finished. The hierarchical controller also manages the calculation of the duration and the finished flag of all extended actions. The controller also has a listener list of stack listeners (from superclass CHierarchicalStackSender). Each component which needs acces to the hierarchical stack needs to be a hierarchical stack listener (CHierarchicStackListener) and must be added to the listener list of the controller. The controller, if added to the listener list of the again then sends each step the states (old and newstate) and the hierarchical stack to his listeners. So all hierarchical semi MDP's must be added as to the controller's listeners, even seme MDP's which are not directly the root of the hierarchy. You can also set the duration of the hierarchical execution, so if an extended action has a already took longer than the maximum duration the extended action is finished by the controller. With this feature you can make a transition from hierarchical to flat execution during learning. The hierarchical controller always returns a primitiv action (the last action on the stack) from his getNextAction method for the agent to execute. All primitiv actions returned from the hierarchical structure must be member of the controllers action set! See also: CHierarchicalStackListener CHierarchicalSemiMarkovDecisionProcess. Constructor & Destructor Documentation CHierarchicalController::CHierarchicalController (  CActionSet *  agentActions, CActionSet *  allActions, CExtendedAction *  rootAction )    creates the hierarchical controller with the actionset he can choose from and the root of the hierarchical structure All primitiv actions returned from the hierarchical structure must be member of the action set! CHierarchicalController::~CHierarchicalController (   )  Member Function Documentation virtual CAction* CHierarchicalController::getAgentAction (  CHierarchicalStack *  stack, CActionDataSet *  actionDataSet )  [protected, virtual]   returns the action for the agent from the stack (the last action on stack) int CHierarchicalController::getMaxHierarchicalExecution (   )    get maximum duration of hierarchical execution virtual CAction* CHierarchicalController::getNextAction (  CStateCollection *  state, CActionDataSet *  actionDataSet )  [virtual]   returns the primitiv action from the action stack Builds and renews missing parts of the action stack Implements CAgentController. virtual void CHierarchicalController::intermediateStep (  CStateCollection *  oldState, CAction *  action, CStateCollection *  newState )  [virtual]   only calls nextStep Reimplemented from CSemiMDPListener. virtual void CHierarchicalController::newEpisode (   )  [virtual]   tells the Listener that a new Episode has startet. Reimplemented from CSemiMDPListener. virtual void CHierarchicalController::nextStep (  CStateCollection *  oldState, CAction *  action, CStateCollection *  newState )  [virtual]   sends the states and action stacks to the listeners. Calculates the duration of the actions (adds the duratoin of the executed primitiv action to each extended action), and the finished flags. If one action is finished, all other actions with lower hierarchy in the action stack get marked as finished too. After that calculation it sends the action stack to all listeners and then it deletes the finished actions from the stack. The missing stack elements are renewed by the method getAction. Reimplemented from CSemiMDPListener. void CHierarchicalController::setMaxHierarchicalExecution (  int  maxExec  )    set maximum duration of hierarchical execution Member Data Documentation CHierarchicalStack* CHierarchicalController::actionStack [protected]   The actual hierarchical stack. CActionSet* CHierarchicalController::agentActions [protected]   The agent's actions. CActionDataSet* CHierarchicalController::hierarchichActionDataSet [protected] CExtendedAction* CHierarchicalController::rootAction [protected]   The root of the hierarchical structure. The documentation for this class was generated from the following file: chierarchiccontroller.h