The Promise of Hierarchical Reinforcement Learning

This idea of temporal abstraction, once incorporated into reinforcement learning (RL), converts it into hierarchical reinforcement learning (HRL). Inspired by Medieval Europe’s Feudal system, this HRL method demonstrates how to create a managerial learning hierarchy in which lords (or managers) learn to assign tasks (or sub-goals) to their serfs (or sub-managers) who, in turn, learn to satisfy them. Unlike Feudal learning, if the action space consists of both primitive actions and options, then an algorithm following the Options framework is proven to converge to an optimal policy.

Source: thegradient.pub