State Abstraction in MAXQ Hierarchical Reinforcement Learning

Thumbnail Image

Date

Authors

Journal Title

Journal ISSN

Volume Title

Publisher

Abstract

Description

Many researchers have explored methods for hierarchical reinforcement learning (RL) with temporal abstractions, in which abstract actions are defined that can perform many primitive actions before terminating. However, little is known about learning with state abstractions, in which aspects of the state space are ignored. In previous work, we developed the MAXQ method for hierarchical RL. In this paper, we define five conditions under which state abstraction can be combined with the MAXQ value function decomposition. We prove that the MAXQ-Q learning algorithm converges under these conditions and show experimentally that state abstraction is important for the successful application of MAXQ-Q learning.
7 pages, 2 figures

Keywords

Machine Learning, I.2.6

Citation

URI

http://salesiana.dossiersoluciones.com/handle/123456789/33205

Consulte el texto completo en el siguiente enlace:

https://arxiv.org/abs/cs/9905015
 http://arxiv.org/abs/cs/9905015

Collections

arXiv