Acceleration Operators in the Value Iteration Algorithms for Markov Decision Processes

Simple item page
dc.creatorShlakhter, Oleksandr
dc.creatorLee, Chi-Guhn
dc.creatorKhmelev, Dmitry
dc.creatorJaber, Nasser
dc.date2005-06-23
dc.date2008-03-27
dc.date.accessioned2026-07-07T09:28:40Z
dc.date.available2026-07-07T09:28:40Z
dc.descriptionWe study the general approach to accelerating the convergence of the most widely used solution method of Markov decision processes with the total expected discounted reward. Inspired by the monotone behavior of the contraction mappings in the feasible set of the linear programming problem equivalent to the MDP, we establish a class of operators that can be used in combination with a contraction mapping operator in the standard value iteration algorithm and its variants. We then propose two such operators, which can be easily implemented as part of the value iteration algorithm and its variants. Numerical studies show that the computational savings can be significant especially when the discount factor approaches 1 and the transition probability matrix becomes dense, in which the standard value iteration algorithm and its variants suffer from slow convergence.
dc.description32 pages, 2 figures, 2 table
dc.identifierhttps://arxiv.org/abs/math/0506489
dc.identifierhttp://arxiv.org/abs/math/0506489
dc.identifier.urihttp://salesiana.dossiersoluciones.com/handle/123456789/157510
dc.subjectOptimization and Control
dc.subjectProbability
dc.subject93E20
dc.titleAcceleration Operators in the Value Iteration Algorithms for Markov Decision Processes
dc.typetext

Files

Collections

arXiv