Section: New Results

Computable approximations for average Markov decision processes in continuous time

In this paper we study the numerical approximation of the optimal long-run average cost of a continuous-time Markov decision process, with Borel state and action spaces, and with bounded transition and reward rates. Our approach uses a suitable discretization of the state and action spaces to approximate the original control model. The approximation error for the optimal average reward is then bounded by a linear combination of coefficients related to the discretization of the state and action spaces, namely, the Wasserstein distance between an underlying probability measure $\mu$ and a measure with finite support, and the Hausdorff distance between the original and the discretized actions sets. When approximating $\mu$ with its empirical probability measure we obtain convergence in probability at an exponential rate. An application to a queueing system is presented.

Authors: Jonatha Anselmi (Inria CQFD), François Dufour (Inria CQFD) and Tomás Prieto-Rumeau.