In this work, we consider the time discretization of stochastic optimal control problems. Under general assumptions on the data, we prove the convergence of the value functions associated with the discrete time problems to the value function of the original problem. Moreover, we prove that any sequence of optimal solutions of discrete problems is minimizing for the continuous one. As a consequence of the Dynamic Programming Principle for the discrete problems, the minimizing sequence can be taken in discrete time feedback form.
Accepté le :
DOI : 10.1051/cocv/2018045
Mots-clés : Stochastic Control, Discrete Time Systems, Dynamic Programming Principle, Value Function, Feedback Control
@article{COCV_2019__25__A63_0,
author = {Bonnans, Joseph Fr\'ed\'eric and Gianatti, Justina and Silva, Francisco J.},
title = {On the time discretization of stochastic optimal control problems: {The} dynamic programming approach},
journal = {ESAIM: Control, Optimisation and Calculus of Variations},
publisher = {EDP-Sciences},
volume = {25},
year = {2019},
doi = {10.1051/cocv/2018045},
zbl = {1447.93373},
mrnumber = {4023121},
language = {en},
url = {http://www.numdam.org/articles/10.1051/cocv/2018045/}
}
TY - JOUR AU - Bonnans, Joseph Frédéric AU - Gianatti, Justina AU - Silva, Francisco J. TI - On the time discretization of stochastic optimal control problems: The dynamic programming approach JO - ESAIM: Control, Optimisation and Calculus of Variations PY - 2019 VL - 25 PB - EDP-Sciences UR - http://www.numdam.org/articles/10.1051/cocv/2018045/ DO - 10.1051/cocv/2018045 LA - en ID - COCV_2019__25__A63_0 ER -
%0 Journal Article %A Bonnans, Joseph Frédéric %A Gianatti, Justina %A Silva, Francisco J. %T On the time discretization of stochastic optimal control problems: The dynamic programming approach %J ESAIM: Control, Optimisation and Calculus of Variations %D 2019 %V 25 %I EDP-Sciences %U http://www.numdam.org/articles/10.1051/cocv/2018045/ %R 10.1051/cocv/2018045 %G en %F COCV_2019__25__A63_0
Bonnans, Joseph Frédéric; Gianatti, Justina; Silva, Francisco J. On the time discretization of stochastic optimal control problems: The dynamic programming approach. ESAIM: Control, Optimisation and Calculus of Variations, Tome 25 (2019), article no. 63. doi : 10.1051/cocv/2018045. http://www.numdam.org/articles/10.1051/cocv/2018045/
[1] and , Infinite imensional analysis, in A Hitchhiker’s Guide. 3rd edn. Springer, Berlin (2006). | MR | Zbl
[2] and , Convergence of approximation schemes for fully nonlinear second order equations. Asymptotic Anal. 4 (1991) 271–283. | DOI | MR | Zbl
[3] and , Stochastic Optimal Control: The Discrete Time Case. Academic Press, New York (1978). | MR | Zbl
[4] and , Perturbation Analysis of Optimization Problems. Springer Series in Operations Research. Springer-Verlag, New York (2000). | MR | Zbl
[5] and , Weak dynamic programming principle for viscosity solutions. SIAM J. Control Optim. 49 (2011) 948–962. | DOI | MR | Zbl
[6] , and , Integral inequalities for convex functions of operators on martingales, in Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability, Vol. 2 of Probability Theory. University of California Press, Berkeley, CA (1972) 223–240. | MR | Zbl
[7] Dolcetta, On a discrete approximation of the Hamilton-Jacobi equation of dynamic programming. Appl. Math. Optim. 10 (1983) 367–377. | DOI | MR | Zbl
[8] and , Approximate solutions of the Bellman equation of deterministic control theory. Appl. Math. Optim. 11 (1984) 161–181. | DOI | MR | Zbl
[9] , Discrete approximation of continuous time stochastic control systems. SIAM J. Control Optim. 21 (1983) 17–40. | DOI | MR | Zbl
[10] , Short proof of a discrete Gronwall inequality. Discrete Appl. Math. 16 (1987) 279–281. | DOI | MR | Zbl
[11] and , Semi-lagrangian schemes for linear and fully non-linear diffusion equations. Math. Comput. 82 (2013) 1433–1462. | DOI | MR | Zbl
[12] and , Controlled Markov processes. Translated from the Russian original by and . Vol. 235 of Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences]. Springer-Verlag, Berlin-New York, (1979). | DOI | MR | Zbl
[13] and , Deterministic and Stochastic Optimal Control. Applications of Mathematics, Springer-Verlag, Berlin-New York, (1975). | MR | Zbl
[14] and , Controlled Markov processes and viscosity solutions. Vol. 25 of Stochastic Modelling and Applied Probability. 2nd edn. Springer, New York (2006). | MR | Zbl
[15] and , Controlled Stochastic Processes. Translated from the Russian by . Springer-Verlag, New York-Heidelberg (1979). | DOI | MR | Zbl
[16] and , Stochastic Differential Equations and Diffusion Processes. North-Holland Publishing Co., Kodansha, Ltd., Amsterdam, New York, Tokyo (1981). | MR | Zbl
[17] , Approximating value functions for controlled degenerate diffusion processes by using piece-wise constant policies. Electron. J. Probab. 4 (1999) 1–19. | DOI | MR | Zbl
[18] , Mean value theorems for stochastic integrals. Ann. Probab. 29 (2001) 385–410. | DOI | MR | Zbl
[19] , Controlled Diffusion Processes, Vol. 14. Springer Science & Business Media, New York, Berlin (2008). | MR | Zbl
[20] , Probability Methods for Approximations in Stochastic Control and for Elliptic Equations. Vol. 129 of Mathematics in Science and Engineering. Academic Press, New York (1977). | MR | Zbl
[21] , Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations. I. The dynamic programming principle and applications. Comm. Part. Diff. Eq. 8 (1983) 1101–1174. | DOI | MR | Zbl
[22] , Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations. II. Viscosity solutions and uniqueness. Comm. Part. Diff. Eq. 8 (1983) 1229–1276. | DOI | MR | Zbl
[23] , Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations. III. Regularity of the optimal cost function. In Nonlinear partial differential equations and their applications. Collège de France seminar, Vol. V (Paris, 1981/1982). Vol. 93 of Research Notes in Mathematics. Pitman, Boston, MA (1983) 95–205. | MR | Zbl
[24] and , A variational formula for stochastic controls and some applications. Special Issue: In honor of Leon Simon. Part 1. Pure Appl. Math. Q. 3 (2007) 539–567. | DOI | MR | Zbl
[25] , Stochastic Control Theory. Dynamic Programming Principle. 2nd edn. Springer, Tokyo (2015). | DOI | MR | Zbl
[26] , , and , The Mathematical Theory of Optimal Processes. Reprint of the 1962 English translation. Gordon & Breach Science Publishers, New York (1986).
[27] , Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics. John Wiley & Sons, Inc., New York (1994). | MR | Zbl
[28] , A Course on Borel Sets. Springer-Verlag, New York (1998). | DOI | MR | Zbl
[29] , Optimal stochastic control, stochastic target problems, and backward SDE. Vol. 29 of Fields Institute Monographs. Fields Institute for Research in Mathematical Sciences, Toronto, ON. With Chapter 13 by A. Tourin. Springer, New York (2013). | MR | Zbl
[30] and , Stochastic Controls, Hamiltonian Systems and HJB Equations. Springer-Verlag, New York, Berlin (2000). | MR | Zbl
[31] and , Controlled random sequences and Markov chains. Russ. Math. Surv. 37 (1982) 239 | DOI | MR | Zbl
Cité par Sources :
