Zero-sum risk-sensitive stochastic differential games with reflecting diffusions in the orthant

Ghosh, Mrinal Kanti; Pradhan, Somnath

ESAIM: Control, Optimisation and Calculus of Variations, Tome 26 (2020), article no. 114.

Suite au passage du modèle économique de la revue en S20, le texte intégral des articles des années 2019 et 2020 est accessible uniquement sur le site de l'éditeur et est réservé aux abonnés.

We study zero-sum games with risk-sensitive cost criterion on the infinite horizon where the state is a controlled reflecting diffusion in the nonnegative orthant. We consider two cost evaluation criteria: discounted cost and ergodic cost. Under certain assumptions, we establish the existence of saddle point equilibria. We obtain our results by studying the corresponding Hamilton–Jacobi–Isaacs equations. For the ergodic cost criterion, exploiting the stochastic representation of the principal eigenfunction, we have completely characterized saddle point equilibrium in the space of stationary Markov strategies.

[1] S. Agmon, A. Douglis and L. Nirenberg, Estimates near the boundary for solutions of elliptic partial differential equations satisfying general boundary conditions I. Comm. Pure Appl. Math. 12 (1959) 623–727. | DOI | MR | Zbl

[2] E. Altman, Flow control using the theory of zero-sum Markov games. IEEE Trans. Automat. Control 39 (1994) 814–818. | DOI | MR | Zbl

[3] H. Amann, Dual Semigroups and second order linear elliptic boundary value problem. Israel J. Math. 45 (1983) 225–254. | DOI | MR | Zbl

[4] H. Amann and J. Lopez-Gomez, A priori bounds and multiple solutions for superlinear indefinite elliptic problems. J. Differ. Equ. 146 (1998) 336–374. | DOI | MR | Zbl

[5] A. Arapostathis and A. Biswas, Infinite horizon risk-sensitive control of diffusions without any blanket stability assumptions. Stoch. Process. Appl. 128 (2018) 1485–1524. | DOI | MR | Zbl

[6] A. Arapostathis, V.S. Borkar and K.S. Kumar, Risk-sensitive control and an abstract Collatz-Wielandt formula. J. Theor. Probab. 29 (2016) 1458–1484. | DOI | MR | Zbl

[7] A. Arapostathis, V.S. Borkar and M.K. Ghosh, Ergodic Control of Diffusion Processes. Cambridge University Press, Cambridge (2012). | MR | Zbl

[8] A. Arapostathis, A. Biswas and S. Saha, Strict monotonicity of principal eigenvalues of elliptic operators in ${ℝ}^d$ and risk-sensitive control. J. Math. Pures Appl. 124 (2019) 169–219. | DOI | MR | Zbl

[9] A. Arapostathis, A. Biswas, V.S. Borkar and K.S. Kumar, A variational characterization of the risk-sensitive average reward for controlled diffusions in ${ℝ}^d$. Preprint (2019). | arXiv | MR

[10] A. Bagchi and K. Suresh Kumar Dynamic asset management with risk-sensitive criterion and non-negative factor constraints: a differential game approach. Stochastics 81 (2009) 503–530. | DOI | MR | Zbl

[11] T. Başar, Nash equilibrium of risk-sensitive nonlinear stochastic differential games, J. Optim. Theory Appl. 100 (1999) 479–498. | DOI | MR | Zbl

[12] A. Basu and M.K. Ghosh, Zero-Sum Risk-sensitive stochastic differential games. Math. Oper. Res. 37 (2012) 437–449. | DOI | MR | Zbl

[13] V.E. Beneš, Existence of optimal strategies based on specified information of a class of stochastic decision problems. SIAM J. Control 8 (1970) 179–188. | DOI | MR | Zbl

[14] A. Bensoussan and J.L. Lions, Impulse control and Quasi-variational Inequalities. Gauthier-Villars, England (1984). | MR

[15] L.D. Berkovitz, The existence of value and saddle point in games of fixed duration. SIAM J. Control Optim. 23 (1985) 172–196. | DOI | MR | Zbl

[16] A. Biswas, An eigenvalue approach to the risk-sensitive control problem in near monotone case. Systems Control Lett. 60 (2011) 181–184. | DOI | MR | Zbl

[17] A. Biswas and S. Saha, Zero-sum stochastic differential games with risk-sensitive cost. App. Math. Optim. 81 (2020) 113–140. | DOI | MR | Zbl

[18] V. Borkar and A. Budhiraja, Ergodic control for constained diffusions: characterization using HJB equation. SIAM J. Control Optim. 43 (2005) 1467–1492. | DOI | MR | Zbl

[19] V.S. Borkar and M.K. Ghosh, Stochastic differential games: occupation measure based approach. J. Optim. Theory Appl. 73 (1992) 359–385. Errata corriege, ibid 88 (1996) 251–252. | DOI | MR | Zbl

[20] A. Budhiraja, An ergodic control problem for constrained diffusion processes: existence of optimal control. SIAM J. Control Optim. 42 (2003) 532–558. | DOI | MR | Zbl

[21] R. Buckdahn, J. Li and M. Quincampoix, Value in mixed strategies for zero-sum stochastic differential games without Isaacs condition. Ann. Probab. 42 (2014) 1724–1768. | DOI | MR | Zbl

[22] P. Cardaliaguet and C. Rainer, Stochastic differential games with asymmetric information. Appl. Math. Optim. 59 (2009) 1–36. | DOI | MR | Zbl

[23] P. Cardaliaguet and C. Rainer, Pathwise strategies for stochastic differential games with an erratum to “Stochastic differential games with asymmetric information”. Appl. Math. Optim. 58 (2013) 75–84. | DOI | MR | Zbl

[24] K.L. Chung and Z. Zhao, From Brownian motion to Schrodinger’s equation, A series of Comprehensive studies in Mathematics Vol. 312. Springer, Berlin (1995). | MR | Zbl

[25] G. Di Fazio and D.K. Palagachev, Oblique derivative problem for elliptic equations in non-divergence form with VMO coefficients. Comment. Math. Univ. Carolin. 37 (1996) 537–556. | MR | Zbl

[26] N. El-Karoui and S. Hamadene, BSDE and risk-sensitive control, zero-sum and nonzero-sum game problems of stochastic functional differential equations. Stochastic Proc. Appl. 107 (2003) 145–169. | DOI | MR | Zbl

[27] R.J. Elliott, The existence of value of a stochastic differential games. SIAM J. Control 14 (1976) 85–94. | DOI | MR | Zbl

[28] R.J. Elliott, Viscosity Solutions and Optimal Control, Pitman Research Notes in Mathematics series, No. 165. Longman, London (1987). | MR | Zbl

[29] R.J. Elliot and M.H.A. Davis, Optimal play in a stochastic differential game. SIAM J. Control Optim 19 (1981) 543–554. | DOI | MR | Zbl

[30] R.J. Elliott and N. Kalton, The existence of value in a differential game. Am. Math. Soc. Memoir 12 (1972) 504–523.

[31] K. Fan, Fixed-point and minimax theorems in locally convex topological linear spaces. Proc. Nat. Acad. Sc. 38 (1952) 121–126. | DOI | MR | Zbl

[32] W.H. Fleming, The convergence problem for differential games. J. Math. Anal. Appl. 3 (1961) 102–116. | DOI | MR | Zbl

[33] W.H. Fleming, The convergence problem for differential games II, in ’Advances in Game Theory’. Ann. Math. Study 52 (1964) 195–210. | MR | Zbl

[34] W.H. Fleming and H.M. Soner, Controlled Markov Processes and viscosity Solutions. Springer-Verlag, Berlin (1991). | MR | Zbl

[35] W.H. Fleming and P.E. Souganidis, On the existence of value functions of two-player, zero-sum stochastic differential games. Indiana Univ. Math. J. 38 (1989) 293–314. | DOI | MR | Zbl

[36] A. Friedman, Differential Games. John Wiley & Sons, Chichester (1971). | MR | Zbl

[37] A. Friedman, Stochastic Differential Equations and Applications, Vol.2. Academic Press, Cambridge (1976). | MR | Zbl

[38] S.K. Gauttam, K. Suresh Kumar and C. Pal, Risk-Sensitive Control Of Reflected Processes On Orthrant. Pure Appl. Funct. Anal. 2 (2017) 477–510. | MR | Zbl

[39] M.K. Ghosh and S.K. Kumar, Zero-sum stochastic differential games with reflecting diffusions. Comput. Appl. Math. 16 (1997) 237–246. | MR | Zbl

[40] M.K. Ghosh and S. Pradhan, Risk-Sensitive Stochastic Differential Game With Reflecting Diffusions. Stoch. Anal. Appl. 36 (2018) 1–27. | DOI | MR | Zbl

[41] M.K. Ghosh and K. Suresh Kumar A stochastic differential game in the orthant. J. Math. Anal. Appl. 265 (2002) 12–37. | DOI | MR | Zbl

[42] D. Gilbarg and N.S. Trudinger, Elliptic Partial Differential Equations of Second Order, 2nd edn. Vol, 224. Springer, Berlin (1983). | MR | Zbl

[43] S. Hamadene and J.-P. Lepeltier, Backward equations, stochastic control and zero-sum stochastic differential games. Stochastics 54 (1995) 221–231. | MR | Zbl

[44] S. Hamadene and J.-P. Lepeltier, Zero-sum stochastic differential games and backward equations. Syst. Control Lett. 24 (1995) 259–263. | DOI | MR | Zbl

[45] S. Hamadene, J.-P. Lepeltier and S. Peng, BSDE with continuous coefficients and zero-sum stochastic differential games, Systems and Control Letters, Backward stochastic differential equations, edited by N. El Karoui and L. Mazlik. Research Notes in Mathematics Vol. 363. Chapman and Hall/CRC, Longman (1997) 115–128. | MR | Zbl

[46] J. Húska, Harnack inequality and exponetential separation for oblique derivative problems on Lipschitz domains. J. Differ. Equ. 226 (2006) 541–557. | DOI | MR | Zbl

[47] D.L. Iglehart and W. Whitt, Multiple channel queues in heavy traffic. Adv. Appl. Prob. 2 (1970) 150–177. | DOI | MR | Zbl

[48] R. Isaacs, Differential Games I - IV, The RAND Corporation Research Memoranda, RM- 1391 1399, 1411, 1486, 1954-55.

[49] R. Isaacs, Differential Games. John Wiley & Sons, Chichester (1965). | Zbl

[50] M.N. Krasovskii and A.I. Subbotin, Game-Theoretical Control Problems. Springer-Verlag, Berlin (1998). | MR | Zbl

[51] H.J. Kushner and L.F. Martins, Limit theorems for pathwise average cost per unit time problems for controlled queues in heavy traffic. Stoch. Stoch. Rep. 42 (1993) 25–51. | DOI | MR | Zbl

[52] H.J. Kushner and K.M. Ramachandran, Optimal and approximately optimal control policies for queues in heavy traffic. Lefschetz Center for Dynamical Systems Report #87-24. SIAM J. Optim. Control 27 (1989) 1293–1318. | DOI | MR | Zbl

[53] A.J. Lemoine, Network of queues – A survey of weak convergence results. Manag. Sci. 24 (1978) 1175–1193. | DOI | MR | Zbl

[54] G. Leoni, A First Course in Sobolev Spaces, Graduate Studies in Mathematics, Vol. 105. AMS, Rhode Island (2009). | MR | Zbl

[55] G.M. Lieberman, Local estimates for subsolutions and supersolutions of oblique derivative problems for general second order elliptic equations. Trans. Am. Math. Soc. 304 (1987) 343–353. | DOI | MR | Zbl

[56] G.M. Lieberman, Oblique derivative problems for elliptic equations. World Scientific, New Jersey (2013). | DOI | MR | Zbl

[57] P.L. Lions and A.S. Sznitman, Stochastic Differential equations with reflecting boundary conditions. Comm. Pure. Appl. Math. 37 (1984) 511–537. | DOI | MR | Zbl

[58] J.L. Menaldi and M. Robin, An Ergodic Control Problem for Reflected Diffusion with Jump. IMA J. Math. Control Inf. 1 (1984) 309. | DOI | Zbl

[59] M. Nisio, Stochastic differential games and viscosity solutions of Isaacs equations. Nagoya Math. J. 110 (1988) 163–184. | DOI | MR | Zbl

[60] S. Patrizi, Principal Eigenvalues for Isaacs Operators with Neumann Boundary Conditions. Nonlinear Differ. Equ. Appl. 16 (2009) 79–107. | DOI | MR | Zbl

[61] L.S. Pontryagin, On the theory of differential games. Russ. Math Surv. 21 (1966) 193–246. | DOI | MR | Zbl

[62] L.S. Pontryagin, V.G. Boltyanskii, R.V. Gamkrelidze and E.F. Mishchcnko, The mathematical theory of optimal processes. Wiley Interscience, New Jersey (1962). | MR | Zbl

[63] D. Possami, N. Touzi and J. Zhang, Zero-sum path-dependent stochastic differential games in weak formulation. Ann. Appl. Prob. 30 (2020) 1415–1457. | MR | Zbl

[64] S. Pradhan, Risk-sensitive ergodic control of reflected diffusion processes in orthant. To appear in: Appl. Math. Optim. (2019). | DOI | MR | Zbl

[65] M.H. Protter and H.F. Weinberger, Maximum Principles in Differential Equations. Springer, Berlin (1984). | MR | Zbl

[66] M.I. Reiman, Open queueing networks in heavy traffic. Math of Oper. Res. 9 (1984) 441–458. | DOI | MR | Zbl

[67] E. Roxin, Axiomatic approach in differential games. J. Optim. Theory Appl. 3 (1969) 153–163. | DOI | MR | Zbl

[68] M. Sirbu, On martingale problems with continuous mixing and values of zero-sum games without the Isaacs condition. SIAM J. Control Optim. 52 (2014) 2877–2890. | DOI | MR | Zbl

[69] N.N. Subbotina, A.I. Subbotin and V.E. Tretjakov, Stochastic and deterministic control; differential inequalities. Probl. Control Inform. Theory 14 (1985) 405–419. | MR | Zbl

[70] P. Varaiya, On the existence of solutions to a differential games. SIAM J. Control 5 (1967) 153–162. | DOI | Zbl

[71] P. Variaya and J. Lin, Existence of saddle points in differential games. SIAM J. Control 7 (1969) 142–157. | Zbl

[72] Z. Wu, J.Yin and C. Wang, Elliptic and Parabolic equations. World Scientific, Singapore (2006). | DOI | Zbl

[73] A. Yu. Veretennikov, On strong and weak solutions of one-dimensional stochastic equations with boundary conditions. Theory Probab. Appl. 26 (1981) 670–686. | DOI | Zbl