Some simple but challenging Markov processes

Malrieu, Florent

Annales de la Faculté des sciences de Toulouse : Mathématiques, Série 6, Numéro Spécial : Conférence “Talking Across Fields” du 24 au 28 mars 2014 à l’Institut de Mathématiques de Toulouse, Tome 24 (2015) no. 4, pp. 857-883.

Ces notes rassemblent l’étude du comportement en temps long de plusieurs processus de Markov déterministes par morceaux. Ces processus ont le double intérêt d’être motivés par la modélisation (biologie, réseaux de communication, chimie,...) et d’impliquer de nombreux outils mathématiques : couplage, approches spectrales, équations aux dérivées partielles non locales ou encore inégalités fonctionnelles. Ces exemples permettent enfin de formuler des questions ouvertes.

In this note, we present few examples of Piecewise Deterministic Markov Processes and their long time behavior. They share two important features: they are related to concrete models (in biology, networks, chemistry,...) and they are mathematically rich. Their mathematical study relies on coupling method, spectral decomposition, PDE technics, functional inequalities. We also relate these simple examples to recent and open problems.

[1] Aldous (D.) and Diaconis (P.).— Strong uniform times and finite random walks, Adv. in Appl. Math. 8, no. 1, p. 69-97 (1987). | MR | Zbl

[2] Bakhtin (Y.) and Hurth (T.).— Invariant densities for dynamical systems with random switching, Nonlinearity 25 no. 10, p. 2937-2952 (2012). | MR | Zbl

[3] Bakhtin (Y.), Hurth (T.), and Mattingly (J. C.).— Regularity of invariant densities for 1D-systems with random switching, arXiv:1406.5425, (2014).

[4] Balde (M.) and Boscain (U.).— , Stability of planar switched systems: the nondiagonalizable case, Commun. Pure Appl. Anal. 7, no. 1, p. 1-21 (2008). | MR | Zbl

[5] Balde (M.), Boscain (U.), and Mason (P.).— A note on stability conditions for planar switched systems, Internat. J. Control 82, no. 10, p. 1882-1888 (2009). | MR | Zbl

[6] Bardet (J.-B.), Christen (A.), Guillin (A.), Malrieu (A.), and Zitt (P.-A.).— Total variation estimates for the TCP process, Electron. J. Probab. 18, no. 10, p. 1-21 (2013). | MR | Zbl

[7] Benaïm (M.), Le Borgne (S.), Malrieu (F.), and Zitt (P.-A.).— Qualitative properties of certain piecewise deterministic Markov processes, Ann. Inst. Henri Poincar? Probab. Stat. 51, no. 3, p. 1040-1075 (2015). | Numdam | MR

[8] Benaïm (M.), Le Borgne (S.), Malrieu (F.), and Zitt (P.-A.).— Quantitative ergodicity for some switched dynamical systems, Electron. Commun. Probab. 17, no. 56, p. 14 (2012). | MR

[9] Benaïm (M.), Le Borgne (S.), Malrieu (F.), and Zitt (P.-A.).— On the stability of planar randomly switched systems, Ann. Appl. Probab. 24, no. 1, p. 292-311 (2014). | MR | Zbl

[10] Berestycki (J.), Bertoin (J.), Haas (B.), and Miermont (G.).— Quelques aspects fractals des fragmentations aléatoires, Quelques interactions entre analyse, probabilités et fractals, Panor. Synthèses, vol. 32, Soc. Math. France, Paris, p. 191-243 (2010). | Zbl

[11] Bertail (P.), S. Clémençon (S.), and Tressou (J.).— A storage model with random release rate for modeling exposure to food contaminants, Math. Biosci. Eng. 5, no. 1, p. 35-60 (2008). | MR | Zbl

[12] Borkovec (M.), Dasgupta (A.), Resnick (S.), and Samorodnitsky (G.).— A single channel on/off model with TCP-like control, Stoch. Models 18, no. 3, p. 333-367 (2002). | MR | Zbl

[13] Boscain (U.),.— Stability of planar switched systems: the linear single input case, SIAM J. Control Optim. 41 no. 1, p. 89-112 (2002). | MR | Zbl

[14] Bouguet (F.).— Quantitative exponential rates of convergence for exposure to food contaminants, To appear in ESAIM PS, arXiv:1310.3948, (2013).

[15] Calvez (V.), Doumic Jauffret (M.), and Gabriel (P.).— Self-similarity in a general aggregation-fragmentation problem. Application to fitness analysis, J. Math. Pures Appl. (9) 98, no. 1, p. 1-27 (2012). | MR | Zbl

[16] Chafaï (D.), Malrieu (F.), and Paroux (K.).— On the long time behavior of the TCP window size process, Stochastic Process. Appl. 120, no. 8, p. 1518-1534 (2010). | MR | Zbl

[17] Cloez (B.) and Hairer (M.).— Exponential ergodicity for Markov processes with random switching, Bernoulli 21, no. 1, p. 505-536 (2015). | MR

[18] Comets (F.), Popov (S.), Schütz (M.), and Vachkovskaia (M.).— Billiards in a general domain with random reflections, Arch. Ration. Mech. Anal. 191, no. 3, p. 497-537 (2009). | MR | Zbl

[19] Crudu (A.), Debussche (A.), Muller (A.), and Radulescu (O.).— Convergence of stochastic gene networks to hybrid piecewise deterministic processes, Ann. Appl. Probab. 22, no. 5, p. 1822-1859 (2012). | MR | Zbl

[20] Davis (M. H. A.).— Piecewise-deterministic Markov processes: a general class of nondiffusion stochastic models, J. Roy. Statist. Soc. Ser. B 46, no. 3, p. 353-388, With discussion (1984). | MR | Zbl

[21] Davis (M. H. A.).— Markov models and optimization, Monographs on Statistics and Applied Probability, vol. 49, Chapman & Hall, London, (1993). | MR | Zbl

[22] Diaconis (P.) and Freedman (P.).— Iterated random functions, SIAM Rev. 41, no. 1, p. 45-76 (1999). | MR | Zbl

[23] Doumic Jauffret (M.) and Gabriel (P.).— Eigenelements of a general aggregation-fragmentation model, Math. Models Methods Appl. Sci. 20, no. 5, p. 757-783 (2010). | MR | Zbl

[24] Dumas (V.), Guillemin (F.), and Robert (Ph.).— A Markovian analysis of additive-increase multiplicative-decrease algorithms, Adv. in Appl. Probab. 34, no. 1, p. 85-111 (2002). | MR | Zbl

[25] Erban (R.) and Othmer (H. G.).— From individual to collective behavior in bacterial chemotaxis, SIAM J. Appl. Math. 65, no. 2, p. 361-391 (electronic) (2004/05). | MR | Zbl

[26] Erban (R.) and Othmer (H. G.).— From signal transduction to spatial pattern formation in E. coli: a paradigm for multiscale modeling in biology, Multiscale Model. Simul. 3, no. 2, p. 362-394 (electronic) (2005). | MR | Zbl

[27] Fontbona (J.), Guérin (H.), and Malrieu (F.).— Quantitative estimates for the long-time behavior of an ergodic variant of the telegraph process, Adv. in Appl. Probab. 44, no. 4, p. 977-994 (2012). | MR | Zbl

[28] Gadat (S.) and Miclo (L.).— Spectral decompositions and ${𝕃}^2$-operator norms of toy hypocoercive semi-groups, Kinet. Relat. Models 6, no. 2, p. 317-372 (2013). | MR | Zbl

[29] Graham (C.) and Robert (Ph.).— Interacting multi-class transmissions in large stochastic networks, Ann. Appl. Probab. 19, no. 6, p. 2334-236 (2009)1. | MR | Zbl

[30] Graham (C.) and Robert (Ph.).— Self-adaptive congestion control for multiclass intermittent connections in a communication network, Queueing Syst. 69, no. 3-4, p. 237-257 (2011). | MR | Zbl

[31] Guillemin (F.), Robert (Ph.), and Zwart (B.).— AIMD algorithms and exponential functionals, Ann. Appl. Probab. 14, no. 1, p. 90-117 (2004). | MR | Zbl

[32] Herrmann (S.) and Vallois (P.).— From persistent random walk to the telegraph noise, Stoch. Dyn. 10, no. 2, p. 161-196 (2010). | MR | Zbl

[33] Hespanha (J. P.).— A model for stochastic hybrid systems with application to communication networks, Nonlinear Anal. 62, no. 8, p. 1353-1383 (2005). | MR | Zbl

[34] Jacobsen (M.).— Point process theory and applications, Probability and its Applications, Birkh?user Boston Inc., Boston, MA, Marked point and piecewise deterministic processes (2006). | MR | Zbl

[35] Kac (M.).— A stochastic model related to the telegrapherÕs equation, Rocky Mountain J. Math. 4, p. 497-509 (1974). | MR | Zbl

[36] Karmakar (R.) and Bose (I.).— Graded and binary responses in stochastic gene expression, Physical Biology 197 no. 1, p. 197-214 (2004).

[37] Lamberton (D.) and Pagès (G.).— A penalized bandit algorithm, Electron. J. Probab. 13, no. 13, p. 341-373 (2008). | MR | Zbl

[38] Lawley (S. D.), Mattingly (J. C.), and Reed (M. C.).— Sensitivity to switching rates in stochastically switched ODEs, Commun. Math. Sci. 12, no. 7, p. 1343-1352 (2014). | MR

[39] van Leeuwaarden (J. S. H.) and Löpker (A. H.).— Transient moments of the TCP window size process, J. Appl. Probab. 45, no. 1, p. 163-175 (2008). | MR | Zbl

[40] van Leeuwaarden (J. S. H.), Löpker (A. H.), and Ott (T. J.).— TCP and iso-stationary transformations, Queueing Syst. 63, no. 1-4, p. 459-475 (2009). | MR | Zbl

[41] Levin (D. A.), Peres (Y.), and Wilmer (E. L.).— Markov chains and mixing times, American Mathematical Society, Providence, RI, With a chapter by James G. Propp and David B. Wilson (2009). | MR | Zbl

[42] Mackey (M. C.), Tyran-Kaminska (M.), and Yvinec (R.).— Dynamic behavior of stochastic gene expression models in the presence of bursting, SIAM J. Appl. Math. 73, no. 5, p. 1830-1852 (2013). | MR | Zbl

[43] Maulik (K.) and Zwart (B.).— Tail asymptotics for exponential functionals of Lévy processes, Stochastic Process. Appl. 116, no. 2, p. 156-177 (2006). | MR | Zbl

[44] Maulik (K.) and Zwart (B.).— An extension of the square root law of TCP, Ann. Oper. Res. 170, p. 217-232 (2009). | MR | Zbl

[45] Miclo (L.) and Monmarché (P.).— , ?tude spectrale minutieuse de processus moins indécis que les autres, Séminaire de Probabilités XLV, Lecture Notes in Math., vol. 2078, Springer, Cham, p. 459-481 (2013). | MR

[46] Mischler (S.) and Scher (J.).— Spectral analysis of semigroups and growth-fragmentation equations, arXiv:1310.7773, (2013).

[47] Monmarché (P.).— Hypocoercive relaxation to equilibrium for some kinetic models, Kinet. Relat. Models 7, no. 2, p. 341-360 (2014). | MR

[48] Monmarché (P.).— On $H^1$ and entropic convergence for contractive PDMP, arXiv:1404.4220 (2014).

[49] Morris (C.) and Lecar (H.).— Voltage oscillations in the barnacle giant muscle fiber, Biophys. J. 35, no. 1, p. 193-213 (1981).

[50] Othmer (H. G.), Dunbar (S. R.), and Alt (W.).— Models of dispersal in biological systems, J. Math. Biol. 26, no. 3, p. 263-298 (1988). | MR | Zbl

[51] Ott (T. J.).— Rate of convergence for the Ôsquare root formulaÕ in the internet transmission control protocol, Adv. in Appl. Probab. 38, no. 4, p. 1132-1154 (2006). | MR | Zbl

[52] Ott (T. J.) and Kemperman (J. H. B.).— Transient behavior of processes in the TCP paradigm, Probab. Engrg. Inform. Sci. 22, no. 3, p. 431-471 (2008). | MR | Zbl

[53] Ott (T. J.), Kemperman (J. H. B.), and Mathis (M.).— The stationary behavior of ideal TCP congestion avoidance, unpublished manuscript available at http://www.teunisott.com/ (1996).

[54] Ott (T. J.) and Swanson (J.).— Asymptotic behavior of a generalized TCP congestion avoidance algorithm, J. Appl. Probab. 44, no. 3, p. 618-635 (2007). | MR | Zbl

[55] Pakdaman (K.), Thieullen (M.), and Wainrib (G.).— Fluid limit theorems for stochastic hybrid systems with application to neuron models, Adv. in Appl. Probab. 42, no. 3, p. 761-794 (2010). | MR | Zbl

[56] Perthame (B.).— Transport equations in biology, Frontiers in Mathematics, Birkhäuser Verlag, Basel (2007). | MR | Zbl

[57] Perthame (B.) and Ryzhik (L.).— Exponential decay for the fragmentation or cell-division equation, J. Differential Equations 210, no. 1, p. 155-177 (2005). | MR | Zbl

[58] Radulescu (O.), Muller (A.), and Crudu (A.).— Théorèmes limites pour des processus de Markov à sauts. Synthèse des résultats et applications en biologie moléculaire, Technique et Science Informatiques 26, no. 3-4, p. 443-469 (2007).

[59] Roberts (G. O.) and Tweedie (R. L.).— Rates of convergence of stochastically monotone and continuous time Markov models, J. Appl. Probab. 37, no. 2, p. 359-373 (2000). | MR | Zbl

[60] Rousset (M.) and Samaey (G.).— Individual-based models for bacterial chemotaxis in the diffusion asymptotics, Math. Models Methods Appl. Sci. 23, no. 11, p. 2005-2037 (2013). | MR

[61] Shao (J.).— Ergodicity of regime-switching diffusions in Wasserstein distances, Stochastic Process. Appl. 125, no. 2, p. 739-758 (2015). | MR

[62] Villani (C.).— Topics in optimal transportation, Graduate Studies in Mathematics, vol. 58, American Mathematical Society, Providence, RI (2003). | MR | Zbl

[63] Wainrib (G.), Thieullen (M.), and Pakdaman (K.).— Intrinsic variability of latency to first-spike, Biol. Cybernet. 103, no. 1, p. 43-56 (2010). | MR | Zbl

[64] Yin (G. G.) and Zhu (C.).— Hybrid switching diffusions, Stochastic Modelling and Applied Probability, vol. 63, Springer, New York, Properties and applications (2010). | MR | Zbl

[65] Yvinec (R.), Zhuge (C.), Lei (J.), and Mackey (M. C.).— Adiabatic reduction of a model of stochastic gene expression with jump Markov process, J. Math. Biol. 68, no. 5, p. 1051-1070 (2014). | MR | Zbl