To establish lists of words with unexpected frequencies in long sequences, for instance in a molecular biology context, one needs to quantify the exceptionality of families of word frequencies in random sequences. To this aim, we study large deviation probabilities of multidimensional word counts for Markov and hidden Markov models. More specifically, we compute local Edgeworth expansions of arbitrary degrees for multivariate partial sums of lattice valued functionals of finite Markov chains. This yields sharp approximations of the associated large deviation probabilities. We also provide detailed simulations. These exhibit in particular previously unreported periodic oscillations, for which we provide theoretical explanations.
Mots clés : Markov chains, hidden Markov models, large deviations, edgeworth expansions, protein and DNA sequences
@article{PS_2010__14__435_0, author = {Pudlo, Pierre}, title = {Large deviations and full edgeworth expansions for finite {Markov} chains with applications to the analysis of genomic sequences}, journal = {ESAIM: Probability and Statistics}, pages = {435--455}, publisher = {EDP-Sciences}, volume = {14}, year = {2010}, doi = {10.1051/ps/2009008}, language = {en}, url = {http://www.numdam.org/articles/10.1051/ps/2009008/} }
TY - JOUR AU - Pudlo, Pierre TI - Large deviations and full edgeworth expansions for finite Markov chains with applications to the analysis of genomic sequences JO - ESAIM: Probability and Statistics PY - 2010 SP - 435 EP - 455 VL - 14 PB - EDP-Sciences UR - http://www.numdam.org/articles/10.1051/ps/2009008/ DO - 10.1051/ps/2009008 LA - en ID - PS_2010__14__435_0 ER -
%0 Journal Article %A Pudlo, Pierre %T Large deviations and full edgeworth expansions for finite Markov chains with applications to the analysis of genomic sequences %J ESAIM: Probability and Statistics %D 2010 %P 435-455 %V 14 %I EDP-Sciences %U http://www.numdam.org/articles/10.1051/ps/2009008/ %R 10.1051/ps/2009008 %G en %F PS_2010__14__435_0
Pudlo, Pierre. Large deviations and full edgeworth expansions for finite Markov chains with applications to the analysis of genomic sequences. ESAIM: Probability and Statistics, Tome 14 (2010), pp. 435-455. doi : 10.1051/ps/2009008. http://www.numdam.org/articles/10.1051/ps/2009008/
[1] Sharp estimates of deviations of the sample mean in many dimensions. Ann. Inst. H. Poincaré Probab. Statist. 33 (1997) 371-385. | Numdam | Zbl
and ,[2] On deviations of the sample mean. Ann. Math. Statist. 31 (1960) 1015-1027. | Zbl
and ,[3] Large-deviation probability and the local dimension of sets, in Proceedings of the 19th Seminar on Stability Problems for Stochastic Models, Vologda, 1998, Part I. (2000), Vol. 99, pp. 1225-1233. | Zbl
and ,[4] Strong large deviation and local limit theorems. Ann. Probab. 21 (1993) 1671-1690. | Zbl
and ,[5] On the first-order Edgeworth expansion for a Markov chain. J. Multivariate Anal. 44 (1993) 345-359. | Zbl
and ,[6] Large deviations techniques and applications. Volume 38 of Appl. Math. (New York). Second edition. Springer-Verlag, New York (1998). | Zbl
and ,[7] Hidden word statistics. J. ACM 53 (2006) 147-183 (electronic).
, and ,[8] Sharp asymptotics of large deviations in Rd. J. Theoret. Probab. 8 (1995) 501-522. | Zbl
,[9] Sharp asymptotics of large deviations for general state-space Markov-additive chains in Rd. Statist. Probab. Lett. 47 (2000) 365-380. | Zbl
,[10] Large deviations of uniformly recurrent Markov additive processes. Adv. Appl. Math. 6 (1985) 373-412. | Zbl
, and ,[11] Saddlepoint approximations. The Clarendon Press Oxford University Press, New York (1995).
,[12] A large deviation inequality for vector functions on finite reversible Markov chains. Ann. Appl. Probab. 17 (2007) 1202-1221. | Zbl
,[13] Theory of Functions, Part I. Elements of the General Theory of Analytic Functions. Dover Publications, New York (1945).
,[14] Spectral theory and limit theorems for geometrically ergodic Markov processes. Ann. Appl. Probab. 13 (2003) 304-362. | Zbl
and ,[15] Optimal Hoeffding bounds for discrete reversible Markov chains. Ann. Appl. Probab. 14 (2004) 958-970. | Zbl
and ,[16] Multiple pattern matching: a Markov chain approach. J. Math. Biol. 56 (2008) 51-92. | Zbl
, and ,[17] Berry-Esseen Central Limit Theorems For Markov Chains. Ph.D. thesis, Harvard University, 1996.
,[18] A convexivity property in the theory of random variables defined on a finite Markov chain. Ann. Math. Statist. 32 (1961) 1260-1270. | Zbl
,[19] Dominating points and the asymptotics of large deviations for random walk on Rd. Ann. Probab. 11 (1983) 158-167. | Zbl
,[20] Markov additive processes, Part I. Eigenvalue properties and limit theorems. Ann. Probab. 15 (1987) 561-592. | Zbl
and ,[21] Motif statistics. In Algorithms - ESA '99, Prague. Lect. Notes Comput. Sci. 1643. Springer, Berlin (1999), pp 194-211. | Zbl
, and ,[22] Numerical solutins for Patterns Statistics on Markov chains. Stat. Appl. Genet. Mol. Biol. 5 (2006). | Zbl
,[23] Pattern Markov chains: optimal Markov chain embedding through deterministic finite automata. J. Appl. Probab. 45 (2008) 226-243. | Zbl
,[24] R Development Core Team, R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria (2003). ISBN 3-900051-00-3.
[25] A unified approach to word occurrence probabilities. Discrete Appl. Math. 104 (2000) 259-280, Combinatorial molecular biology. | Zbl
,[26] Rare events and conditional events on random strings. Discrete Math. Theor. Comput. Sci. 6 (2004) 191-213 (electronic). | Zbl
and ,[27] On pattern frequency occurrences in a Markovian sequence. Algorithmica 22 (1998) 631-649. | Zbl
and ,[28] Applied Combinatorics on Words. In Encyclopedia of Mathematics and its Applications, Vol. 105, chap. Statistics on Words with Applications to Biological Sequences. Cambridge University Press (2005).
, and ,[29] Exact distribution of word occurrences in a random sequence of letters. J. Appl. Probab. 36 (1999) 179-193. | Zbl
and ,[30] Improved compound Poisson approximation for the number of occurrences of any rare word family in a stationary Markov chain. Adv. Appl. Probab. 39 (2007) 128-140. | Zbl
and ,[31] Compound Poisson approximation of word counts in DNA sequences. ESAIM: PS 1 (1997) 1-16. | Numdam | Zbl
,[32] Matrices, volume 216 of Graduate Texts Math.. Springer-Verlag, New York (2002). Theory and applications, translated from the 2001 French original. | Zbl
,[33] Waiting times for clumps of patterns and for structured motifs in random sequences. Discrete Appl. Math. 155 (2007) 868-880. | Zbl
, and ,Cité par Sources :