An explicit solution for the problem of probability reconstruction when only the averages of random variables are known is given by the maximum entropy method. We use this method to reconstruct a function constrained to a convex set (no linear constraint) using a finite number of its generalized moments (linear constraint). A sequence of entropy maximization problems is considered. The nth problem consists in the reconstruction of a probability distribution on , the projection of on , whose mean satisfies a constraint approximating the initial linear constraint (generalized moments). When n approaches infinity this gives a solution for the initial problem as the limit of the sequence of means of maximum entropy distributions on . We call this technique the maximum entropy method on the mean (M.E.M) because linear constraints are only on the mean of the distribution to be reconstructed. We mainly study the case where is a band of continuous functions. We find a reconstruction family, each element of this family only depends of referenced measures used for the sequence of entropy problems. We show that the M.E.M method is equivalent to a concav criteria maximization. We then use the M.E.M method to construct a numerically computable criteria to solve generalized moments problem on a bounded band of continuous functions. In the last chapter we discuss statistical applications of the method.
Keywords: Maximum of entropy, Functionna.1 estimation, Generalized moments, Moment problems, Large deviations
@phdthesis{BJHTUP11_1989__0248__P0_0,
author = {Gamboa, Fabrice},
title = {M\'ethode du maximum d'entropie sur la moyenne et applications},
series = {Th\`eses d'Orsay},
publisher = {Universit\'e de Paris-Sud Centre d'Orsay},
number = {248},
year = {1989},
language = {fr},
url = {http://www.numdam.org/item/BJHTUP11_1989__0248__P0_0/}
}
Gamboa, Fabrice. Méthode du maximum d'entropie sur la moyenne et applications. Thèses d'Orsay, no. 248 (1989), 145 p. http://numdam.org/item/BJHTUP11_1989__0248__P0_0/
[Ag-Bouc, 80]. , : Minimum of h-divergence and maximum likelihood estimation, Jour Inf Sci, 3, pp 296-307. (1980). | MR | Zbl
[Az-Rug, 77]. , : Mélanges d'équations différentielles et grands écarts à la loi des grands nombres, Z. Wahr, 38, pp 1-54. (1977). | MR | Zbl | DOI
[Barn, 78]. : Imformation and exponential families in statistical theory, Wiley. (1978). | MR | Zbl
[Bret, 79]. : Formule de Chernoff pour les lois empiriques de variables à valeurs dans des espaces généraux, Grandes déviations et applications statistiques, Ast 68, pp 33-52, Soc Math Fran. (1979) | Zbl | Numdam
[Brez, 83]. : Analyse fonctionnelle, théorie et applications, Masson. (1983) | MR | Zbl
[Bric, 84]. : Maximum entropy and the foundations of direct methods, Acta Cryst, A40, pp 410-445. (1984). | DOI
[Burg, 75]. : Maximum entropy spectral analysis, Ph D thesis, Dept of Geophysics, Standford University. (1975).
[Chov, 61]. : On normalized entropy and the extensions of a positive definite function, J Math Mec, 10, pp 927-945. (1961). | MR | Zbl
[Ciar, 82]. : Introduction à l'analyse numérique matricielle et à l'optimisation, Masson. (1982). | MR | Zbl
[Cot-Fort-Mal, 83]. , , : Large deviations and rare events in the study of stochastic algorithms, I.E.E.E Trans on Aut Cont, Vol 28, pp 907-920. (1983). | MR | Zbl
[Csis, 67]. : Information type measures of probability distributions and indirect observations, Studia Sci, Math Hungar, 2, pp 299-318. (1967). | MR | Zbl
[Csis, 75]. : I-divergence geometry of distributions, Ann Prob, 3, pp 146- 158. (1975). | MR | Zbl | DOI
[Csis, 84]. : Sanov property, generalized I-projection, and a conditional limit theorem, Ann Prob, 12, pp 768-793. (1984) | MR | Zbl | DOI
[Dac-Duf, 82]. , : Probabilités et statistiques, Tome 1, Masson. (1982). | Zbl
[Dac-Duf, 83]. , : Probabilités et statistiques, Tome 2, Masson. (1983). | Zbl
[Dac, 84]. : Reconstruction des phases en cristallographie par maximum d'entropie, Seminaire Bourbaki, 628. (1984). | Numdam | Zbl
[Dac-Gam, 89]. , : Maximisation de l'entropie sous contraintes non linéaires, Prépublication, Orsay. (1989). | MR | Zbl
[El, 85]. : Entropy large deviations and statistical mechanics, Springer Verlag. (1985). | MR | Zbl
[Far, 84]. : Extension of maximum entropy spectral analysis, I.E.E Proceedings, Vol 131.(1984).
[Gam-Gas, 89]. , : Maximum d'entropie et problème des moments: cas multidimensionnel, Prépublication, Orsay. (1989). | MR | Zbl
[Gas, 86]. : Problème sommatoire par maximum d'entropie, Notes C.R.A.S, t.303, Serie I, pp 675-680. (1986). | MR | Zbl
[Gih-Skor, 74]. , : The theory of stochastic processes, Springer Verlag. (1974). | MR | Zbl
[Gren-Szeg, 58]. , : Toeplitz forms and their applications, University of California Press. (1958). | MR
[Gull-Skil, 84]. , : Maximum entropy method in image processing, I.E.E Proceedings, Vol 131, pp 646-659. (1984).
[Gull-Skil, 85]. , : The entropy of an image, Maximum entropy and Bayesian methods in inverse problems, D. Reidel Publishing company, pp 287-301. (1985)
[Gzyl, 88]. : El metodo de maxima entropia, Prepublication, Universidad de los Andes, Merida. (1988). (A paraître en anglais chez Springer Verlag (1990)
[Jayn, 57]. : Imformation theory and statistical mechanics, Phys Rev, Vol 106, pp 620-630. (1957). | MR | Zbl | DOI
[Jayn, 82]. : On the rational of maximum entropy methods, Proc of the I.E.E.E 70, pp 939-952. (1982).
[Krein-Nud, 77]. , : The Markov moment problem and extremal problems, Am Math Soc, Vol 50. (1977). | Zbl
[Land, 87]. : Maximum entropy and the moment problem, Bull Am Math Soc, Vol 16, pp 47-77. (1987). | MR | Zbl | DOI
[Lip, 86]. : A maximum entropy for expert systems, Brown University thesis. (1986).
[Max, 85]. Maximum entropy and Bayesian methods in inverse problems, D. Reidel Publishing company. (1985)
[Moham, 87]. : Synthèse de Fourier multidimensionnelle à maximum d'entropie: Application à la reconstruction tomographique d'images, Thèse d'état, Paris 11.(1987).
[Nav, 85]. : On the maximum entropy estimate of electron density function, Act Cryst, A41, pp 232-241. (1985). | DOI
[Nav, 86]. : The use of non local constraints in maximum entropy electron density reconstruction, Act Cryst, A42, pp 212-222. (1986). | DOI
[Rob, 89]. : An entropy concentration theorem, Prépublication. Grenoble (1989). | MR | Zbl
[Rock, 70]. : Convex analysis, Princeton University Press. (1970). | MR | Zbl | DOI
[Ros, 85]. : Stationary sequences and random fields, Birckauser Boston. (1985). | MR | Zbl | DOI
[Schoenb, 64]. : Spline functions and the problem of graduation, Proc Nat Acad Sci U.S.A, 52, pp 947-950. (1964) | MR | Zbl | DOI
[Seg, 87]. : Reconstruction de la densité spectrale par maximum d'entropie cas d-dimensionnel, Notes C.R.A.S, t.305, Serie I, pp 517-520. (1987). | MR | Zbl
[Seg, 88]. : Extension de fonctions de type positif et entropie associée. Cas multidimensionnel, Prépuplication, Orsay. (1988). | Zbl | Numdam
[Shore-John, 80]. , : Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy, I.E.E.E Trans Inf Theory, Vol 26, pp 26-36. (1980). | MR | Zbl | DOI
[Van-Cov, 81]. , : Maximum entropy and conditional probability, I.E.E.E Trans Inf Theory, Vol 27, pp 483-489. (1981). | MR | Zbl | DOI
[Wahba, 83]. : Bayesian confidence intervals for the cross-validated smoothing spline, J.R Stat Soc B, 45, pp 133-150. (1983). | MR | Zbl
[Wragg-Dow, 70]. , : Fitting continuous probability density functions over using information theory ideas, I.E.E.E Trans Inf Theory, Vol 16, pp 226-230. (1970). | Zbl | DOI
[Zh-Os-Har, 87]. , , : A differential equation approach to maximum entropy image reconstruction, I.E.E.E Trans A.S.S.P, Vol 35, pp 208-225. (1987). | MR
