Soit un couple aléatoire à valeurs dans et de loi inconnue. Soient des répliques i.i.d. de , de loi empirique associée . Soit un dictionnaire composé de fonctions. Pour tout , on note . Soit fonction de perte donnée que l’on suppose convexe en la seconde variable. On note . On étudie le problème de minimisation du risque empirique pénalisé suivant
Let be a random couple in with unknown distribution . Let be i.i.d. copies of , being their empirical distribution. Let be a dictionary consisting of functions. For , denote . Let be a given loss function, which is convex with respect to the second variable. Denote . We study the following penalized empirical risk minimization problem
Mots clés : empirical risk, penalized empirical risk, ℓ_p-penalty, sparsity, oracle inequalities
@article{AIHPB_2009__45_1_7_0, author = {Koltchinskii, Vladimir}, title = {Sparsity in penalized empirical risk minimization}, journal = {Annales de l'I.H.P. Probabilit\'es et statistiques}, pages = {7--57}, publisher = {Gauthier-Villars}, volume = {45}, number = {1}, year = {2009}, doi = {10.1214/07-AIHP146}, mrnumber = {2500227}, zbl = {1168.62044}, language = {en}, url = {http://www.numdam.org/articles/10.1214/07-AIHP146/} }
TY - JOUR AU - Koltchinskii, Vladimir TI - Sparsity in penalized empirical risk minimization JO - Annales de l'I.H.P. Probabilités et statistiques PY - 2009 SP - 7 EP - 57 VL - 45 IS - 1 PB - Gauthier-Villars UR - http://www.numdam.org/articles/10.1214/07-AIHP146/ DO - 10.1214/07-AIHP146 LA - en ID - AIHPB_2009__45_1_7_0 ER -
%0 Journal Article %A Koltchinskii, Vladimir %T Sparsity in penalized empirical risk minimization %J Annales de l'I.H.P. Probabilités et statistiques %D 2009 %P 7-57 %V 45 %N 1 %I Gauthier-Villars %U http://www.numdam.org/articles/10.1214/07-AIHP146/ %R 10.1214/07-AIHP146 %G en %F AIHPB_2009__45_1_7_0
Koltchinskii, Vladimir. Sparsity in penalized empirical risk minimization. Annales de l'I.H.P. Probabilités et statistiques, Tome 45 (2009) no. 1, pp. 7-57. doi : 10.1214/07-AIHP146. http://www.numdam.org/articles/10.1214/07-AIHP146/
[1] Risk bounds for model selection via penalization. Probab. Theory Related Fields 113 (1999) 301-413. | MR | Zbl
, and .[2] Local Rademacher complexities. Ann. Statist. 33 (2005) 1497-1537. | MR | Zbl
, and .[3] Lectures on Modern Convex Optimization. Analysis, Algorithms and Engineering Applications. MPS/SIAM, Series on Optimization, Philadelphia, 2001. | MR | Zbl
and .[4] Aggregation for Gaussian regression. Ann. Statist. 35 (2007) 1674-1697. | MR
, and .[5] Sparsity oracle inequalities for the LASSO. Electron. J. Statist. 1 (2007) 169-194. | MR | Zbl
, and .[6] The Dantzig selector statistical estimation when p is much larger than n. Ann. Statist. 35 (2007) 2313-2351. | MR | Zbl
and .[7] Error correction via linear programming. In Proc. 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS05) 295-308. IEEE, Pittsburgh, PA, 2005.
, , and .[8] Stable signal recovery from incomplete and inaccurate measurements. Comm. Pure Appl. Math. 59 (2006) 1207-1223. | MR | Zbl
, and .[9] Statistical Learning Theory and Stochastic Optimization. Springer, New York, 2004. | MR | Zbl
.[10] For most large underdetermined systems of equations the minimal ℓ1-norm near-solution approximates the sparsest near-solution. Preprint, 2004. | Zbl
.[11] For most large underdetermined systems of linear equations the minimal ℓ1-norm solution is also the sparsest solution. Comm. Pure Appl. Math. 59 (2006) 797-829. | MR | Zbl
.[12] Compressed sensing. IEEE Trans. Inform. Theory 52 (2006) 1289-1306. | MR
.[13] Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Trans. Inform. Theory 52 (2006) 6-18. | MR
, and .[14] High-dimensional generalized linear models and the Lasso. Ann. Statist. 36 (2008) 614-645. | MR | Zbl
.[15] Model selection and aggregation in sparse classification problems. Oberwolfach Reports Meeting on Statistical and Probabilistic Methods of Model Selection, October, 2005.
.[16] Local Rademacher complexities and oracle inequalities in risk mnimization. Ann. Statist. 34 (2006) 2593-2656. | MR | Zbl
.[17] Complexities of convex combinations and bounding the generalization error in classification. Ann. Statist. 33 (2005) 1455-1496. | MR | Zbl
and .[18] Probability in Banach Spaces. Springer, New York, 1991. | MR | Zbl
and .[19] Some applications of concentration inequalities to statistics. Ann. Fac. Sci. Tolouse (IX) 9 (2000) 245-303. | Numdam | MR | Zbl
.[20] Concentration Inequalities and Model Selection. Springer, Berlin, 2007. | MR | Zbl
.[21] Reconstruction and subgaussian operators in Asymptotic Geometric Analysis. Geom. Funct. Anal. 17 (2007) 1248-1282. | MR | Zbl
, and .[22] High-dimensional graphs and variable selection with the LASSO. Ann. Statist. 34 (2006) 1436-1462. | MR | Zbl
and .[23] Topics in non-parametric statistics. In Ecole d'Et'e de Probabilités de Saint-Flour XXVIII, 1998 85-277. P. Bernard (Ed). Springer, New York, 2000. | MR | Zbl
.[24] Geometric approach to error correcting codes and reconstruction of signals. Int. Math. Res. Not. 64 (2005) 4019-4041. | MR | Zbl
and .[25] Regression shrinkage and selection via Lasso. J. Royal Statist. Soc. Ser. B 58 (1996) 267-288. | MR | Zbl
.[26] Optimal rates of aggregation. In Proc. 16th Annual Conference on Learning Theory (COLT) and 7th Annual Workshop on Kernel Machines, 303-313. Lecture Notes in Artificial Intelligence 2777. Springer, New York, 2003.
.[27] Weak Convergence and Empirical Processes. Springer, New York, 1996. | MR | Zbl
and .[28] Mixing strategies for density estimation. Ann. Statist. 28 (2000) 75-87. | MR | Zbl
.[29] Aggregating regression procedures for a better performance. Bernoulli 10 (2004) 25-47. | MR | Zbl
.[30] 7 (2006) 2541-2563. | MR
and . On model selection consistency of LASSO. J. Mach. Learn. Res.Cité par Sources :