Non-asymptotic oracle inequalities for the Lasso and Group Lasso in high dimensional logistic model

Kwemou, Marius

doi:10.1051/ps/2015020

Kwemou, Marius ^1,²

¹ Laboratoire de Mathématique et modélisation d’Evry UMR CNRS 8071- USC INRA, Université d’Évry Val d’Essonne, Evry, France.
² LERSTAD, Université Gaston Berger de Saint-Louis, Sénégal.

ESAIM: Probability and Statistics, Tome 20 (2016), pp. 309-331.

Résumé

We consider the problem of estimating a function $f_{0}$ in logistic regression model. We propose to estimate this function $f_{0}$ by a sparse approximation build as a linear combination of elements of a given dictionary of $p$ functions. This sparse approximation is selected by the Lasso or Group Lasso procedure. In this context, we state non asymptotic oracle inequalities for Lasso and Group Lasso under restricted eigenvalue assumption as introduced in [P.J. Bickel, Y. Ritov and A.B. Tsybakov, Ann. Statist. 37 (2009) 1705–1732].

Reçu le : 2014-07-01
Accepté le : 2015-12-04

MR Zbl

DOI : 10.1051/ps/2015020

Classification : 62H12, 62J12, 62J07, 62G20
Mots-clés : Logistic model, Lasso, Group Lasso, high-dimensional, oracle inequality

Affiliations des auteurs :

Kwemou, Marius ^{1, 2}

@article{PS_2016__20__309_0,
     author = {Kwemou, Marius},
     title = {Non-asymptotic oracle inequalities for the {Lasso} and {Group} {Lasso} in high dimensional logistic model},
     journal = {ESAIM: Probability and Statistics},
     pages = {309--331},
     publisher = {EDP-Sciences},
     volume = {20},
     year = {2016},
     doi = {10.1051/ps/2015020},
     mrnumber = {3533711},
     zbl = {1353.62054},
     language = {en},
     url = {http://www.numdam.org/articles/10.1051/ps/2015020/}
}

TY  - JOUR
AU  - Kwemou, Marius
TI  - Non-asymptotic oracle inequalities for the Lasso and Group Lasso in high dimensional logistic model
JO  - ESAIM: Probability and Statistics
PY  - 2016
SP  - 309
EP  - 331
VL  - 20
PB  - EDP-Sciences
UR  - http://www.numdam.org/articles/10.1051/ps/2015020/
DO  - 10.1051/ps/2015020
LA  - en
ID  - PS_2016__20__309_0
ER  -

%0 Journal Article
%A Kwemou, Marius
%T Non-asymptotic oracle inequalities for the Lasso and Group Lasso in high dimensional logistic model
%J ESAIM: Probability and Statistics
%D 2016
%P 309-331
%V 20
%I EDP-Sciences
%U http://www.numdam.org/articles/10.1051/ps/2015020/
%R 10.1051/ps/2015020
%G en
%F PS_2016__20__309_0

Kwemou, Marius. Non-asymptotic oracle inequalities for the Lasso and Group Lasso in high dimensional logistic model. ESAIM: Probability and Statistics, Tome 20 (2016), pp. 309-331. doi : 10.1051/ps/2015020. http://www.numdam.org/articles/10.1051/ps/2015020/

Bibliographie
Cité par

H. Akaike, Information theory and an extension of the maximum likelihood principle. In Second International Symposium on Information Theory (Tsahkadsor, 1971). Akadémiai Kiadó, Budapest (1973) 267–281. | MR | Zbl

F. Bach, Self-concordant analysis for logistic regression. Electron. J. Statist. 4 (2010) 384–414. | DOI | MR | Zbl

P.L. Bartlett, S. Mendelson and J. Neeman, 1-regularized linear regression: persistence and oracle inequalities. Probab. Theory Relat. Fields 154 (2012) 193–224. | DOI | MR | Zbl

P.J. Bickel, Y. Ritov and A.B. Tsybakov, Simultaneous analysis of Lasso and Dantzig selector. Ann. Statist. 37 (2009) 1705–1732. | DOI | MR | Zbl

S. Boucheron, G. Lugosi and O. Bousquet, Concentration inequalities. Adv. Lect. Machine Learn. (2004) 208–240. | Zbl

F. Bunea, A.B. Tsybakov and M.H. Wegkamp, Aggregation and sparsity via $l_{1}$ penalized least squares. In Learning theory, vol. 4005 of Lect. Notes Comput. Sci. Springer, Berlin (2006) 379–391. | MR | Zbl

F. Bunea, A.B. Tsybakov and M.H. Wegkamp, Aggregation for Gaussian regression. Ann. Statist. 35 (2007) 1674–1697. | DOI | MR | Zbl

F. Bunea, A. Tsybakov and M. Wegkamp, Sparsity oracle inequalities for the Lasso. Electron. J. Statist. 1 (2007) 169–194. | DOI | MR | Zbl

C. Chesneau and M. Hebiri, Some theoretical results on the grouped variables lasso. Math. Methods Statist. 17 (2008) 317–326. | DOI | MR | Zbl

J. Friedman, T. Hastie and R. Tibshirani, Regularization paths for generalized linear models via coordinate descent. J. Statist. Software 33 (2010) 1. | DOI

M. Garcia–Magariños, A. Antoniadis, R. Cao and W. González–Manteiga, Lasso logistic regression, GSoft and the cyclic coordinate descent algorithm: application to gene expression data. Stat. Appl. Genet. Mol. Biol. 9 (2010) 30. | DOI | MR | Zbl

T. Hastie, Non-parametric logistic regression. SLAC PUB-3160 (1983).

J. Huang, S. Ma and CH Zhang, The iterated lasso for high–dimensional logistic regression. Technical Report 392 (2008).

J. Huang, J.L. Horowitz and F. Wei, Variable selection in nonparametric additive models. Ann. Statist. 38 (2010) 2282. | DOI | MR | Zbl

G.M James, P. Radchenko and J. Lv, Dasso: connections between the dantzig selector and lasso. J. Roy. Statist. Soc. Ser. B 71 (2009) 127–142. | DOI | MR | Zbl

K. Knight and W. Fu, Asymptotics for lasso-type estimators. Ann. Statist. 28 (2000) 1356–1378. | MR | Zbl

K. Lounici, M. Pontil, A.B. Tsybakov and S. Van De Geer, Taking advantage of sparsity in multi-task learning. In COLT’09 (2009).

K. Lounici, M. Pontil, S. Van De Geer and A.B. Tsybakov, Oracle inequalities and optimal inference under group sparsity. Ann. Statist. 39 (2011) 2164–2204. | DOI | MR | Zbl

P. Massart, Concentration inequalities and model selection. Lectures from the 33rd Summer School on Probability Theory held in Saint-Flour, July 6–23, 2003. With a foreword by Jean Picard. Vol. 1896 of Lect. Notes Math. Springer, Berlin (2007). | MR | Zbl

P. Massart and C. Meynet, The Lasso as an $ℓ_{1}$ -ball model selection procedure. Electron. J. Statist. 5 (2011) 669–687. | DOI | MR | Zbl

J. Mcauley, J. Ming, D. Stewart and P. Hanna, Subband correlation and robust speech recognition. IEEE Trans. Speech Audio Process. 13 (2005) 956–964. | DOI

L. Meier, S. Van De Geer and P. Bühlmann, The group Lasso for logistic regression. J. Roy. Statist. Soc. Ser. B 70 (2008) 53–71. | DOI | MR | Zbl

L. Meier, S. Van De Geer and P. Bühlmann, High-dimensional additive modeling. Ann. Statist. 37 (2009) 3779–3821. | DOI | MR | Zbl

N. Meinshausen and P. Bühlmann, High-dimensional graphs and variable selection with the lasso. Ann. Statist. 34 (2006) 1436–1462. | DOI | MR | Zbl

N. Meinshausen and B. Yu, Lasso-type recovery of sparse representations for high-dimensional data. Ann. Statist. 37 (2009) 246–270. | DOI | MR | Zbl

Y. Nardi and A. Rinaldo, On the asymptotic properties of the group lasso estimator for linear models. Electron. J. Statist. 2 (2008) 605–633. | DOI | MR | Zbl

S.N. Negahban, P. Ravikumar, M.J. Wainwright and B. Yu, A unified framework for high-dimensional analysis of $M$ -estimators with decomposable regularizers. Statist. Sci. 27 (2012) 538–557. | DOI | Zbl

Y. Nesterov and A. Nemirovskii, Interior-point polynomial algorithms in convex programming. Vol. 13 of SIAM Studies in Applied Mathematics. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA (1994). | Zbl

M.R. Osborne, B. Presnell and B.A. Turlach, A new approach to variable selection in least squares problems. IMA J. Numer. Anal. 20 (2000) 389–403. | DOI | Zbl

M.Y. Park and T. Hastie, $L_{1}$ -regularization path algorithm for generalized linear models. J. Roy. Statist. Soc. Ser. B 69 (2007) 659–677. | DOI | Zbl

B. Tarigan and S.A. Van De Geer, Classifiers of support vector machine type with $l_{1}$ complexity regularization. Bernoulli 12 (2006) 1045–1076. | DOI | Zbl

R. Tibshirani, Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58 (1996) 267–288. | MR | Zbl

P. Ravikumar, J. Lafferty, H. Liu and L. Wasserman, Sparse additive models. J. Roy. Statist. Soc. Ser. B 71 (2009) 1009–1030. | DOI | Zbl

G. Schwarz, Estimating the dimension of a model. Ann. Statist. 6 (1978) 461–464. | DOI | Zbl

S.A. Van De Geer, High-dimensional generalized linear models and the lasso. Ann. Statist. 36 (2008) 614–645. | DOI | Zbl

S.A. Van De Geer and P. Bühlmann, On the conditions used to prove oracle results for the Lasso. Electron. J. Statist. 3 (2009) 1360–1392. | DOI | Zbl

T.T. Wu, Y.F. Chen, T. Hastie, E. Sobel and K. Lange, Genome-wide association analysis by lasso penalized logistic regression. Bioinform. 25 (2009) 714–721. | DOI

M. Yuan and Y. Lin, Model selection and estimation in regression with grouped variables. J. Roy. Statist. Soc. Ser. B 68 (2006) 49–67. | DOI | Zbl

C.-H. Zhang and J. Huang, The sparsity and bias of the LASSO selection in high-dimensional linear regression. Ann. Statist. 36 (2008) 1567–1594. | Zbl

P. Zhao and B. Yu, On model selection consistency of Lasso. J. Mach. Learn. Res. 7 (2006) 2541–2563. | Zbl

H. Zou, The adaptive lasso and its oracle properties. J. Am. Statist. Assoc. 101 (2006) 1418–1429. | DOI | Zbl

Cité par Sources :