This paper presents a practical and simple fully nonparametric multivariate smoothing procedure that adapts to the underlying smoothness of the true regression function. Our estimator is easily computed by successive application of existing base smoothers (without the need of selecting an optimal smoothing parameter), such as thin-plate spline or kernel smoothers. The resulting smoother has better out of sample predictive capabilities than the underlying base smoother, or competing structurally constrained models (MARS, GAM) for small dimension (3 ≤ d ≤ 7) and moderate sample size n ≤ 1000. Moreover our estimator is still useful when d > 10 and to our knowledge, no other adaptive fully nonparametric regression estimator is available without constrained assumption such as additivity for example. On a real example, the Boston Housing Data, our method reduces the out of sample prediction error by 20%. An R package ibr, available at CRAN, implements the proposed multivariate nonparametric method in R.
Mots-clés : nonparametric regression, smoother, kernel, thin-plate splines, stopping rules
@article{PS_2014__18__483_0,
author = {Cornillon, Pierre-Andr\'e and Hengartner, N. W. and Matzner-L{\o}ber, E.},
title = {Recursive bias estimation for multivariate regression smoothers},
journal = {ESAIM: Probability and Statistics},
pages = {483--502},
publisher = {EDP-Sciences},
volume = {18},
year = {2014},
doi = {10.1051/ps/2013046},
language = {en},
url = {http://www.numdam.org/articles/10.1051/ps/2013046/}
}
TY - JOUR AU - Cornillon, Pierre-André AU - Hengartner, N. W. AU - Matzner-Løber, E. TI - Recursive bias estimation for multivariate regression smoothers JO - ESAIM: Probability and Statistics PY - 2014 SP - 483 EP - 502 VL - 18 PB - EDP-Sciences UR - http://www.numdam.org/articles/10.1051/ps/2013046/ DO - 10.1051/ps/2013046 LA - en ID - PS_2014__18__483_0 ER -
%0 Journal Article %A Cornillon, Pierre-André %A Hengartner, N. W. %A Matzner-Løber, E. %T Recursive bias estimation for multivariate regression smoothers %J ESAIM: Probability and Statistics %D 2014 %P 483-502 %V 18 %I EDP-Sciences %U http://www.numdam.org/articles/10.1051/ps/2013046/ %R 10.1051/ps/2013046 %G en %F PS_2014__18__483_0
Cornillon, Pierre-André; Hengartner, N. W.; Matzner-Løber, E. Recursive bias estimation for multivariate regression smoothers. ESAIM: Probability and Statistics, Tome 18 (2014), pp. 483-502. doi : 10.1051/ps/2013046. http://www.numdam.org/articles/10.1051/ps/2013046/
[1] , Computationally efficient classes of higher-order kernel functions. Can. J. Statist. 23 (1995) 21-27. | MR | Zbl
[2] , Using adaptive bagging to debias regressions. Technical Report 547, Dpt of Statist., UC Berkeley (1999).
[3] and , Estimating optimal transformation for multiple regression and correlation. J. Amer. Stat. Assoc. 80 (1995) 580-598. | MR | Zbl
[4] and , Boosting with the l2 loss: Regression and classification. J. Amer. Stat. Assoc. 98 (2003) 324-339. | MR | Zbl
[5] , and , Recursive bias estimation and l2 boosting. Technical report, ArXiv:0801.4629 (2008).
[6] P.-A. Cornillon, N. Hengartner and Matzner-Løber, ibr: Iterative Bias Reduction. CRAN (2010). http://cran.r-project.org/web/packages/ibr/index.html.
[7] P.-A. Cornillon, N. Hengartner, N. Jégou and Matzner-Løber, Iterative bias reduction: a comparative study. Statist. Comput. (2012).
[8] and , Smoothing noisy data with spline functions. Numer. Math. 31 (1979) 377-403. | MR | Zbl
[9] and , On boosting kernel regression. J. Statist. Plan. Infer. 138 (2008) 2483-2498. | MR | Zbl
[10] , Nonparametric regression and spline smoothing. Dekker, 2nd edition (1999). | MR | Zbl
[11] , An introduction to probability and its applications, vol. 2. Wiley (1966). | MR | Zbl
[12] , Multivariate adaptive regression splines. Ann. Statist. 19 (1991) 337-407. | MR | Zbl
[13] , Greedy function approximation: A gradient boosting machine. Ann. Statist. 28 (1189-1232) (2001). | MR | Zbl
[14] and , Projection pursuit regression. J. Amer. Statist. Assoc. 76 (817-823) (1981). | MR
[15] , and , Additive logistic regression: a statistical view of boosting. Ann. Statist. 28 (2000) 337-407. | MR | Zbl
[16] , Smoothing spline ANOVA models. Springer (2002). | MR | Zbl
[17] , , and , A Distribution-Free Theory of Nonparametric Regression. Springer Verlag (2002). | MR | Zbl
[18] and , Hedonic prices and the demand for clean air. J. Environ. Econ. Manag. (1978) 81-102. | Zbl
[19] and , Generalized Additive Models. Chapman & Hall (1995). | MR | Zbl
[20] and , Matrix analysis. Cambridge (1985). | MR | Zbl
[21] , and , Smoothing parameter selection in nonparametric regression using and improved akaike information criterion. J. Roy. Stat. Soc. B 60 (1998) 271-294. | MR | Zbl
[22] , Asymptotically minimax adaptive estimation. I: upper bounds. optimally adaptive estimates. Theory Probab. Appl. 37 (1991) 682-697. | MR | Zbl
[23] , Asymptotic optimality for Cp, CL, cross-validation and generalized cross-validation: Discrete index set. Ann. Statist. 15 (1987) 958-975. | MR | Zbl
[24] , Additive logistic regression: a statistical view of boosting: Discussion. Ann. Statist. 28 (2000) 393-400. | MR | Zbl
[25] , Analyse IV applications à la théorie de la mesure. Hermann (1993). | Zbl
[26] and , Some comments on the asymptotic behavior of robust smoothers, in Smoothing Techniques for Curve Estimation, edited by T. Gasser and M. Rosenblatt. Springer-Verlag (1979) 191-195. | MR | Zbl
[27] , Explanatory Data Analysis. Addison-Wesley (1977). | Zbl
[28] , Convergence rates for multivariate smoothing spline functions. J. Approx. Theory (1988) 1-27. | MR | Zbl
[29] , Smoothing Noisy Data with Multivariate Splines and Generalized Cross-Validation. PhD thesis, University of Wisconsin (1982). | MR
[30] , Thin plate regression splines. J. R. Statist. Soc. B 65 (2003) 95-114. | MR | Zbl
[31] , Combining different procedures for adaptive regression. J. Mult. Analysis 74 (2000) 135-161. | MR | Zbl
Cité par Sources :
