Empirical Bayes Conditional Density Estimation
Keywords:Adaptive estimation, Bayesian nonparametrics, Conditional density, Dimension reduction, Hölder spaces, minimax rates of convergence
The problem of nonparametric estimation of the conditional density of a response, given a vector of explanatory variables, is classical and of prominent importance in many prediction problems since the conditional density provides a more comprehensive description of the association between the response and the predictor than, for instance, does the regression function. The problem has applications across different fields like economy, actuarial sciences and medicine. We investigate empirical Bayes estimation of conditional densities establishing that an automatic data-driven selection of the prior hyper-parameters in infinite mixtures of Gaussian kernels, with predictor-dependent mixing weights, can lead to estimators whose performance is on par with that of frequentist estimators in being minimax-optimal (up to logarithmic factors) rate adaptive over classes of locally Hölder smooth conditional densities and in performing an adaptive dimension reduction if the response is independent of (some of) the explanatory variables which, containing no information about the response, are irrelevant to the purpose of estimating its conditional density.
K. BERTIN, C. LACOUR, V. RIVOIRARD (2015). Adaptive pointwise estimation of conditional density function. Annales de l’Institut Henri Poincaré (to appear).
S. DONNET, V. RIVOIRARD, J. ROUSSEAU, C. SCRICCIOLO (2014). Posterior concentration rates for empirical Bayes procedures, with applications to Dirichlet Process mixtures. URL http://arxiv.org/pdf/1406.4406.pdf.
S. EFROMOVICH (2007). Conditional density estimation in a regression setting. The Annals of Statistics, 35, no. 6, pp. 2504–2535.
S. EFROMOVICH (2010). Oracle inequality for conditional density estimation and an actuarial example. Annals of the Institute of Statistical Mathematics, 62, no. 2, pp. 249–275.
S. GHOSAL, A. VAN DER VAART (2007). Posterior convergence rates of Dirichlet mixtures at smooth densities. The Annals of Statistics, 35, no. 2, pp. 697–723.
P. HALL, J. RACINE, Q. LI (2004). Cross-validation and the estimation of conditional probability densities. Journal of the American Statistical Association, 99, no. 468, pp. 1015–1026.
W. KRUIJER, J. ROUSSEAU, A. VAN DER VAART (2010). Adaptive Bayesian density estimation with location-scale mixtures. Electronic Journal of Statistics, 4, pp. 1225–1257.
A. NORETS, D. PATI (2014). Adaptive Bayesian estimation of conditional densities. URL http://arxiv.org/pdf/1408.5355.pdf.
D. PATI, D. B. DUNSON, S. T. TOKDAR (2013). Posterior consistency in conditional distribution estimation. Journal of Multivariate Analysis, 116, pp. 456–472.
S. PETRONE, S. RIZZELLI, J. ROUSSEAU, C. SCRICCIOLO (2014a). Empirical Bayes methods in classical and Bayesian inference. METRON, 72, no. 2, pp. 201–215.
S. PETRONE, J. ROUSSEAU, C. SCRICCIOLO (2014b). Bayes and empirical Bayes: do they merge? Biometrika, 101, no. 2, pp. 285–302.
A. RODRÍGUEZ, D. B. DUNSON (2011). Nonparametric Bayesian models through probit stick-breaking processes. Bayesian Analysis, 6, no. 1, pp. 145–177.
C. SCRICCIOLO (2015). Bayesian adaptation. Journal of Statistical Planning and Inference (in press).
W. SHEN, S. T. TOKDAR, S. GHOSAL (2013). Adaptive Bayesian multivariate density estimation with Dirichlet mixtures. Biometrika, 100, no. 3, pp. 623–640.