Compressed Gaussian Process for Manifold Regression

Authors: 
Rajarshi Guhaniyogi, David B. Dunson
Duke University

Sep 25 2013

Nonparametric regression for massive numbers of samples (n) and features (p) is
an important problem. We propose a Bayesian approach for scaling up Gaussian
process (GP) regression to big n and p settings using random compression. The
proposed compressed GP is particularly motivated by the setting in which features
can be projected to a low-dimensional manifold with minimal loss of information
about the response. Conditionally on a random compression matrix and a smoothness
parameter, the posterior and posterior predictive distributions are available
analytically. Running the analysis in parallel for many random compression matrices
and smoothness parameters, model averaging is used to combine the results.
The algorithm can be implemented rapidly even in very big n and p problems, has
strong theoretical justification, and is found to yield state of the art predictive performance.

Keywords: 

Big data, Compressed regression , Gaussian process , Gaussian random projection , Large p, large n , Manifold regression

Manuscript: 

PDF icon 2013-08.pdf