Posterior contraction in sparse Bayesian factor models for massive covariance matrices

Authors: 
Debdeep Pati, Anirban Bhattacharya, Natesh S. Pillai, and David B. Dunson
Duke University, Duke University, Harvard University, Duke University

Mar 21 2012

Sparse Bayesian factor models are routinely implemented for parsimonious dependence modeling and dimensionality reduction in high-dimensional applications. We provide theoretical understanding of such Bayesian procedures in terms of posterior convergence rates in inferring high-dimensional covariance matrices where the dimension can be potentially larger than the sample size.
Under relevant sparsity assumptions on the true covariance matrix, we show that commonly-used point mass mixture priors on the factor loadings lead to consistent estimation in the operator norm even when $p \gg n$. One of our major contributions is to develop a new class of continuous shrinkage priors and provide insights into their concentration around sparse vectors. Using such priors for the factor loadings, we obtain the same rate as obtained with point mass mixture priors. To obtain the convergence rates, we construct test functions to separate points in the space of high-dimensional covariance matrices using insights from random matrix theory; the tools developed may be of independent interest.

Keywords: 

Bayesian estimation, Covariance matrix, Factor model, Rate of convergence, Shrinkage, Sparsity}

Manuscript: 

PDF icon 2012-05.pdf