Bayesian recursive variable selection

Authors: 
Li Ma
Duke University

Feb 28 2012

In this work we introduce a new model space prior for Bayesian variable selection in linear regression. This prior is designed based on a recursive constructive procedure that randomly generates models by including variables in a stagewise fashion. We provide a recipe for carrying out Bayesian variable selection and model averaging using this prior, and show that it possesses several desirable features. In particular, it is conjugate—the posterior is the same stepwise procedure. Moreover, the posterior parameters can be computed analytically through a sequence of recursive computation. This property is particularly desirable in high dimensional settings as it provides an alternative approach for exploring the posterior model space without resorting to Markov Chain Monte Carlo (MCMC), whose convergence behavior is often hard to guarantee and difficult to evaluate in such cases. In addition, the prior also allows flexible ways to incorporate structural features of the model space such as the dependence structure among the predictors. In particular, we illuatrate how the prior can be specified to take into account model space redundancy arising from strong correlation among potential predictors.

Manuscript: 

PDF icon 2012-04.pdf

BibTeX Citation: 

@TechReport{Ma2012,
      Author = "Li Ma",
       Title = "Bayesian recursive variable selection", 
        Year = 2012,
 Institution = "Duke University Department of Statistical Science",
        Type = "Discussion Paper",
      Number = "2012-04",
}