Senior Quantitative Researcher in Statistical Arbitrage Trading
Bayesian Tree Models
This dissertation presents the statistical framework of Bayesian analysis of tree models with various applications. Prior specification for such models and development of algorithms for sampling from the posterior distributions are both challenging problems. This thesis addresses each of these issues and extends the Bayesian tree model in several ways, including data resampling (Dirichlet process prior), random threshold in the splitting rules, and autoregressive processes for modeling nonlinear structure in time series data. We also demonstrate various numerical techniques to reduce computational burden. This thesis is divided into two parts. The first part, including the first three chapters, mainly describes the general framework of our Bayesian tree model. Chapte 1 introduces the formal definition of binary tree models. Several key aspects, including tree structure, splitting rules and leaf node distributions, are discussed, setting the foundation for the discussion of prior specification and posterior exploration. Chapter 2 discusses the prior specification for tree models in detail. A Pinball prior for the tree generating process is defined; this allows for the combination of an explicit specification of a distribution for both the tree size and the tree shape. Both the data-dependent and data-independent prior for splitting rules are discussed. Comparisons are made with existing prior specifications. Chapter 3 develops an efficient method for simulation from the posterior tree model space. The core computational innovations involve a novel Metropolis--Hastings method that can dramatically improve the convergence and mixing properties of MCMC methods for Bayesian tree analysis. Existing MCMC methods simulate Bayesian tree models using very local MCMC moves, proposing only small changes. Our new Metropolis--Hastings move makes large changes in the tree, but is at the same time local in that it leaves unchanged the partition of observations into leaf nodes. The second part of this thesis gives several examples. Chapter 4 presents a synthetic data example, illustrating basic proposals and restructure proposal in detail. By exploring this simple example, we illustrate the convergence of our MCMC method. Chapter 5 provides a more complicated example. We present exploratory tools to diagnose the convergence problem, make comparisons with existing MCMC methods, and introduce an importance sampling method to reduce the computational burden in assessing prediction validity. The remaining chapters present extensions of Bayesian tree models. Chapter 6 presents a resampling method inspired by a study in proteomics. A Dirichlet process prior for resampling is introduced and discussed. Chapter 7 introduces the idea and methods of random thresholds in tree models, leading to a novel class of ``smooth threshold'' trees. Chapter 8 develops an autoregressive tree model, aiming to model nonlinear structure in time series data, with some illuminating examples. Appendix A describes the implementation of Bayesian tree models and demonstrates the usage of C++ code, developed as part of this research.