Sponsor
Statistical Science
Recent theoretical and computational developments on infinite Bayesian mixtures
Friday, September 4,
-
Speaker(s): Filippo Ascolani
Mixture models driven by a Dirichlet process are among the most widely used tools in Bayesian nonparametric statistics, yet fundamental questions about both their inferential behaviour and their scalability to large datasets remain open. In this talk I present recent progress on both fronts.
On the theoretical side, I study the asymptotic behaviour of the posterior on the mixing measure when the data are truly generated by a finite mixture. The posterior turns out to be adaptive to the unknown number of components: the mass it places on the superfluous atoms of the stick-breaking representation vanishes at a near-parametric rate, yielding a nearly optimal contraction rate for the mixing measure in Wasserstein distance. Underlying this is a striking phase transition: approximating the mixing measure beyond the parametric scale requires a number of components that grows logarithmically in the sample size. This also shapes the clustering behaviour: the total number of clusters grows logarithmically, mirroring the prior, while the fraction of observations falling outside the leading clusters shrinks polynomially fast. These results translate directly into guarantees for truncation-based approximations, clarifying how many components are needed to recover the density, the mixing measure, and the clustering of the exact posterior.
On the computational side, I introduce a simple non-reversible sampling scheme for Bayesian mixtures, applicable whether the number of components is fixed or random. The method substantially outperforms classical reversible samplers in many regimes of interest-particularly during convergence and when mixture components overlap appreciably. I will show that its asymptotic variance can never be much worse than that of the standard sampler, and present a scaling-limit analysis suggesting that it can reduce convergence time from quadratic to linear in the number of observations. Time permitting, I will explain why the structure of mixture models makes them an especially natural setting for non-reversible discrete samplers.
On the theoretical side, I study the asymptotic behaviour of the posterior on the mixing measure when the data are truly generated by a finite mixture. The posterior turns out to be adaptive to the unknown number of components: the mass it places on the superfluous atoms of the stick-breaking representation vanishes at a near-parametric rate, yielding a nearly optimal contraction rate for the mixing measure in Wasserstein distance. Underlying this is a striking phase transition: approximating the mixing measure beyond the parametric scale requires a number of components that grows logarithmically in the sample size. This also shapes the clustering behaviour: the total number of clusters grows logarithmically, mirroring the prior, while the fraction of observations falling outside the leading clusters shrinks polynomially fast. These results translate directly into guarantees for truncation-based approximations, clarifying how many components are needed to recover the density, the mixing measure, and the clustering of the exact posterior.
On the computational side, I introduce a simple non-reversible sampling scheme for Bayesian mixtures, applicable whether the number of components is fixed or random. The method substantially outperforms classical reversible samplers in many regimes of interest-particularly during convergence and when mixture components overlap appreciably. I will show that its asymptotic variance can never be much worse than that of the standard sampler, and present a scaling-limit analysis suggesting that it can reduce convergence time from quadratic to linear in the number of observations. Time permitting, I will explain why the structure of mixture models makes them an especially natural setting for non-reversible discrete samplers.
