Overcoming weakly identifiable mixture models with more exchangeable data

Long Nguyen, University of Michigan

Friday, March 6, 2020 - 3:30pm

Mixture models are a powerful tool for representing complex forms of probability densities, but also for interpreting the latent heterogeneity of within data populations. Parameter estimation in mixture models can be very inefficient, however, when the parameters are
only weakly identifiable. This is the situation where the parameters are perfectly identifiable, but the Fisher matrix may be singular, resulting in extremely slow learning rates of the posterior distribution. It remains open if a sophisticated prior construction, or an objective Bayes prior specification, or a clever parameterization technique is available to overcome such inefficiency. On the other hand, more data will help in a big way, provided one may collect more exchangeable measurements from the same latent subpopulations, and working with a mixture of product distributions instead.
In this talk I will describe posterior contraction behavior of de Finetti's mixing measure that arise from such mixture models for exchangeable sequences of observations.  The roles of the number of sequences and the sequence lengths will be carefully examined,
where it will be shown that posterior contraction of the mixing measure occurs at the parametric rate even when the original mixture model is weakly identifiable or not identifiable in the first place.  The heart of this result is a family of inverse bounds for mixtures
of product distributions that is achieved via a harmonic analysis of such latent structure models. These inverse bounds are applicable to broad classes of probability kernels composing the mixture model of product distributions for both continuous and discrete domain.
Examples of interest include the case the probability kernel is not or only weakly identifiable, the case where the kernel is itself a mixture distribution as in hierarchical models, and the case the kernel may not have a density with respect to a dominating measure on an abstract domain such as the Dirichlet processes.

This work is joint with Yun Wei (University of Michigan).

Seminars generally take place in 116 Old Chemistry Building on Fridays from 3:30 - 4:30 pm. For additional information contact: karen.whitesell@duke.edu or phone 919-684-8029. Sorry, but we do not have reprints available. Please feel free to contact the authors by email for follow-up information, articles, etc. Reception following seminar in 203B Old Chemistry.

Old Chemistry 116

Location Info