Approximated Bayesian Inference for Massive Streaming Data

Authors: 
Rajarshi Guhaniyogi, Rebecca M. Willett, David B. Dunson
Duke University

Sep 23 2013

Extracting meaningful information out of massive streaming data is a significant challenge due to the high dimensionality of the inference problem and limits on available computational power and memory. While Bayesian models often convey significant inferential advantages, standard computational algorithms relying on Markov chain Monte Carlo are infeasible to apply. This motivates online variational approximations, which are increasingly popular but have no theoretical guarantees. We propose a novel framework based on online learning theory for assessing the performance of a broad class of online variational approximations. These analyses show that after T streaming observations, a new online variational Bayes approximation yields losses which are within a factor of O(T(-1/2)) of those for an uncomputable batch variational approximation. The new algorithm is compared with competitors using several examples.

Keywords: 

functional regret, MCMC, online learning, streaming data, variational Bayes

Manuscript: 

PDF icon 2013-07.pdf