Abstract
This article proposes a split-merge Markov chain algorithm to address the problem of inefficient sampling for conjugate Dirichlet process mixture models. Traditional Markov chain Monte Carlo methods for Bayesian mixture models, such as Gibbs sampling, can become trapped in isolated modes corresponding to an inappropriate clustering of data points. This article describes a Metropolis-Hastings procedure that can escape such local modes by splitting or merging mixture components. Our algorithm employs a new technique in which an appropriate proposal for splitting or merging components is obtained by using a restricted Gibbs sampling scan. We demonstrate empirically that our method outperforms the Gibbs sampler in situations where two or more components are similar in structure.
Keywords
Gibbs samplingMarkov chain Monte CarloMetropolis–Hastings algorithmHierarchical Dirichlet processDirichlet distributionRejection samplingDirichlet processMarkov chainSlice samplingComputer scienceAlgorithmMonte Carlo methodMerge (version control)Hybrid Monte CarloMathematicsBayesian probabilityArtificial intelligenceLatent Dirichlet allocationMachine learningStatisticsTopic model
Affiliated Institutions
Related Publications
Gibbs Sampling for Bayesian Non-Conjugate and Hierarchical Models by Using Auxiliary Variables
Summary We demonstrate the use of auxiliary (or latent) variables for sampling non-standard densities which arise in the context of the Bayesian analysis of non-conjugate and hi...
Understanding the Metropolis-Hastings Algorithm
Abstract We provide a detailed, introductory exposition of the Metropolis-Hastings algorithm, a powerful Markov chain method to simulate multivariate distributions. A simple, in...
Riemann Manifold Langevin and Hamiltonian Monte Carlo Methods
Summary The paper proposes Metropolis adjusted Langevin and Hamiltonian Monte Carlo sampling methods defined on the Riemann manifold to resolve the shortcomings of existing Mont...
Probabilistic Inference Using Markov Chain Monte Carlo Methods
Probabilistic inference is an attractive approach to uncertain reasoning and empirical learning in artificial intelligence. Computational difficulties arise, however, because pr...
The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo
Hamiltonian Monte Carlo (HMC) is a Markov chain Monte Carlo (MCMC) algorithm that avoids the random walk behavior and sensitivity to correlated parameters that plague many MCMC ...
Publication Info
- Year
- 2004
- Type
- article
- Volume
- 13
- Issue
- 1
- Pages
- 158-182
- Citations
- 464
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
464
OpenAlex
Cite This
Sonia Jain,
Radford M. Neal
(2004).
A Split-Merge Markov chain Monte Carlo Procedure for the Dirichlet Process Mixture Model.
Journal of Computational and Graphical Statistics
, 13
(1)
, 158-182.
https://doi.org/10.1198/1061860043001
Identifiers
- DOI
- 10.1198/1061860043001