Dirichlet Process Gaussian Mixture Models

Excellent articles on the topic:
Collapsed Gibbs Sampling for Dirichlet Process Gaussian Mixture Models

Infinite Mixture Models with Nonparametric Bayes and the Dirichlet Process
- To be honest, not a big fan of the above article as I think the metaphor is slightly deviating and I find it rather distracting… well whatever, still a good introduction! Especially the Recap section comparing each generative process side by side!

In the following trying to take some notes as I walk through the Python code of Gibbs sampling for DPGMM.

1. Initialisation

• 2-dimensional Gaussians
• 50 samples
  -> Same input data as Gaussian Mixture Model Ellipsoids from Scikit-learn example
  

2. Learning GMM, DPGMM using Scikit examples

# Fit a mixture of gaussians with EM using five componentsgmm = mixture.GMM(n_components=5, covariance_type='full')gmm.fit(X)# Fit a dirichlet process mixture of gaussians using five componentsdpgmm = mixture.DPGMM(n_components=5, covariance_type='full')dpgmm.fit(X)

3. Learn DPGMM using Gibbs Sampling

dpmm = DPMM(n_components=-1) # -1, 1, 2, 5# n_components is the number of initial clusters (at random, TODO k-means init)# -1 means that we initialize with 1 cluster per pointdpmm.fit_collapsed_Gibbs(X)

3.1 Create DPMM3.2 Create Conjugate Prior Gausssian3.3 Initialise DPMM with data3.4 Initialise Prior Gaussian with data