Spectral Chinese Restaurant Processes: Nonparametric Clustering Based on Similarities
- Abstract: We introduce a new nonparametric clustering model which combines the recently proposed distance-dependent Chinese restaurant process (dd-CRP) and non-linear, spectral methods for dimensionality reduction. Our model retains the ability of nonparametric methods to learn the number of clusters from data. At the same time it addresses two key limitations of nonparametric Bayesian methods: modeling data that are not exchangeable and have many correlated features. Spectral methods use the similarity between documents to map them into a low-dimensional spectral space where we then compare several clustering methods. Our experiments on handwritten digits and text documents show that nonparametric methods such as the CRP or dd-CRP can perform as well as or better than k-means and also recover the true number of clusters. We improve the performance of the dd-CRP in spectral space by incorporating the original similarity matrix in its prior. This simple modification results in better performance than all other methods we compared to. We offer a new formulation and first experimental evaluation of a general Gibbs sampler for mixture modeling with distance-dependent CR Ps?.
Download Paper
Bibtex
@INPROCEEDINGS{SocherMaasManning2011:spectralCRP,
author = {Richard Socher and Andrew Maas and Christopher D. Manning},
title = {Spectral Chinese Restaurant Processes: Nonparametric Clustering Based on Similarities},
booktitle = {Fourteenth International Conference on Artificial Intelligence and Statistics (AISTATS)},
year = {2011}
}
Comments
For remarks, critical comments or other thoughts on the paper.
Hi Cheng,
The idea of the sd-CRP is to combine the original similarities in its prior and the information in the reduced dimensional space in the likelihood. So it is related and using the spectral space. I hope that helps.
Best,
Richarde
Hi, socher, i have something not to understand in this paper. As far as i understand, sd-CRP you proposed is developed on the Blei's DDCRP(we call it the original ddcrp)incorporating the similarity of documents.dd-crp in your paper refers to the original ddcrp using the distance between points in spectral space. As shown in the above figure, your sd-crp and dd-crp are different branches. However, in your paper, you said that "sd-crp is modified version of the dd-crp for clustering in spectral space". sd-crp is not related with spectral space. Why did you present that? it is my confuse. Regards. Cheng