A Bayesian Hierarchical Topographic Clustering Method Motivated by the Self-Organizing Map

Abstract:

Kohonen's self-organizing map (SOM) has been widely applied as a clustering method for a variety of reasons, especially including its automatic vector quantization and topographic preservation properties. Inspired by the topographic preservation properties of the SOM, this paper considers a clustering method based on Bayesian hierarchical models. Stationary Gaussian processes on a latent grid in $\mathbb{Z}^2$ serve as prior distributions for latent data prototypes and provide the core self-organization. To accommodate high-dimensional data, we model within-cluster data  structure using principal component approximations for covariance matrices. Using available distribution theory on special manifolds, the heterogeneous parts of the covariance matrices have Bingham-von Mises-Fisher distributions as their component-conditional posterior distributions, providing important computational tractability. Latent labeling variables bridge between the data and prototype models, and are approximately projected onto the latent grid, thereby providing the topographic nature of the clustering.

 A posterior (conditional on the data for all latent variables and parameters) risk is defined for clusterings that accounts for both data partitioning and topographic preservation. Markov Chain Monte Carlo (MCMC) methods are employed for sampling the posterior and an approximately optimal clustering rule is defined as an empirical (or approximate version of the) optimizer of the risk function. Numerical studies on simulated and real data compare the performance of the new method to that of other clustering algorithms. Convergence of the numerical algorithm is studied. Connections between the proposed method and Markov Random Fields, kernel smoothing, and the original SOM are also explored.