Soft Cluster Ensembles

Abstract:

Cluster Ensembles is a framework for combining multiple partitionings obtained from separate clustering runs into a final consensus clustering. This framework has attracted much interest recently because of its numerous practical applications, and a variety of approaches including Graph Partitioning, Maximum Likelihood, Genetic algorithms, and Voting-Merging have been proposed. The vast majority of these approaches accept hard clusterings as input. There are, however, many clustering algorithms such as EM and fuzzy c-means that naturally output soft partitionings of data, and forcibly hardening these partitions before obtaining a consensus potentially involves loss of valuable information. In this chapter we propose several consensus algorithms that accept soft clusterings and experiment over many real-life datasets to show, both conceptually and empirically, that using soft clusterings as input does offer significant advantages, especially when dealing with vertically partitioned data.