[
https://issues.apache.org/jira/browse/SPARK-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14136601#comment-14136601
]
Yu Ishikawa commented on SPARK-2429:
------------------------------------
Hi [~rnowling],
I have read the papers which you referred.
The complexity of Divisive Hievarchical K-Means by Lamrous, et al. seems to be
at least greater than O(n^2). Because it uses Silhouette score. In my opinion,
Its first part is difficult to be scalable.
[http://en.wikipedia.org/wiki/Silhouette_(clustering)]
So I try to implement a hierarchical clustering with a bisecting algorithm as
Freeman did. And then benchmark it and share the result with you. Please give
me some days.
thanks
> Hierarchical Implementation of KMeans
> -------------------------------------
>
> Key: SPARK-2429
> URL: https://issues.apache.org/jira/browse/SPARK-2429
> Project: Spark
> Issue Type: New Feature
> Components: MLlib
> Reporter: RJ Nowling
> Priority: Minor
>
> Hierarchical clustering algorithms are widely used and would make a nice
> addition to MLlib. Clustering algorithms are useful for determining
> relationships between clusters as well as offering faster assignment.
> Discussion on the dev list suggested the following possible approaches:
> * Top down, recursive application of KMeans
> * Reuse DecisionTree implementation with different objective function
> * Hierarchical SVD
> It was also suggested that support for distance metrics other than Euclidean
> such as negative dot or cosine are necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
