[
https://issues.apache.org/jira/browse/SPARK-5405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Derrick Burns updated SPARK-5405:
-
Description:
The MLLIB clusterer works well for low (200) dimensional data. However,
performance is linear with the number of dimensions. So, for practical
purposes, it is not very useful for high dimensional data.
Depending on the data type, one can embed the high dimensional data into lower
dimensional spaces in a distance-preserving way. The Spark clusterer should
support such embedding.
An example implementation that supports high dimensional data is here:
https://github.com/derrickburns/generalized-kmeans-clustering
was:
The MLLIB clusterer works well for low (200) dimensional data. However,
performance is linear with the number of dimensions. So, for practical
purposes, it is not very useful for high dimensional data.
Depending on the data type, one can embed the high dimensional data into lower
dimensional spaces in a distance-preserving way. The Spark clusterer should
support such embedding.
Spark clusterer should support high dimensional data
Key: SPARK-5405
URL: https://issues.apache.org/jira/browse/SPARK-5405
Project: Spark
Issue Type: New Feature
Components: MLlib
Affects Versions: 1.2.0
Reporter: Derrick Burns
Original Estimate: 504h
Remaining Estimate: 504h
The MLLIB clusterer works well for low (200) dimensional data. However,
performance is linear with the number of dimensions. So, for practical
purposes, it is not very useful for high dimensional data.
Depending on the data type, one can embed the high dimensional data into
lower dimensional spaces in a distance-preserving way. The Spark clusterer
should support such embedding.
An example implementation that supports high dimensional data is here:
https://github.com/derrickburns/generalized-kmeans-clustering
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
