What is the feature dimension? I saw you used 100 partitions. How many cores does your cluster have? -Xiangrui
On Tue, Oct 14, 2014 at 1:51 PM, Ray <[email protected]> wrote: > Hi guys, > > An interesting thing, for the input dataset which has 1.5 million vectors, > if set the KMeans's k_value = 100 or k_value = 50, it hangs as mentioned > above. However, if decrease k_value = 10, the same error still appears in > the log but the application finished successfully, without observable > hanging. > > Hopefully this provides more information. > > Thanks. > > Ray > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Spark-KMeans-hangs-at-reduceByKey-collectAsMap-tp16413p16417.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
