there for almost 1 hour.
I guess I can only go with random initialization in KMeans.
Thanks again for your help.
Ray
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-KMeans-hangs-at-reduceByKey-collectAsMap-tp16413p16530.html
Sent from the Apache Spark User List
:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-KMeans-hangs-at-reduceByKey-collectAsMap-tp16413.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr
observable
hanging.
Hopefully this provides more information.
Thanks.
Ray
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-KMeans-hangs-at-reduceByKey-collectAsMap-tp16413p16417.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-KMeans-hangs-at-reduceByKey-collectAsMap-tp16413p16417.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr
.1001560.n3.nabble.com/Spark-KMeans-hangs-at-reduceByKey-collectAsMap-tp16413p16428.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
:58:03 PM
Subject: Re: Spark KMeans hangs at reduceByKey / collectAsMap
Hi Xiangrui,
The input dataset has 1.5 million sparse vectors. Each sparse vector has a
dimension(cardinality) of 9153 and has less than 15 nonzero elements.
Yes, if I set num-executors = 200, from the hadoop cluster
be an active stage with an incomplete progress bar in
the UI. Am I wrong?
Thanks, Burak!
Ray
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-KMeans-hangs-at-reduceByKey-collectAsMap-tp16413p16438.html
Sent from the Apache Spark User List mailing
-
From: Ray ray-w...@outlook.com
To: u...@spark.incubator.apache.org
Sent: Tuesday, October 14, 2014 2:58:03 PM
Subject: Re: Spark KMeans hangs at reduceByKey / collectAsMap
Hi Xiangrui,
The input dataset has 1.5 million sparse vectors. Each sparse vector has a
dimension(cardinality) of 9153
, October 14, 2014 2:58:03 PM
Subject: Re: Spark KMeans hangs at reduceByKey / collectAsMap
Hi Xiangrui,
The input dataset has 1.5 million sparse vectors. Each sparse vector has a
dimension(cardinality) of 9153 and has less than 15 nonzero elements.
Yes, if I set num-executors = 200, from
, it just finished quickly~~
In your test on mnis8m, did you use KMeans++ as initialization mode? How
long it takes?
Thanks again for your help.
Ray
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-KMeans-hangs-at-reduceByKey-collectAsMap-tp16413p16450
. This time, it just finished quickly~~
In your test on mnis8m, did you use KMeans++ as initialization mode? How
long it takes?
Thanks again for your help.
Ray
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-KMeans-hangs-at-reduceByKey
11 matches
Mail list logo