Re: KMeans: expensiveness of large vectors

2014-07-29 Thread Xiangrui Meng
a great thing. > > Out of curiosity, how did communication work before torrent? Did everything > have to go back to the master / driver first? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/KMeans-expensiveness-of-large-vec

Re: KMeans: expensiveness of large vectors

2014-07-29 Thread durin
Development is really rapid here, that's a great thing. Out of curiosity, how did communication work before torrent? Did everything have to go back to the master / driver first? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/KMeans-expensiveness-of-

Re: KMeans: expensiveness of large vectors

2014-07-28 Thread Xiangrui Meng
in a reasonable time. I guess using torrent helps a lot in this > case. > > > Best regards, > Simon > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/KMeans-expensiveness-of-large-vectors-tp10614p10833.html > Sent from

Re: KMeans: expensiveness of large vectors

2014-07-28 Thread durin
: http://apache-spark-user-list.1001560.n3.nabble.com/KMeans-expensiveness-of-large-vectors-tp10614p10833.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: KMeans: expensiveness of large vectors

2014-07-28 Thread Xiangrui Meng
ns, broadcasting overhead will get very large, as the centers > are still in a dense representation. > Do you know of any way to improve performance then? > > > Best regards, > Simon > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3

Re: KMeans: expensiveness of large vectors

2014-07-28 Thread durin
en? Best regards, Simon -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/KMeans-expensiveness-of-large-vectors-tp10614p10804.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: KMeans: expensiveness of large vectors

2014-07-27 Thread Xiangrui Meng
ore expensive than having many of these vectors? > A pointer to the exact part of the source would be fantastic, but even a > general explanation would help me. > > > Best regards, > Simon > > > > -- > View this message in context: > http://apache-spark-user-list.1

KMeans: expensiveness of large vectors

2014-07-24 Thread durin
nsive than having many of these vectors? A pointer to the exact part of the source would be fantastic, but even a general explanation would help me. Best regards, Simon -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/KMeans-expensiveness-of-large-ve