Re: K Nearest Neighbours

2015-07-18 Thread Gylfi
? Does this make any sense? :) Regards, Gylfi. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/K-Nearest-Neighbours-tp23759p23899.html Sent from the Apache Spark User List mailing list archive at Nabble.com

K Nearest Neighbours

2015-07-10 Thread Carsten Schnober
Hi, I have the following problem, which is a kind of special case of k nearest neighbours. I have an Array of Vectors (v1) and an RDD[(Long, Vector)] of pairs of vectors with indexes (v2). The array v1 easily fits into a single node's memory (~100 entries), but v2 is very large (millions