Answering my own question here. This may not be efficient, but this is
what I came up with:
rdd1.coalesce(N).glom.zip(rdd2.coalesce(N).glom).map { case(x,y) = x++y}
On Wed, Mar 26, 2014 at 11:11 AM, Walrus theCat walrusthe...@gmail.comwrote:
Hi,
I want to do something like this:
rdd3 = rdd1.coalesce(N).partitions.zip(rdd2.coalesce(N).partitions)
I realize the above will get me something like
Array[(partition,partition)].
I hope you see what I'm going for here -- any tips on how to accomplish
this?
Thanks