this requires evaluation of the rdd to do the count. val x: RDD[X] = ... val y: RDD[X] = ... x.cache val z = if(x.count < thres) x.union(y) else x On Oct 27, 2014 7:51 PM, "Josh J" <joshjd...@gmail.com> wrote:
Hi, How could I combine rdds? I would like to combine two RDDs if the count in an RDD is not above some threshold. Thanks, Josh
