combine rdds?
Hi, How could I combine rdds? I would like to combine two RDDs if the count in an RDD is not above some threshold. Thanks, Josh
Re: combine rdds?
this requires evaluation of the rdd to do the count. val x: RDD[X] = ... val y: RDD[X] = ... x.cache val z = if(x.count < thres) x.union(y) else x On Oct 27, 2014 7:51 PM, "Josh J" wrote: Hi, How could I combine rdds? I would like to combine two RDDs if the count in an RDD is n